fix: disable drop_params to preserve streaming metadata in LiteLLM

- Set drop_params: false in litellm_settings
- Set modify_params: false in litellm_settings
- Set drop_params: false in default_litellm_params
- Commented out LITELLM_DROP_PARAMS env var
- Removed --drop_params command flag

These settings were stripping critical streaming parameters causing
vLLM streaming responses to collapse into empty deltas
This commit is contained in:
2025-11-21 18:46:33 +01:00
parent c58b5d36ba
commit 01a345979b
2 changed files with 5 additions and 6 deletions

View File

@@ -100,7 +100,7 @@ services:
ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}
LITELLM_MASTER_KEY: ${AI_LITELLM_API_KEY}
DATABASE_URL: postgresql://${AI_DB_USER}:${AI_DB_PASSWORD}@ai_postgres:5432/litellm
LITELLM_DROP_PARAMS: 'true'
# LITELLM_DROP_PARAMS: 'true' # DISABLED: Was breaking streaming
NO_DOCS: 'true'
NO_REDOC: 'true'
# Performance optimizations
@@ -115,8 +115,7 @@ services:
'--host',
'0.0.0.0',
'--port',
'4000',
'--drop_params'
'4000'
]
depends_on:
- ai_postgres