valknar/runpod
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
18cd87fbd1a107d6d337bbc2f1f0c71d33b3fa86
runpod/models/vllm
History
Sebastian Krüger 7f1890517d fix: enable eager execution for proper token streaming in vLLM 
- Set enforce_eager=True to disable CUDA graphs which were batching outputs
- Add disable_log_stats=True for better streaming performance
- This ensures AsyncLLMEngine yields tokens incrementally instead of returning complete response
2025-11-21 18:25:50 +01:00
..
requirements.txt
refactor: clean Docker files and restore standalone model services
2025-11-21 16:17:38 +01:00
server.py
fix: enable eager execution for proper token streaming in vLLM
2025-11-21 18:25:50 +01:00