Files
runpod-ai-orchestrator/services/vllm
Sebastian Krüger f8694653d0
All checks were successful
Build and Push RunPod Docker Image / build-and-push (push) Successful in 13s
fix: adjust VRAM for 24K context based on actual usage
Based on error output, model uses ~17.5GB (not 15GB estimated).
- Llama: 85% VRAM for 24576 context (3GB KV cache)
- BGE: 6% VRAM (reduced to fit)
- Total: 91%

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-30 22:46:34 +01:00
..
2025-11-26 17:15:08 +01:00