Files
runpod-ai-orchestrator/services/vllm/config_llama.yaml
Sebastian Krüger c969d10eaf
All checks were successful
Build and Push RunPod Docker Image / build-and-push (push) Successful in 14s
feat: increase Llama context to 32K with 95% VRAM
For larger input requirements, increased max-model-len from 20480 to 32768.
BGE remains available but cannot run concurrently at this VRAM level.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-30 22:37:11 +01:00

238 B