424090f2f2
Created comprehensive guide for configuring LLMX with LiteLLM: - Environment variable setup - Configuration file format - Correct usage patterns (separate provider and model) - Common error troubleshooting - Model name format documentation Key points: - Use `-c model_provider=litellm` NOT `litellm:` prefix - Set LITELLM_BASE_URL and LITELLM_API_KEY - LiteLLM provider uses Chat Completions API (wire_api=chat) 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
2.0 KiB
LLMX with LiteLLM Configuration Guide
Quick Start
1. Set Environment Variables
export LITELLM_BASE_URL="https://llm.ai.pivoine.art/v1"
export LITELLM_API_KEY="your-litellm-master-key"
2. Create Configuration File
Create ~/.llmx/config.toml:
model_provider = "litellm"
model = "anthropic/claude-sonnet-4-20250514"
3. Run LLMX
# Use default config
llmx "hello world"
# Override model
llmx -m "openai/gpt-4" "hello world"
# Override provider and model
llmx -c model_provider=litellm -m "anthropic/claude-sonnet-4-20250514" "hello"
Important Notes
DO NOT use provider prefix in model name
❌ Wrong: llmx -m "litellm:anthropic/claude-sonnet-4-20250514"
✅ Correct: llmx -c model_provider=litellm -m "anthropic/claude-sonnet-4-20250514"
LLMX uses separate provider and model parameters, not a combined provider:model syntax.
Provider Selection
The provider determines which API endpoint and format to use:
litellm→ Uses Chat Completions API (/v1/chat/completions)openai→ Uses Responses API (/v1/responses) - NOT compatible with LiteLLM
Model Names
LiteLLM uses provider/model format:
anthropic/claude-sonnet-4-20250514openai/gpt-4openai/gpt-4o
Check your LiteLLM configuration for available models.
Troubleshooting
Error: "prompt_cache_key: Extra inputs are not permitted"
Cause: Using wrong provider (defaults to OpenAI which uses Responses API)
Fix: Add -c model_provider=litellm or set model_provider = "litellm" in config
Error: "Invalid model name passed in model=litellm:..."
Cause: Including provider prefix in model name
Fix: Remove the litellm: prefix, use just the model name
Error: "Model provider litellm not found"
Cause: Using old binary without LiteLLM provider
Fix: Use the newly built binary at llmx-rs/target/release/llmx
Binary Location
Latest binary with LiteLLM support:
/home/valknar/Projects/codex/llmx/llmx-rs/target/release/llmx