valknar/llmx
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Phase 3: LiteLLM Integration

Browse Source 
- Added LiteLLM as built-in model provider in model_provider_info.rs:
  - Default base_url: http://localhost:4000/v1 (configurable via LITELLM_BASE_URL)
  - Uses Chat wire API (OpenAI-compatible)
  - Requires LITELLM_API_KEY environment variable
  - No OpenAI auth required (simple bearer token)
  - Positioned as first provider in list

- Updated default models to use LiteLLM format:
  - Changed from "gpt-5-codex" to "anthropic/claude-sonnet-4-20250514"
  - Updated all default model constants (OPENAI_DEFAULT_MODEL, etc.)
  - Uses provider/model format compatible with LiteLLM

- Provider configuration:
  - Supports base_url override via environment variable
  - Includes helpful env_key_instructions pointing to LiteLLM docs
  - Uses standard retry/timeout defaults

This makes LLMX work out-of-the-box with LiteLLM proxy, supporting
multiple providers (Anthropic, OpenAI, etc.) through a single interface.

🤖 Generated with Claude Code
This commit is contained in:
Sebastian Krüger
2025-11-11 14:33:00 +01:00
parent cb8d941adf
commit 27909b7495
2 changed files with 30 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
llmx-rs/core/src/config/mod.rs
View File

@@ -58,12 +58,13 @@ pub mod edit;
pub mod profile;
pub mod types;
// Default models for LLMX using LiteLLM format (provider/model)
#[cfg(target_os = "windows")]
pub const OPENAI_DEFAULT_MODEL: &str = "gpt-5";
pub const OPENAI_DEFAULT_MODEL: &str = "anthropic/claude-sonnet-4-20250514";
#[cfg(not(target_os = "windows"))]
pub const OPENAI_DEFAULT_MODEL: &str = "gpt-5-codex";
const OPENAI_DEFAULT_REVIEW_MODEL: &str = "gpt-5-codex";
pub const GPT_5_CODEX_MEDIUM_MODEL: &str = "gpt-5-codex";
pub const OPENAI_DEFAULT_MODEL: &str = "anthropic/claude-sonnet-4-20250514";
const OPENAI_DEFAULT_REVIEW_MODEL: &str = "anthropic/claude-sonnet-4-20250514";
pub const GPT_5_CODEX_MEDIUM_MODEL: &str = "anthropic/claude-sonnet-4-20250514";
/// Maximum number of bytes of the documentation that will be embedded. Larger
/// files are *silently truncated* to this size so we do not take up too much of

28
llmx-rs/core/src/model_provider_info.rs
View File

@@ -267,10 +267,32 @@ pub fn built_in_model_providers() -> HashMap<String, ModelProviderInfo> {
use ModelProviderInfo as P;
// We do not want to be in the business of adjucating which third-party
// providers are bundled with Codex CLI, so we only include the OpenAI and
// open source ("oss") providers by default. Users are encouraged to add to
// `model_providers` in config.toml to add their own providers.
// providers are bundled with LLMX CLI, so we include LiteLLM as the default,
// along with OpenAI and open source ("oss") providers. Users are encouraged
// to add to `model_providers` in config.toml to add their own providers.
[
(
"litellm",
P {
name: "LiteLLM".into(),
// Allow users to override the default LiteLLM endpoint
base_url: std::env::var("LITELLM_BASE_URL")
.ok()
.filter(|v| !v.trim().is_empty())
.or_else(|| Some("http://localhost:4000/v1".into())),
env_key: Some("LITELLM_API_KEY".into()),
env_key_instructions: Some("Set LITELLM_API_KEY to your LiteLLM master key. See https://docs.litellm.ai/ for setup.".into()),
experimental_bearer_token: None,
wire_api: WireApi::Chat,
query_params: None,
http_headers: None,
env_http_headers: None,
request_max_retries: None,
stream_max_retries: None,
stream_idle_timeout_ms: None,
requires_openai_auth: false,
},
),
(
"openai",
P {