feat: Complete LLMX v0.1.0 - Rebrand from Codex with LiteLLM Integration
This release represents a comprehensive transformation of the codebase from Codex to LLMX, enhanced with LiteLLM integration to support 100+ LLM providers through a unified API. ## Major Changes ### Phase 1: Repository & Infrastructure Setup - Established new repository structure and branching strategy - Created comprehensive project documentation (CLAUDE.md, LITELLM-SETUP.md) - Set up development environment and tooling configuration ### Phase 2: Rust Workspace Transformation - Renamed all Rust crates from `codex-*` to `llmx-*` (30+ crates) - Updated package names, binary names, and workspace members - Renamed core modules: codex.rs → llmx.rs, codex_delegate.rs → llmx_delegate.rs - Updated all internal references, imports, and type names - Renamed directories: codex-rs/ → llmx-rs/, codex-backend-openapi-models/ → llmx-backend-openapi-models/ - Fixed all Rust compilation errors after mass rename ### Phase 3: LiteLLM Integration - Integrated LiteLLM for multi-provider LLM support (Anthropic, OpenAI, Azure, Google AI, AWS Bedrock, etc.) - Implemented OpenAI-compatible Chat Completions API support - Added model family detection and provider-specific handling - Updated authentication to support LiteLLM API keys - Renamed environment variables: OPENAI_BASE_URL → LLMX_BASE_URL - Added LLMX_API_KEY for unified authentication - Enhanced error handling for Chat Completions API responses - Implemented fallback mechanisms between Responses API and Chat Completions API ### Phase 4: TypeScript/Node.js Components - Renamed npm package: @codex/codex-cli → @valknar/llmx - Updated TypeScript SDK to use new LLMX APIs and endpoints - Fixed all TypeScript compilation and linting errors - Updated SDK tests to support both API backends - Enhanced mock server to handle multiple API formats - Updated build scripts for cross-platform packaging ### Phase 5: Configuration & Documentation - Updated all configuration files to use LLMX naming - Rewrote README and documentation for LLMX branding - Updated config paths: ~/.codex/ → ~/.llmx/ - Added comprehensive LiteLLM setup guide - Updated all user-facing strings and help text - Created release plan and migration documentation ### Phase 6: Testing & Validation - Fixed all Rust tests for new naming scheme - Updated snapshot tests in TUI (36 frame files) - Fixed authentication storage tests - Updated Chat Completions payload and SSE tests - Fixed SDK tests for new API endpoints - Ensured compatibility with Claude Sonnet 4.5 model - Fixed test environment variables (LLMX_API_KEY, LLMX_BASE_URL) ### Phase 7: Build & Release Pipeline - Updated GitHub Actions workflows for LLMX binary names - Fixed rust-release.yml to reference llmx-rs/ instead of codex-rs/ - Updated CI/CD pipelines for new package names - Made Apple code signing optional in release workflow - Enhanced npm packaging resilience for partial platform builds - Added Windows sandbox support to workspace - Updated dotslash configuration for new binary names ### Phase 8: Final Polish - Renamed all assets (.github images, labels, templates) - Updated VSCode and DevContainer configurations - Fixed all clippy warnings and formatting issues - Applied cargo fmt and prettier formatting across codebase - Updated issue templates and pull request templates - Fixed all remaining UI text references ## Technical Details **Breaking Changes:** - Binary name changed from `codex` to `llmx` - Config directory changed from `~/.codex/` to `~/.llmx/` - Environment variables renamed (CODEX_* → LLMX_*) - npm package renamed to `@valknar/llmx` **New Features:** - Support for 100+ LLM providers via LiteLLM - Unified authentication with LLMX_API_KEY - Enhanced model provider detection and handling - Improved error handling and fallback mechanisms **Files Changed:** - 578 files modified across Rust, TypeScript, and documentation - 30+ Rust crates renamed and updated - Complete rebrand of UI, CLI, and documentation - All tests updated and passing **Dependencies:** - Updated Cargo.lock with new package names - Updated npm dependencies in llmx-cli - Enhanced OpenAPI models for LLMX backend This release establishes LLMX as a standalone project with comprehensive LiteLLM integration, maintaining full backward compatibility with existing functionality while opening support for a wide ecosystem of LLM providers. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Sebastian Krüger <support@pivoine.art>
This commit is contained in:
Sebastian Krüger
363
llmx-rs/core/tests/chat_completions_payload.rs
Normal file
363
llmx-rs/core/tests/chat_completions_payload.rs
Normal file
@@ -0,0 +1,363 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use core_test_support::load_default_config_for_test;
|
||||
use futures::StreamExt;
|
||||
use llmx_app_server_protocol::AuthMode;
|
||||
use llmx_core::ContentItem;
|
||||
use llmx_core::LocalShellAction;
|
||||
use llmx_core::LocalShellExecAction;
|
||||
use llmx_core::LocalShellStatus;
|
||||
use llmx_core::ModelClient;
|
||||
use llmx_core::ModelProviderInfo;
|
||||
use llmx_core::Prompt;
|
||||
use llmx_core::ResponseItem;
|
||||
use llmx_core::WireApi;
|
||||
use llmx_core::spawn::LLMX_SANDBOX_NETWORK_DISABLED_ENV_VAR;
|
||||
use llmx_otel::otel_event_manager::OtelEventManager;
|
||||
use llmx_protocol::ConversationId;
|
||||
use llmx_protocol::models::ReasoningItemContent;
|
||||
use serde_json::Value;
|
||||
use tempfile::TempDir;
|
||||
use wiremock::Mock;
|
||||
use wiremock::MockServer;
|
||||
use wiremock::ResponseTemplate;
|
||||
use wiremock::matchers::method;
|
||||
use wiremock::matchers::path;
|
||||
|
||||
fn network_disabled() -> bool {
|
||||
std::env::var(LLMX_SANDBOX_NETWORK_DISABLED_ENV_VAR).is_ok()
|
||||
}
|
||||
|
||||
async fn run_request(input: Vec<ResponseItem>) -> Value {
|
||||
let server = MockServer::start().await;
|
||||
|
||||
let template = ResponseTemplate::new(200)
|
||||
.insert_header("content-type", "text/event-stream")
|
||||
.set_body_raw(
|
||||
"data: {\"choices\":[{\"delta\":{}}]}\n\ndata: [DONE]\n\n",
|
||||
"text/event-stream",
|
||||
);
|
||||
|
||||
Mock::given(method("POST"))
|
||||
.and(path("/v1/chat/completions"))
|
||||
.respond_with(template)
|
||||
.expect(1)
|
||||
.mount(&server)
|
||||
.await;
|
||||
|
||||
let provider = ModelProviderInfo {
|
||||
name: "mock".into(),
|
||||
base_url: Some(format!("{}/v1", server.uri())),
|
||||
env_key: None,
|
||||
env_key_instructions: None,
|
||||
experimental_bearer_token: None,
|
||||
wire_api: WireApi::Chat,
|
||||
query_params: None,
|
||||
http_headers: None,
|
||||
env_http_headers: None,
|
||||
request_max_retries: Some(0),
|
||||
stream_max_retries: Some(0),
|
||||
stream_idle_timeout_ms: Some(5_000),
|
||||
requires_openai_auth: false,
|
||||
};
|
||||
|
||||
let llmx_home = match TempDir::new() {
|
||||
Ok(dir) => dir,
|
||||
Err(e) => panic!("failed to create TempDir: {e}"),
|
||||
};
|
||||
let mut config = load_default_config_for_test(&llmx_home);
|
||||
config.model_provider_id = provider.name.clone();
|
||||
config.model_provider = provider.clone();
|
||||
config.show_raw_agent_reasoning = true;
|
||||
let effort = config.model_reasoning_effort;
|
||||
let summary = config.model_reasoning_summary;
|
||||
let config = Arc::new(config);
|
||||
|
||||
let conversation_id = ConversationId::new();
|
||||
|
||||
let otel_event_manager = OtelEventManager::new(
|
||||
conversation_id,
|
||||
config.model.as_str(),
|
||||
config.model_family.slug.as_str(),
|
||||
None,
|
||||
Some("test@test.com".to_string()),
|
||||
Some(AuthMode::ChatGPT),
|
||||
false,
|
||||
"test".to_string(),
|
||||
);
|
||||
|
||||
let client = ModelClient::new(
|
||||
Arc::clone(&config),
|
||||
None,
|
||||
otel_event_manager,
|
||||
provider,
|
||||
effort,
|
||||
summary,
|
||||
conversation_id,
|
||||
llmx_protocol::protocol::SessionSource::Exec,
|
||||
);
|
||||
|
||||
let mut prompt = Prompt::default();
|
||||
prompt.input = input;
|
||||
|
||||
let mut stream = match client.stream(&prompt).await {
|
||||
Ok(s) => s,
|
||||
Err(e) => panic!("stream chat failed: {e}"),
|
||||
};
|
||||
while let Some(event) = stream.next().await {
|
||||
if let Err(e) = event {
|
||||
panic!("stream event error: {e}");
|
||||
}
|
||||
}
|
||||
|
||||
let requests = match server.received_requests().await {
|
||||
Some(reqs) => reqs,
|
||||
None => panic!("request not made"),
|
||||
};
|
||||
match requests[0].body_json() {
|
||||
Ok(v) => v,
|
||||
Err(e) => panic!("invalid json body: {e}"),
|
||||
}
|
||||
}
|
||||
|
||||
fn user_message(text: &str) -> ResponseItem {
|
||||
ResponseItem::Message {
|
||||
id: None,
|
||||
role: "user".to_string(),
|
||||
content: vec![ContentItem::InputText {
|
||||
text: text.to_string(),
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
fn assistant_message(text: &str) -> ResponseItem {
|
||||
ResponseItem::Message {
|
||||
id: None,
|
||||
role: "assistant".to_string(),
|
||||
content: vec![ContentItem::OutputText {
|
||||
text: text.to_string(),
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
fn reasoning_item(text: &str) -> ResponseItem {
|
||||
ResponseItem::Reasoning {
|
||||
id: String::new(),
|
||||
summary: Vec::new(),
|
||||
content: Some(vec![ReasoningItemContent::ReasoningText {
|
||||
text: text.to_string(),
|
||||
}]),
|
||||
encrypted_content: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn function_call() -> ResponseItem {
|
||||
ResponseItem::FunctionCall {
|
||||
id: None,
|
||||
name: "f".to_string(),
|
||||
arguments: "{}".to_string(),
|
||||
call_id: "c1".to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
fn local_shell_call() -> ResponseItem {
|
||||
ResponseItem::LocalShellCall {
|
||||
id: Some("id1".to_string()),
|
||||
call_id: None,
|
||||
status: LocalShellStatus::InProgress,
|
||||
action: LocalShellAction::Exec(LocalShellExecAction {
|
||||
command: vec!["echo".to_string()],
|
||||
timeout_ms: Some(1_000),
|
||||
working_directory: None,
|
||||
env: None,
|
||||
user: None,
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
fn messages_from(body: &Value) -> Vec<Value> {
|
||||
match body["messages"].as_array() {
|
||||
Some(arr) => arr.clone(),
|
||||
None => panic!("messages array missing"),
|
||||
}
|
||||
}
|
||||
|
||||
fn first_assistant(messages: &[Value]) -> &Value {
|
||||
match messages.iter().find(|msg| msg["role"] == "assistant") {
|
||||
Some(v) => v,
|
||||
None => panic!("assistant message not present"),
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn omits_reasoning_when_none_present() {
|
||||
if network_disabled() {
|
||||
println!(
|
||||
"Skipping test because it cannot execute when network is disabled in an LLMX sandbox."
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
let body = run_request(vec![user_message("u1"), assistant_message("a1")]).await;
|
||||
let messages = messages_from(&body);
|
||||
let assistant = first_assistant(&messages);
|
||||
|
||||
assert_eq!(assistant["content"], Value::String("a1".into()));
|
||||
assert!(assistant.get("reasoning").is_none());
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn attaches_reasoning_to_previous_assistant() {
|
||||
if network_disabled() {
|
||||
println!(
|
||||
"Skipping test because it cannot execute when network is disabled in an LLMX sandbox."
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
let body = run_request(vec![
|
||||
user_message("u1"),
|
||||
assistant_message("a1"),
|
||||
reasoning_item("rA"),
|
||||
])
|
||||
.await;
|
||||
let messages = messages_from(&body);
|
||||
let assistant = first_assistant(&messages);
|
||||
|
||||
assert_eq!(assistant["content"], Value::String("a1".into()));
|
||||
assert_eq!(assistant["reasoning"], Value::String("rA".into()));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn attaches_reasoning_to_function_call_anchor() {
|
||||
if network_disabled() {
|
||||
println!(
|
||||
"Skipping test because it cannot execute when network is disabled in an LLMX sandbox."
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
let body = run_request(vec![
|
||||
user_message("u1"),
|
||||
reasoning_item("rFunc"),
|
||||
function_call(),
|
||||
])
|
||||
.await;
|
||||
let messages = messages_from(&body);
|
||||
let assistant = first_assistant(&messages);
|
||||
|
||||
assert_eq!(assistant["reasoning"], Value::String("rFunc".into()));
|
||||
let tool_calls = match assistant["tool_calls"].as_array() {
|
||||
Some(arr) => arr,
|
||||
None => panic!("tool call list missing"),
|
||||
};
|
||||
assert_eq!(tool_calls.len(), 1);
|
||||
assert_eq!(tool_calls[0]["type"], Value::String("function".into()));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn attaches_reasoning_to_local_shell_call() {
|
||||
if network_disabled() {
|
||||
println!(
|
||||
"Skipping test because it cannot execute when network is disabled in an LLMX sandbox."
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
let body = run_request(vec![
|
||||
user_message("u1"),
|
||||
reasoning_item("rShell"),
|
||||
local_shell_call(),
|
||||
])
|
||||
.await;
|
||||
let messages = messages_from(&body);
|
||||
let assistant = first_assistant(&messages);
|
||||
|
||||
assert_eq!(assistant["reasoning"], Value::String("rShell".into()));
|
||||
assert_eq!(
|
||||
assistant["tool_calls"][0]["type"],
|
||||
Value::String("local_shell_call".into())
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn drops_reasoning_when_last_role_is_user() {
|
||||
if network_disabled() {
|
||||
println!(
|
||||
"Skipping test because it cannot execute when network is disabled in an LLMX sandbox."
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
let body = run_request(vec![
|
||||
assistant_message("aPrev"),
|
||||
reasoning_item("rHist"),
|
||||
user_message("uNew"),
|
||||
])
|
||||
.await;
|
||||
let messages = messages_from(&body);
|
||||
assert!(messages.iter().all(|msg| msg.get("reasoning").is_none()));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn ignores_reasoning_before_last_user() {
|
||||
if network_disabled() {
|
||||
println!(
|
||||
"Skipping test because it cannot execute when network is disabled in an LLMX sandbox."
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
let body = run_request(vec![
|
||||
user_message("u1"),
|
||||
assistant_message("a1"),
|
||||
user_message("u2"),
|
||||
reasoning_item("rAfterU1"),
|
||||
])
|
||||
.await;
|
||||
let messages = messages_from(&body);
|
||||
assert!(messages.iter().all(|msg| msg.get("reasoning").is_none()));
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn skips_empty_reasoning_segments() {
|
||||
if network_disabled() {
|
||||
println!(
|
||||
"Skipping test because it cannot execute when network is disabled in an LLMX sandbox."
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
let body = run_request(vec![
|
||||
user_message("u1"),
|
||||
assistant_message("a1"),
|
||||
reasoning_item(""),
|
||||
reasoning_item(" "),
|
||||
])
|
||||
.await;
|
||||
let messages = messages_from(&body);
|
||||
let assistant = first_assistant(&messages);
|
||||
assert!(assistant.get("reasoning").is_none());
|
||||
}
|
||||
|
||||
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
|
||||
async fn suppresses_duplicate_assistant_messages() {
|
||||
if network_disabled() {
|
||||
println!(
|
||||
"Skipping test because it cannot execute when network is disabled in an LLMX sandbox."
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
let body = run_request(vec![assistant_message("dup"), assistant_message("dup")]).await;
|
||||
let messages = messages_from(&body);
|
||||
let assistant_messages: Vec<_> = messages
|
||||
.iter()
|
||||
.filter(|msg| msg["role"] == "assistant")
|
||||
.collect();
|
||||
assert_eq!(assistant_messages.len(), 1);
|
||||
assert_eq!(
|
||||
assistant_messages[0]["content"],
|
||||
Value::String("dup".into())
|
||||
);
|
||||
}
|
||||
Reference in New Issue
Block a user