feat: Complete LLMX v0.1.0 - Rebrand from Codex with LiteLLM Integration

This release represents a comprehensive transformation of the codebase from Codex to LLMX, enhanced with LiteLLM integration to support 100+ LLM providers through a unified API. ## Major Changes ### Phase 1: Repository & Infrastructure Setup - Established new repository structure and branching strategy - Created comprehensive project documentation (CLAUDE.md, LITELLM-SETUP.md) - Set up development environment and tooling configuration ### Phase 2: Rust Workspace Transformation - Renamed all Rust crates from `codex-*` to `llmx-*` (30+ crates) - Updated package names, binary names, and workspace members - Renamed core modules: codex.rs → llmx.rs, codex_delegate.rs → llmx_delegate.rs - Updated all internal references, imports, and type names - Renamed directories: codex-rs/ → llmx-rs/, codex-backend-openapi-models/ → llmx-backend-openapi-models/ - Fixed all Rust compilation errors after mass rename ### Phase 3: LiteLLM Integration - Integrated LiteLLM for multi-provider LLM support (Anthropic, OpenAI, Azure, Google AI, AWS Bedrock, etc.) - Implemented OpenAI-compatible Chat Completions API support - Added model family detection and provider-specific handling - Updated authentication to support LiteLLM API keys - Renamed environment variables: OPENAI_BASE_URL → LLMX_BASE_URL - Added LLMX_API_KEY for unified authentication - Enhanced error handling for Chat Completions API responses - Implemented fallback mechanisms between Responses API and Chat Completions API ### Phase 4: TypeScript/Node.js Components - Renamed npm package: @codex/codex-cli → @valknar/llmx - Updated TypeScript SDK to use new LLMX APIs and endpoints - Fixed all TypeScript compilation and linting errors - Updated SDK tests to support both API backends - Enhanced mock server to handle multiple API formats - Updated build scripts for cross-platform packaging ### Phase 5: Configuration & Documentation - Updated all configuration files to use LLMX naming - Rewrote README and documentation for LLMX branding - Updated config paths: ~/.codex/ → ~/.llmx/ - Added comprehensive LiteLLM setup guide - Updated all user-facing strings and help text - Created release plan and migration documentation ### Phase 6: Testing & Validation - Fixed all Rust tests for new naming scheme - Updated snapshot tests in TUI (36 frame files) - Fixed authentication storage tests - Updated Chat Completions payload and SSE tests - Fixed SDK tests for new API endpoints - Ensured compatibility with Claude Sonnet 4.5 model - Fixed test environment variables (LLMX_API_KEY, LLMX_BASE_URL) ### Phase 7: Build & Release Pipeline - Updated GitHub Actions workflows for LLMX binary names - Fixed rust-release.yml to reference llmx-rs/ instead of codex-rs/ - Updated CI/CD pipelines for new package names - Made Apple code signing optional in release workflow - Enhanced npm packaging resilience for partial platform builds - Added Windows sandbox support to workspace - Updated dotslash configuration for new binary names ### Phase 8: Final Polish - Renamed all assets (.github images, labels, templates) - Updated VSCode and DevContainer configurations - Fixed all clippy warnings and formatting issues - Applied cargo fmt and prettier formatting across codebase - Updated issue templates and pull request templates - Fixed all remaining UI text references ## Technical Details **Breaking Changes:** - Binary name changed from `codex` to `llmx` - Config directory changed from `~/.codex/` to `~/.llmx/` - Environment variables renamed (CODEX_* → LLMX_*) - npm package renamed to `@valknar/llmx` **New Features:** - Support for 100+ LLM providers via LiteLLM - Unified authentication with LLMX_API_KEY - Enhanced model provider detection and handling - Improved error handling and fallback mechanisms **Files Changed:** - 578 files modified across Rust, TypeScript, and documentation - 30+ Rust crates renamed and updated - Complete rebrand of UI, CLI, and documentation - All tests updated and passing **Dependencies:** - Updated Cargo.lock with new package names - Updated npm dependencies in llmx-cli - Enhanced OpenAPI models for LLMX backend This release establishes LLMX as a standalone project with comprehensive LiteLLM integration, maintaining full backward compatibility with existing functionality while opening support for a wide ecosystem of LLM providers. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Sebastian Krüger <support@pivoine.art>
2025-11-12 20:40:44 +01:00
parent 052b052832
commit 3c7efc58c8
1248 changed files with 10085 additions and 9580 deletions
--- a/llmx-rs/app-server/tests/suite/send_message.rs
+++ b/llmx-rs/app-server/tests/suite/send_message.rs
@@ -0,0 +1,391 @@
+use anyhow::Result;
+use app_test_support::McpProcess;
+use app_test_support::create_final_assistant_message_sse_response;
+use app_test_support::create_mock_chat_completions_server;
+use app_test_support::to_response;
+use llmx_app_server_protocol::AddConversationListenerParams;
+use llmx_app_server_protocol::AddConversationSubscriptionResponse;
+use llmx_app_server_protocol::InputItem;
+use llmx_app_server_protocol::JSONRPCNotification;
+use llmx_app_server_protocol::JSONRPCResponse;
+use llmx_app_server_protocol::NewConversationParams;
+use llmx_app_server_protocol::NewConversationResponse;
+use llmx_app_server_protocol::RequestId;
+use llmx_app_server_protocol::SendUserMessageParams;
+use llmx_app_server_protocol::SendUserMessageResponse;
+use llmx_protocol::ConversationId;
+use llmx_protocol::models::ContentItem;
+use llmx_protocol::models::ResponseItem;
+use llmx_protocol::protocol::RawResponseItemEvent;
+use pretty_assertions::assert_eq;
+use std::path::Path;
+use tempfile::TempDir;
+use tokio::time::timeout;
+
+const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
+
+#[tokio::test]
+async fn test_send_message_success() -> Result<()> {
+    // Spin up a mock completions server that immediately ends the LLMX turn.
+    // Two LLMX turns hit the mock model (session start + send-user-message). Provide two SSE responses.
+    let responses = vec![
+        create_final_assistant_message_sse_response("Done")?,
+        create_final_assistant_message_sse_response("Done")?,
+    ];
+    let server = create_mock_chat_completions_server(responses).await;
+
+    // Create a temporary LLMX home with config pointing at the mock server.
+    let llmx_home = TempDir::new()?;
+    create_config_toml(llmx_home.path(), &server.uri())?;
+
+    // Start MCP server process and initialize.
+    let mut mcp = McpProcess::new(llmx_home.path()).await?;
+    timeout(DEFAULT_READ_TIMEOUT, mcp.initialize()).await??;
+
+    // Start a conversation using the new wire API.
+    let new_conv_id = mcp
+        .send_new_conversation_request(NewConversationParams {
+            ..Default::default()
+        })
+        .await?;
+    let new_conv_resp: JSONRPCResponse = timeout(
+        DEFAULT_READ_TIMEOUT,
+        mcp.read_stream_until_response_message(RequestId::Integer(new_conv_id)),
+    )
+    .await??;
+    let NewConversationResponse {
+        conversation_id, ..
+    } = to_response::<_>(new_conv_resp)?;
+
+    // 2) addConversationListener
+    let add_listener_id = mcp
+        .send_add_conversation_listener_request(AddConversationListenerParams {
+            conversation_id,
+            experimental_raw_events: false,
+        })
+        .await?;
+    let add_listener_resp: JSONRPCResponse = timeout(
+        DEFAULT_READ_TIMEOUT,
+        mcp.read_stream_until_response_message(RequestId::Integer(add_listener_id)),
+    )
+    .await??;
+    let AddConversationSubscriptionResponse { subscription_id: _ } =
+        to_response::<_>(add_listener_resp)?;
+
+    // Now exercise sendUserMessage twice.
+    send_message("Hello", conversation_id, &mut mcp).await?;
+    send_message("Hello again", conversation_id, &mut mcp).await?;
+    Ok(())
+}
+
+#[expect(clippy::expect_used)]
+async fn send_message(
+    message: &str,
+    conversation_id: ConversationId,
+    mcp: &mut McpProcess,
+) -> Result<()> {
+    // Now exercise sendUserMessage.
+    let send_id = mcp
+        .send_send_user_message_request(SendUserMessageParams {
+            conversation_id,
+            items: vec![InputItem::Text {
+                text: message.to_string(),
+            }],
+        })
+        .await?;
+
+    let response: JSONRPCResponse = timeout(
+        DEFAULT_READ_TIMEOUT,
+        mcp.read_stream_until_response_message(RequestId::Integer(send_id)),
+    )
+    .await??;
+
+    let _ok: SendUserMessageResponse = to_response::<SendUserMessageResponse>(response)?;
+
+    // Verify the task_finished notification is received.
+    // Note this also ensures that the final request to the server was made.
+    let task_finished_notification: JSONRPCNotification = timeout(
+        DEFAULT_READ_TIMEOUT,
+        mcp.read_stream_until_notification_message("llmx/event/task_complete"),
+    )
+    .await??;
+    let serde_json::Value::Object(map) = task_finished_notification
+        .params
+        .expect("notification should have params")
+    else {
+        panic!("task_finished_notification should have params");
+    };
+    assert_eq!(
+        map.get("conversationId")
+            .expect("should have conversationId"),
+        &serde_json::Value::String(conversation_id.to_string())
+    );
+
+    let raw_attempt = tokio::time::timeout(
+        std::time::Duration::from_millis(200),
+        mcp.read_stream_until_notification_message("llmx/event/raw_response_item"),
+    )
+    .await;
+    assert!(
+        raw_attempt.is_err(),
+        "unexpected raw item notification when not opted in"
+    );
+    Ok(())
+}
+
+#[tokio::test]
+async fn test_send_message_raw_notifications_opt_in() -> Result<()> {
+    let responses = vec![create_final_assistant_message_sse_response("Done")?];
+    let server = create_mock_chat_completions_server(responses).await;
+
+    let llmx_home = TempDir::new()?;
+    create_config_toml(llmx_home.path(), &server.uri())?;
+
+    let mut mcp = McpProcess::new(llmx_home.path()).await?;
+    timeout(DEFAULT_READ_TIMEOUT, mcp.initialize()).await??;
+
+    let new_conv_id = mcp
+        .send_new_conversation_request(NewConversationParams {
+            developer_instructions: Some("Use the test harness tools.".to_string()),
+            ..Default::default()
+        })
+        .await?;
+    let new_conv_resp: JSONRPCResponse = timeout(
+        DEFAULT_READ_TIMEOUT,
+        mcp.read_stream_until_response_message(RequestId::Integer(new_conv_id)),
+    )
+    .await??;
+    let NewConversationResponse {
+        conversation_id, ..
+    } = to_response::<_>(new_conv_resp)?;
+
+    let add_listener_id = mcp
+        .send_add_conversation_listener_request(AddConversationListenerParams {
+            conversation_id,
+            experimental_raw_events: true,
+        })
+        .await?;
+    let add_listener_resp: JSONRPCResponse = timeout(
+        DEFAULT_READ_TIMEOUT,
+        mcp.read_stream_until_response_message(RequestId::Integer(add_listener_id)),
+    )
+    .await??;
+    let AddConversationSubscriptionResponse { subscription_id: _ } =
+        to_response::<_>(add_listener_resp)?;
+
+    let send_id = mcp
+        .send_send_user_message_request(SendUserMessageParams {
+            conversation_id,
+            items: vec![InputItem::Text {
+                text: "Hello".to_string(),
+            }],
+        })
+        .await?;
+
+    let developer = read_raw_response_item(&mut mcp, conversation_id).await;
+    assert_developer_message(&developer, "Use the test harness tools.");
+
+    let instructions = read_raw_response_item(&mut mcp, conversation_id).await;
+    assert_instructions_message(&instructions);
+
+    let environment = read_raw_response_item(&mut mcp, conversation_id).await;
+    assert_environment_message(&environment);
+
+    let response: JSONRPCResponse = timeout(
+        DEFAULT_READ_TIMEOUT,
+        mcp.read_stream_until_response_message(RequestId::Integer(send_id)),
+    )
+    .await??;
+    let _ok: SendUserMessageResponse = to_response::<SendUserMessageResponse>(response)?;
+
+    let user_message = read_raw_response_item(&mut mcp, conversation_id).await;
+    assert_user_message(&user_message, "Hello");
+
+    let assistant_message = read_raw_response_item(&mut mcp, conversation_id).await;
+    assert_assistant_message(&assistant_message, "Done");
+
+    let _ = tokio::time::timeout(
+        std::time::Duration::from_millis(250),
+        mcp.read_stream_until_notification_message("llmx/event/task_complete"),
+    )
+    .await;
+
+    Ok(())
+}
+
+#[tokio::test]
+async fn test_send_message_session_not_found() -> Result<()> {
+    // Start MCP without creating an LLMX session
+    let llmx_home = TempDir::new()?;
+    let mut mcp = McpProcess::new(llmx_home.path()).await?;
+    timeout(DEFAULT_READ_TIMEOUT, mcp.initialize()).await??;
+
+    let unknown = ConversationId::new();
+    let req_id = mcp
+        .send_send_user_message_request(SendUserMessageParams {
+            conversation_id: unknown,
+            items: vec![InputItem::Text {
+                text: "ping".to_string(),
+            }],
+        })
+        .await?;
+
+    // Expect an error response for unknown conversation.
+    let err = timeout(
+        DEFAULT_READ_TIMEOUT,
+        mcp.read_stream_until_error_message(RequestId::Integer(req_id)),
+    )
+    .await??;
+    assert_eq!(err.id, RequestId::Integer(req_id));
+    Ok(())
+}
+
+// ---------------------------------------------------------------------------
+// Helpers
+// ---------------------------------------------------------------------------
+
+fn create_config_toml(llmx_home: &Path, server_uri: &str) -> std::io::Result<()> {
+    let config_toml = llmx_home.join("config.toml");
+    std::fs::write(
+        config_toml,
+        format!(
+            r#"
+model = "mock-model"
+approval_policy = "never"
+sandbox_mode = "danger-full-access"
+
+model_provider = "mock_provider"
+
+[model_providers.mock_provider]
+name = "Mock provider for test"
+base_url = "{server_uri}/v1"
+wire_api = "chat"
+request_max_retries = 0
+stream_max_retries = 0
+"#
+        ),
+    )
+}
+
+#[expect(clippy::expect_used)]
+async fn read_raw_response_item(
+    mcp: &mut McpProcess,
+    conversation_id: ConversationId,
+) -> ResponseItem {
+    let raw_notification: JSONRPCNotification = timeout(
+        DEFAULT_READ_TIMEOUT,
+        mcp.read_stream_until_notification_message("llmx/event/raw_response_item"),
+    )
+    .await
+    .expect("llmx/event/raw_response_item notification timeout")
+    .expect("llmx/event/raw_response_item notification resp");
+
+    let serde_json::Value::Object(params) = raw_notification
+        .params
+        .expect("llmx/event/raw_response_item should have params")
+    else {
+        panic!("llmx/event/raw_response_item should have params");
+    };
+
+    let conversation_id_value = params
+        .get("conversationId")
+        .and_then(|value| value.as_str())
+        .expect("raw response item should include conversationId");
+
+    assert_eq!(
+        conversation_id_value,
+        conversation_id.to_string(),
+        "raw response item conversation mismatch"
+    );
+
+    let msg_value = params
+        .get("msg")
+        .cloned()
+        .expect("raw response item should include msg payload");
+
+    let event: RawResponseItemEvent =
+        serde_json::from_value(msg_value).expect("deserialize raw response item");
+    event.item
+}
+
+fn assert_instructions_message(item: &ResponseItem) {
+    match item {
+        ResponseItem::Message { role, content, .. } => {
+            assert_eq!(role, "user");
+            let texts = content_texts(content);
+            let is_instructions = texts
+                .iter()
+                .any(|text| text.starts_with("# AGENTS.md instructions for "));
+            assert!(
+                is_instructions,
+                "expected instructions message, got {texts:?}"
+            );
+        }
+        other => panic!("expected instructions message, got {other:?}"),
+    }
+}
+
+fn assert_developer_message(item: &ResponseItem, expected_text: &str) {
+    match item {
+        ResponseItem::Message { role, content, .. } => {
+            assert_eq!(role, "developer");
+            let texts = content_texts(content);
+            assert_eq!(
+                texts,
+                vec![expected_text],
+                "expected developer instructions message, got {texts:?}"
+            );
+        }
+        other => panic!("expected developer instructions message, got {other:?}"),
+    }
+}
+
+fn assert_environment_message(item: &ResponseItem) {
+    match item {
+        ResponseItem::Message { role, content, .. } => {
+            assert_eq!(role, "user");
+            let texts = content_texts(content);
+            assert!(
+                texts
+                    .iter()
+                    .any(|text| text.contains("<environment_context>")),
+                "expected environment context message, got {texts:?}"
+            );
+        }
+        other => panic!("expected environment message, got {other:?}"),
+    }
+}
+
+fn assert_user_message(item: &ResponseItem, expected_text: &str) {
+    match item {
+        ResponseItem::Message { role, content, .. } => {
+            assert_eq!(role, "user");
+            let texts = content_texts(content);
+            assert_eq!(texts, vec![expected_text]);
+        }
+        other => panic!("expected user message, got {other:?}"),
+    }
+}
+
+fn assert_assistant_message(item: &ResponseItem, expected_text: &str) {
+    match item {
+        ResponseItem::Message { role, content, .. } => {
+            assert_eq!(role, "assistant");
+            let texts = content_texts(content);
+            assert_eq!(texts, vec![expected_text]);
+        }
+        other => panic!("expected assistant message, got {other:?}"),
+    }
+}
+
+fn content_texts(content: &[ContentItem]) -> Vec<&str> {
+    content
+        .iter()
+        .filter_map(|item| match item {
+            ContentItem::InputText { text } | ContentItem::OutputText { text } => {
+                Some(text.as_str())
+            }
+            _ => None,
+        })
+        .collect()
+}