feat: Complete LLMX v0.1.0 - Rebrand from Codex with LiteLLM Integration

This release represents a comprehensive transformation of the codebase from Codex to LLMX,
enhanced with LiteLLM integration to support 100+ LLM providers through a unified API.

## Major Changes

### Phase 1: Repository & Infrastructure Setup
- Established new repository structure and branching strategy
- Created comprehensive project documentation (CLAUDE.md, LITELLM-SETUP.md)
- Set up development environment and tooling configuration

### Phase 2: Rust Workspace Transformation
- Renamed all Rust crates from `codex-*` to `llmx-*` (30+ crates)
- Updated package names, binary names, and workspace members
- Renamed core modules: codex.rs → llmx.rs, codex_delegate.rs → llmx_delegate.rs
- Updated all internal references, imports, and type names
- Renamed directories: codex-rs/ → llmx-rs/, codex-backend-openapi-models/ → llmx-backend-openapi-models/
- Fixed all Rust compilation errors after mass rename

### Phase 3: LiteLLM Integration
- Integrated LiteLLM for multi-provider LLM support (Anthropic, OpenAI, Azure, Google AI, AWS Bedrock, etc.)
- Implemented OpenAI-compatible Chat Completions API support
- Added model family detection and provider-specific handling
- Updated authentication to support LiteLLM API keys
- Renamed environment variables: OPENAI_BASE_URL → LLMX_BASE_URL
- Added LLMX_API_KEY for unified authentication
- Enhanced error handling for Chat Completions API responses
- Implemented fallback mechanisms between Responses API and Chat Completions API

### Phase 4: TypeScript/Node.js Components
- Renamed npm package: @codex/codex-cli → @valknar/llmx
- Updated TypeScript SDK to use new LLMX APIs and endpoints
- Fixed all TypeScript compilation and linting errors
- Updated SDK tests to support both API backends
- Enhanced mock server to handle multiple API formats
- Updated build scripts for cross-platform packaging

### Phase 5: Configuration & Documentation
- Updated all configuration files to use LLMX naming
- Rewrote README and documentation for LLMX branding
- Updated config paths: ~/.codex/ → ~/.llmx/
- Added comprehensive LiteLLM setup guide
- Updated all user-facing strings and help text
- Created release plan and migration documentation

### Phase 6: Testing & Validation
- Fixed all Rust tests for new naming scheme
- Updated snapshot tests in TUI (36 frame files)
- Fixed authentication storage tests
- Updated Chat Completions payload and SSE tests
- Fixed SDK tests for new API endpoints
- Ensured compatibility with Claude Sonnet 4.5 model
- Fixed test environment variables (LLMX_API_KEY, LLMX_BASE_URL)

### Phase 7: Build & Release Pipeline
- Updated GitHub Actions workflows for LLMX binary names
- Fixed rust-release.yml to reference llmx-rs/ instead of codex-rs/
- Updated CI/CD pipelines for new package names
- Made Apple code signing optional in release workflow
- Enhanced npm packaging resilience for partial platform builds
- Added Windows sandbox support to workspace
- Updated dotslash configuration for new binary names

### Phase 8: Final Polish
- Renamed all assets (.github images, labels, templates)
- Updated VSCode and DevContainer configurations
- Fixed all clippy warnings and formatting issues
- Applied cargo fmt and prettier formatting across codebase
- Updated issue templates and pull request templates
- Fixed all remaining UI text references

## Technical Details

**Breaking Changes:**
- Binary name changed from `codex` to `llmx`
- Config directory changed from `~/.codex/` to `~/.llmx/`
- Environment variables renamed (CODEX_* → LLMX_*)
- npm package renamed to `@valknar/llmx`

**New Features:**
- Support for 100+ LLM providers via LiteLLM
- Unified authentication with LLMX_API_KEY
- Enhanced model provider detection and handling
- Improved error handling and fallback mechanisms

**Files Changed:**
- 578 files modified across Rust, TypeScript, and documentation
- 30+ Rust crates renamed and updated
- Complete rebrand of UI, CLI, and documentation
- All tests updated and passing

**Dependencies:**
- Updated Cargo.lock with new package names
- Updated npm dependencies in llmx-cli
- Enhanced OpenAPI models for LLMX backend

This release establishes LLMX as a standalone project with comprehensive LiteLLM
integration, maintaining full backward compatibility with existing functionality
while opening support for a wide ecosystem of LLM providers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Sebastian Krüger <support@pivoine.art>

llmx-rs/protocol/src/account.rs

@@ -0,0 +1,20 @@
+use schemars::JsonSchema;
+use serde::Deserialize;
+use serde::Serialize;
+use ts_rs::TS;
+
+#[derive(Serialize, Deserialize, Copy, Clone, Debug, PartialEq, Eq, JsonSchema, TS, Default)]
+#[serde(rename_all = "lowercase")]
+#[ts(rename_all = "lowercase")]
+pub enum PlanType {
+    #[default]
+    Free,
+    Plus,
+    Pro,
+    Team,
+    Business,
+    Enterprise,
+    Edu,
+    #[serde(other)]
+    Unknown,
+}

llmx-rs/protocol/src/approvals.rs

@@ -0,0 +1,63 @@
+use std::collections::HashMap;
+use std::path::PathBuf;
+
+use crate::parse_command::ParsedCommand;
+use crate::protocol::FileChange;
+use schemars::JsonSchema;
+use serde::Deserialize;
+use serde::Serialize;
+use ts_rs::TS;
+
+#[derive(Debug, Clone, Copy, Deserialize, Serialize, PartialEq, Eq, Hash, JsonSchema, TS)]
+#[serde(rename_all = "snake_case")]
+pub enum SandboxRiskLevel {
+    Low,
+    Medium,
+    High,
+}
+
+#[derive(Debug, Clone, Deserialize, Serialize, PartialEq, Eq, JsonSchema, TS)]
+pub struct SandboxCommandAssessment {
+    pub description: String,
+    pub risk_level: SandboxRiskLevel,
+}
+
+impl SandboxRiskLevel {
+    pub fn as_str(&self) -> &'static str {
+        match self {
+            Self::Low => "low",
+            Self::Medium => "medium",
+            Self::High => "high",
+        }
+    }
+}
+
+#[derive(Debug, Clone, Deserialize, Serialize, JsonSchema, TS)]
+pub struct ExecApprovalRequestEvent {
+    /// Identifier for the associated exec call, if available.
+    pub call_id: String,
+    /// The command to be executed.
+    pub command: Vec<String>,
+    /// The command's working directory.
+    pub cwd: PathBuf,
+    /// Optional human-readable reason for the approval (e.g. retry without sandbox).
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub reason: Option<String>,
+    /// Optional model-provided risk assessment describing the blocked command.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub risk: Option<SandboxCommandAssessment>,
+    pub parsed_cmd: Vec<ParsedCommand>,
+}
+
+#[derive(Debug, Clone, Deserialize, Serialize, JsonSchema, TS)]
+pub struct ApplyPatchApprovalRequestEvent {
+    /// Responses API call id for the associated patch apply call, if available.
+    pub call_id: String,
+    pub changes: HashMap<PathBuf, FileChange>,
+    /// Optional explanatory reason (e.g. request for extra write access).
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub reason: Option<String>,
+    /// When set, the agent is asking the user to allow writes under this root for the remainder of the session.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub grant_root: Option<PathBuf>,
+}

llmx-rs/protocol/src/config_types.rs

@@ -0,0 +1,87 @@
+use schemars::JsonSchema;
+use serde::Deserialize;
+use serde::Serialize;
+use strum_macros::Display;
+use strum_macros::EnumIter;
+use ts_rs::TS;
+
+/// See https://platform.openai.com/docs/guides/reasoning?api-mode=responses#get-started-with-reasoning
+#[derive(
+    Debug,
+    Serialize,
+    Deserialize,
+    Default,
+    Clone,
+    Copy,
+    PartialEq,
+    Eq,
+    Display,
+    JsonSchema,
+    TS,
+    EnumIter,
+)]
+#[serde(rename_all = "lowercase")]
+#[strum(serialize_all = "lowercase")]
+pub enum ReasoningEffort {
+    Minimal,
+    Low,
+    #[default]
+    Medium,
+    High,
+}
+
+/// A summary of the reasoning performed by the model. This can be useful for
+/// debugging and understanding the model's reasoning process.
+/// See https://platform.openai.com/docs/guides/reasoning?api-mode=responses#reasoning-summaries
+#[derive(
+    Debug, Serialize, Deserialize, Default, Clone, Copy, PartialEq, Eq, Display, JsonSchema, TS,
+)]
+#[serde(rename_all = "lowercase")]
+#[strum(serialize_all = "lowercase")]
+pub enum ReasoningSummary {
+    #[default]
+    Auto,
+    Concise,
+    Detailed,
+    /// Option to disable reasoning summaries.
+    None,
+}
+
+/// Controls output length/detail on GPT-5 models via the Responses API.
+/// Serialized with lowercase values to match the OpenAI API.
+#[derive(
+    Debug, Serialize, Deserialize, Default, Clone, Copy, PartialEq, Eq, Display, JsonSchema, TS,
+)]
+#[serde(rename_all = "lowercase")]
+#[strum(serialize_all = "lowercase")]
+pub enum Verbosity {
+    Low,
+    #[default]
+    Medium,
+    High,
+}
+
+#[derive(
+    Deserialize, Debug, Clone, Copy, PartialEq, Default, Serialize, Display, JsonSchema, TS,
+)]
+#[serde(rename_all = "kebab-case")]
+#[strum(serialize_all = "kebab-case")]
+pub enum SandboxMode {
+    #[serde(rename = "read-only")]
+    #[default]
+    ReadOnly,
+
+    #[serde(rename = "workspace-write")]
+    WorkspaceWrite,
+
+    #[serde(rename = "danger-full-access")]
+    DangerFullAccess,
+}
+
+#[derive(Debug, Serialize, Deserialize, Clone, Copy, PartialEq, Eq, Display, JsonSchema, TS)]
+#[serde(rename_all = "lowercase")]
+#[strum(serialize_all = "lowercase")]
+pub enum ForcedLoginMethod {
+    Chatgpt,
+    Api,
+}

llmx-rs/protocol/src/conversation_id.rs

@@ -0,0 +1,81 @@
+use std::fmt::Display;
+
+use schemars::JsonSchema;
+use schemars::r#gen::SchemaGenerator;
+use schemars::schema::Schema;
+use serde::Deserialize;
+use serde::Serialize;
+use ts_rs::TS;
+use uuid::Uuid;
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq, TS, Hash)]
+#[ts(type = "string")]
+pub struct ConversationId {
+    uuid: Uuid,
+}
+
+impl ConversationId {
+    pub fn new() -> Self {
+        Self {
+            uuid: Uuid::now_v7(),
+        }
+    }
+
+    pub fn from_string(s: &str) -> Result<Self, uuid::Error> {
+        Ok(Self {
+            uuid: Uuid::parse_str(s)?,
+        })
+    }
+}
+
+impl Default for ConversationId {
+    fn default() -> Self {
+        Self::new()
+    }
+}
+
+impl Display for ConversationId {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        write!(f, "{}", self.uuid)
+    }
+}
+
+impl Serialize for ConversationId {
+    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
+    where
+        S: serde::Serializer,
+    {
+        serializer.collect_str(&self.uuid)
+    }
+}
+
+impl<'de> Deserialize<'de> for ConversationId {
+    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
+    where
+        D: serde::Deserializer<'de>,
+    {
+        let value = String::deserialize(deserializer)?;
+        let uuid = Uuid::parse_str(&value).map_err(serde::de::Error::custom)?;
+        Ok(Self { uuid })
+    }
+}
+
+impl JsonSchema for ConversationId {
+    fn schema_name() -> String {
+        "ConversationId".to_string()
+    }
+
+    fn json_schema(generator: &mut SchemaGenerator) -> Schema {
+        <String>::json_schema(generator)
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    #[test]
+    fn test_conversation_id_default_is_not_zeroes() {
+        let id = ConversationId::default();
+        assert_ne!(id.uuid, Uuid::nil());
+    }
+}

llmx-rs/protocol/src/custom_prompts.rs

@@ -0,0 +1,20 @@
+use schemars::JsonSchema;
+use serde::Deserialize;
+use serde::Serialize;
+use std::path::PathBuf;
+use ts_rs::TS;
+
+/// Base namespace for custom prompt slash commands (without trailing colon).
+/// Example usage forms constructed in code:
+/// - Command token after '/': `"{PROMPTS_CMD_PREFIX}:name"`
+/// - Full slash prefix: `"/{PROMPTS_CMD_PREFIX}:"`
+pub const PROMPTS_CMD_PREFIX: &str = "prompts";
+
+#[derive(Serialize, Deserialize, Debug, Clone, JsonSchema, TS)]
+pub struct CustomPrompt {
+    pub name: String,
+    pub path: PathBuf,
+    pub content: String,
+    pub description: Option<String>,
+    pub argument_hint: Option<String>,
+}

llmx-rs/protocol/src/items.rs

@@ -0,0 +1,159 @@
+use crate::protocol::AgentMessageEvent;
+use crate::protocol::AgentReasoningEvent;
+use crate::protocol::AgentReasoningRawContentEvent;
+use crate::protocol::EventMsg;
+use crate::protocol::UserMessageEvent;
+use crate::protocol::WebSearchEndEvent;
+use crate::user_input::UserInput;
+use schemars::JsonSchema;
+use serde::Deserialize;
+use serde::Serialize;
+use ts_rs::TS;
+
+#[derive(Debug, Clone, Deserialize, Serialize, TS, JsonSchema)]
+pub enum TurnItem {
+    UserMessage(UserMessageItem),
+    AgentMessage(AgentMessageItem),
+    Reasoning(ReasoningItem),
+    WebSearch(WebSearchItem),
+}
+
+#[derive(Debug, Clone, Deserialize, Serialize, TS, JsonSchema)]
+pub struct UserMessageItem {
+    pub id: String,
+    pub content: Vec<UserInput>,
+}
+
+#[derive(Debug, Clone, Deserialize, Serialize, TS, JsonSchema)]
+pub enum AgentMessageContent {
+    Text { text: String },
+}
+
+#[derive(Debug, Clone, Deserialize, Serialize, TS, JsonSchema)]
+pub struct AgentMessageItem {
+    pub id: String,
+    pub content: Vec<AgentMessageContent>,
+}
+
+#[derive(Debug, Clone, Deserialize, Serialize, TS, JsonSchema)]
+pub struct ReasoningItem {
+    pub id: String,
+    pub summary_text: Vec<String>,
+    #[serde(default)]
+    pub raw_content: Vec<String>,
+}
+
+#[derive(Debug, Clone, Deserialize, Serialize, TS, JsonSchema)]
+pub struct WebSearchItem {
+    pub id: String,
+    pub query: String,
+}
+
+impl UserMessageItem {
+    pub fn new(content: &[UserInput]) -> Self {
+        Self {
+            id: uuid::Uuid::new_v4().to_string(),
+            content: content.to_vec(),
+        }
+    }
+
+    pub fn as_legacy_event(&self) -> EventMsg {
+        EventMsg::UserMessage(UserMessageEvent {
+            message: self.message(),
+            images: Some(self.image_urls()),
+        })
+    }
+
+    pub fn message(&self) -> String {
+        self.content
+            .iter()
+            .map(|c| match c {
+                UserInput::Text { text } => text.clone(),
+                _ => String::new(),
+            })
+            .collect::<Vec<String>>()
+            .join("")
+    }
+
+    pub fn image_urls(&self) -> Vec<String> {
+        self.content
+            .iter()
+            .filter_map(|c| match c {
+                UserInput::Image { image_url } => Some(image_url.clone()),
+                _ => None,
+            })
+            .collect()
+    }
+}
+
+impl AgentMessageItem {
+    pub fn new(content: &[AgentMessageContent]) -> Self {
+        Self {
+            id: uuid::Uuid::new_v4().to_string(),
+            content: content.to_vec(),
+        }
+    }
+
+    pub fn as_legacy_events(&self) -> Vec<EventMsg> {
+        self.content
+            .iter()
+            .map(|c| match c {
+                AgentMessageContent::Text { text } => EventMsg::AgentMessage(AgentMessageEvent {
+                    message: text.clone(),
+                }),
+            })
+            .collect()
+    }
+}
+
+impl ReasoningItem {
+    pub fn as_legacy_events(&self, show_raw_agent_reasoning: bool) -> Vec<EventMsg> {
+        let mut events = Vec::new();
+        for summary in &self.summary_text {
+            events.push(EventMsg::AgentReasoning(AgentReasoningEvent {
+                text: summary.clone(),
+            }));
+        }
+
+        if show_raw_agent_reasoning {
+            for entry in &self.raw_content {
+                events.push(EventMsg::AgentReasoningRawContent(
+                    AgentReasoningRawContentEvent {
+                        text: entry.clone(),
+                    },
+                ));
+            }
+        }
+
+        events
+    }
+}
+
+impl WebSearchItem {
+    pub fn as_legacy_event(&self) -> EventMsg {
+        EventMsg::WebSearchEnd(WebSearchEndEvent {
+            call_id: self.id.clone(),
+            query: self.query.clone(),
+        })
+    }
+}
+
+impl TurnItem {
+    pub fn id(&self) -> String {
+        match self {
+            TurnItem::UserMessage(item) => item.id.clone(),
+            TurnItem::AgentMessage(item) => item.id.clone(),
+            TurnItem::Reasoning(item) => item.id.clone(),
+            TurnItem::WebSearch(item) => item.id.clone(),
+        }
+    }
+
+    pub fn as_legacy_events(&self, show_raw_agent_reasoning: bool) -> Vec<EventMsg> {
+        match self {
+            TurnItem::UserMessage(item) => vec![item.as_legacy_event()],
+            TurnItem::AgentMessage(item) => item.as_legacy_events(),
+            TurnItem::WebSearch(item) => vec![item.as_legacy_event()],
+            TurnItem::Reasoning(item) => item.as_legacy_events(show_raw_agent_reasoning),
+        }
+    }
+}

llmx-rs/protocol/src/lib.rs

@@ -0,0 +1,14 @@
+pub mod account;
+mod conversation_id;
+pub use conversation_id::ConversationId;
+pub mod approvals;
+pub mod config_types;
+pub mod custom_prompts;
+pub mod items;
+pub mod message_history;
+pub mod models;
+pub mod num_format;
+pub mod parse_command;
+pub mod plan_tool;
+pub mod protocol;
+pub mod user_input;

llmx-rs/protocol/src/message_history.rs

@@ -0,0 +1,11 @@
+use schemars::JsonSchema;
+use serde::Deserialize;
+use serde::Serialize;
+use ts_rs::TS;
+
+#[derive(Serialize, Deserialize, Debug, Clone, JsonSchema, TS)]
+pub struct HistoryEntry {
+    pub conversation_id: String,
+    pub ts: u64,
+    pub text: String,
+}

llmx-rs/protocol/src/models.rs

@@ -0,0 +1,690 @@
+use std::collections::HashMap;
+
+use base64::Engine;
+use llmx_utils_image::load_and_resize_to_fit;
+use mcp_types::CallToolResult;
+use mcp_types::ContentBlock;
+use serde::Deserialize;
+use serde::Deserializer;
+use serde::Serialize;
+use serde::ser::Serializer;
+use ts_rs::TS;
+
+use crate::user_input::UserInput;
+use llmx_git::GhostCommit;
+use llmx_utils_image::error::ImageProcessingError;
+use schemars::JsonSchema;
+
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, JsonSchema, TS)]
+#[serde(tag = "type", rename_all = "snake_case")]
+pub enum ResponseInputItem {
+    Message {
+        role: String,
+        content: Vec<ContentItem>,
+    },
+    FunctionCallOutput {
+        call_id: String,
+        output: FunctionCallOutputPayload,
+    },
+    McpToolCallOutput {
+        call_id: String,
+        result: Result<CallToolResult, String>,
+    },
+    CustomToolCallOutput {
+        call_id: String,
+        output: String,
+    },
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, JsonSchema, TS)]
+#[serde(tag = "type", rename_all = "snake_case")]
+pub enum ContentItem {
+    InputText { text: String },
+    InputImage { image_url: String },
+    OutputText { text: String },
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, JsonSchema, TS)]
+#[serde(tag = "type", rename_all = "snake_case")]
+pub enum ResponseItem {
+    Message {
+        #[serde(default, skip_serializing)]
+        #[ts(skip)]
+        id: Option<String>,
+        role: String,
+        content: Vec<ContentItem>,
+    },
+    Reasoning {
+        #[serde(default, skip_serializing)]
+        #[ts(skip)]
+        id: String,
+        summary: Vec<ReasoningItemReasoningSummary>,
+        #[serde(default, skip_serializing_if = "should_serialize_reasoning_content")]
+        #[ts(optional)]
+        content: Option<Vec<ReasoningItemContent>>,
+        encrypted_content: Option<String>,
+    },
+    LocalShellCall {
+        /// Set when using the chat completions API.
+        #[serde(default, skip_serializing)]
+        #[ts(skip)]
+        id: Option<String>,
+        /// Set when using the Responses API.
+        call_id: Option<String>,
+        status: LocalShellStatus,
+        action: LocalShellAction,
+    },
+    FunctionCall {
+        #[serde(default, skip_serializing)]
+        #[ts(skip)]
+        id: Option<String>,
+        name: String,
+        // The Responses API returns the function call arguments as a *string* that contains
+        // JSON, not as an already‑parsed object. We keep it as a raw string here and let
+        // Session::handle_function_call parse it into a Value. This exactly matches the
+        // Chat Completions + Responses API behavior.
+        arguments: String,
+        call_id: String,
+    },
+    // NOTE: The input schema for `function_call_output` objects that clients send to the
+    // OpenAI /v1/responses endpoint is NOT the same shape as the objects the server returns on the
+    // SSE stream. When *sending* we must wrap the string output inside an object that includes a
+    // required `success` boolean. To ensure we serialize exactly the expected shape we introduce
+    // a dedicated payload struct and flatten it here.
+    FunctionCallOutput {
+        call_id: String,
+        output: FunctionCallOutputPayload,
+    },
+    CustomToolCall {
+        #[serde(default, skip_serializing)]
+        #[ts(skip)]
+        id: Option<String>,
+        #[serde(default, skip_serializing_if = "Option::is_none")]
+        #[ts(optional)]
+        status: Option<String>,
+
+        call_id: String,
+        name: String,
+        input: String,
+    },
+    CustomToolCallOutput {
+        call_id: String,
+        output: String,
+    },
+    // Emitted by the Responses API when the agent triggers a web search.
+    // Example payload (from SSE `response.output_item.done`):
+    // {
+    //   "id":"ws_...",
+    //   "type":"web_search_call",
+    //   "status":"completed",
+    //   "action": {"type":"search","query":"weather: San Francisco, CA"}
+    // }
+    WebSearchCall {
+        #[serde(default, skip_serializing)]
+        #[ts(skip)]
+        id: Option<String>,
+        #[serde(default, skip_serializing_if = "Option::is_none")]
+        #[ts(optional)]
+        status: Option<String>,
+        action: WebSearchAction,
+    },
+    // Generated by the harness but considered exactly as a model response.
+    GhostSnapshot {
+        ghost_commit: GhostCommit,
+    },
+    #[serde(other)]
+    Other,
+}
+
+fn should_serialize_reasoning_content(content: &Option<Vec<ReasoningItemContent>>) -> bool {
+    match content {
+        Some(content) => !content
+            .iter()
+            .any(|c| matches!(c, ReasoningItemContent::ReasoningText { .. })),
+        None => false,
+    }
+}
+
+fn local_image_error_placeholder(
+    path: &std::path::Path,
+    error: impl std::fmt::Display,
+) -> ContentItem {
+    ContentItem::InputText {
+        text: format!(
+            "LLMX could not read the local image at `{}`: {}",
+            path.display(),
+            error
+        ),
+    }
+}
+
+impl From<ResponseInputItem> for ResponseItem {
+    fn from(item: ResponseInputItem) -> Self {
+        match item {
+            ResponseInputItem::Message { role, content } => Self::Message {
+                role,
+                content,
+                id: None,
+            },
+            ResponseInputItem::FunctionCallOutput { call_id, output } => {
+                Self::FunctionCallOutput { call_id, output }
+            }
+            ResponseInputItem::McpToolCallOutput { call_id, result } => {
+                let output = match result {
+                    Ok(result) => FunctionCallOutputPayload::from(&result),
+                    Err(tool_call_err) => FunctionCallOutputPayload {
+                        content: format!("err: {tool_call_err:?}"),
+                        success: Some(false),
+                        ..Default::default()
+                    },
+                };
+                Self::FunctionCallOutput { call_id, output }
+            }
+            ResponseInputItem::CustomToolCallOutput { call_id, output } => {
+                Self::CustomToolCallOutput { call_id, output }
+            }
+        }
+    }
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, JsonSchema, TS)]
+#[serde(rename_all = "snake_case")]
+pub enum LocalShellStatus {
+    Completed,
+    InProgress,
+    Incomplete,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, JsonSchema, TS)]
+#[serde(tag = "type", rename_all = "snake_case")]
+pub enum LocalShellAction {
+    Exec(LocalShellExecAction),
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, JsonSchema, TS)]
+pub struct LocalShellExecAction {
+    pub command: Vec<String>,
+    pub timeout_ms: Option<u64>,
+    pub working_directory: Option<String>,
+    pub env: Option<HashMap<String, String>>,
+    pub user: Option<String>,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, JsonSchema, TS)]
+#[serde(tag = "type", rename_all = "snake_case")]
+pub enum WebSearchAction {
+    Search {
+        query: String,
+    },
+    #[serde(other)]
+    Other,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, JsonSchema, TS)]
+#[serde(tag = "type", rename_all = "snake_case")]
+pub enum ReasoningItemReasoningSummary {
+    SummaryText { text: String },
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, JsonSchema, TS)]
+#[serde(tag = "type", rename_all = "snake_case")]
+pub enum ReasoningItemContent {
+    ReasoningText { text: String },
+    Text { text: String },
+}
+
+impl From<Vec<UserInput>> for ResponseInputItem {
+    fn from(items: Vec<UserInput>) -> Self {
+        Self::Message {
+            role: "user".to_string(),
+            content: items
+                .into_iter()
+                .map(|c| match c {
+                    UserInput::Text { text } => ContentItem::InputText { text },
+                    UserInput::Image { image_url } => ContentItem::InputImage { image_url },
+                    UserInput::LocalImage { path } => match load_and_resize_to_fit(&path) {
+                        Ok(image) => ContentItem::InputImage {
+                            image_url: image.into_data_url(),
+                        },
+                        Err(err) => {
+                            tracing::warn!("Failed to resize image {}: {}", path.display(), err);
+                            if matches!(&err, ImageProcessingError::Read { .. }) {
+                                local_image_error_placeholder(&path, &err)
+                            } else {
+                                match std::fs::read(&path) {
+                                    Ok(bytes) => {
+                                        let Some(mime_guess) = mime_guess::from_path(&path).first()
+                                        else {
+                                            return local_image_error_placeholder(
+                                                &path,
+                                                "unsupported MIME type (unknown)",
+                                            );
+                                        };
+                                        let mime = mime_guess.essence_str().to_owned();
+                                        if !mime.starts_with("image/") {
+                                            return local_image_error_placeholder(
+                                                &path,
+                                                format!("unsupported MIME type `{mime}`"),
+                                            );
+                                        }
+                                        let encoded =
+                                            base64::engine::general_purpose::STANDARD.encode(bytes);
+                                        ContentItem::InputImage {
+                                            image_url: format!("data:{mime};base64,{encoded}"),
+                                        }
+                                    }
+                                    Err(read_err) => {
+                                        tracing::warn!(
+                                            "Skipping image {} – could not read file: {}",
+                                            path.display(),
+                                            read_err
+                                        );
+                                        local_image_error_placeholder(&path, &read_err)
+                                    }
+                                }
+                            }
+                        }
+                    },
+                })
+                .collect::<Vec<ContentItem>>(),
+        }
+    }
+}
+
+/// If the `name` of a `ResponseItem::FunctionCall` is either `container.exec`
+/// or shell`, the `arguments` field should deserialize to this struct.
+#[derive(Deserialize, Debug, Clone, PartialEq, JsonSchema, TS)]
+pub struct ShellToolCallParams {
+    pub command: Vec<String>,
+    pub workdir: Option<String>,
+
+    /// This is the maximum time in milliseconds that the command is allowed to run.
+    #[serde(alias = "timeout")]
+    pub timeout_ms: Option<u64>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub with_escalated_permissions: Option<bool>,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub justification: Option<String>,
+}
+
+/// Responses API compatible content items that can be returned by a tool call.
+/// This is a subset of ContentItem with the types we support as function call outputs.
+#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, JsonSchema, TS)]
+#[serde(tag = "type", rename_all = "snake_case")]
+pub enum FunctionCallOutputContentItem {
+    // Do not rename, these are serialized and used directly in the responses API.
+    InputText { text: String },
+    // Do not rename, these are serialized and used directly in the responses API.
+    InputImage { image_url: String },
+}
+
+/// The payload we send back to OpenAI when reporting a tool call result.
+///
+/// `content` preserves the historical plain-string payload so downstream
+/// integrations (tests, logging, etc.) can keep treating tool output as
+/// `String`. When an MCP server returns richer data we additionally populate
+/// `content_items` with the structured form that the Responses/Chat
+/// Completions APIs understand.
+#[derive(Debug, Default, Clone, PartialEq, JsonSchema, TS)]
+pub struct FunctionCallOutputPayload {
+    pub content: String,
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub content_items: Option<Vec<FunctionCallOutputContentItem>>,
+    // TODO(jif) drop this.
+    pub success: Option<bool>,
+}
+
+#[derive(Deserialize)]
+#[serde(untagged)]
+enum FunctionCallOutputPayloadSerde {
+    Text(String),
+    Items(Vec<FunctionCallOutputContentItem>),
+}
+
+// The Responses API expects two *different* shapes depending on success vs failure:
+//   • success → output is a plain string (no nested object)
+//   • failure → output is an object { content, success:false }
+impl Serialize for FunctionCallOutputPayload {
+    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
+    where
+        S: Serializer,
+    {
+        if let Some(items) = &self.content_items {
+            items.serialize(serializer)
+        } else {
+            serializer.serialize_str(&self.content)
+        }
+    }
+}
+
+impl<'de> Deserialize<'de> for FunctionCallOutputPayload {
+    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
+    where
+        D: Deserializer<'de>,
+    {
+        match FunctionCallOutputPayloadSerde::deserialize(deserializer)? {
+            FunctionCallOutputPayloadSerde::Text(content) => Ok(FunctionCallOutputPayload {
+                content,
+                ..Default::default()
+            }),
+            FunctionCallOutputPayloadSerde::Items(items) => {
+                let content = serde_json::to_string(&items).map_err(serde::de::Error::custom)?;
+                Ok(FunctionCallOutputPayload {
+                    content,
+                    content_items: Some(items),
+                    success: None,
+                })
+            }
+        }
+    }
+}
+
+impl From<&CallToolResult> for FunctionCallOutputPayload {
+    fn from(call_tool_result: &CallToolResult) -> Self {
+        let CallToolResult {
+            content,
+            structured_content,
+            is_error,
+        } = call_tool_result;
+
+        let is_success = is_error != &Some(true);
+
+        if let Some(structured_content) = structured_content
+            && !structured_content.is_null()
+        {
+            match serde_json::to_string(structured_content) {
+                Ok(serialized_structured_content) => {
+                    return FunctionCallOutputPayload {
+                        content: serialized_structured_content,
+                        success: Some(is_success),
+                        ..Default::default()
+                    };
+                }
+                Err(err) => {
+                    return FunctionCallOutputPayload {
+                        content: err.to_string(),
+                        success: Some(false),
+                        ..Default::default()
+                    };
+                }
+            }
+        }
+
+        let serialized_content = match serde_json::to_string(content) {
+            Ok(serialized_content) => serialized_content,
+            Err(err) => {
+                return FunctionCallOutputPayload {
+                    content: err.to_string(),
+                    success: Some(false),
+                    ..Default::default()
+                };
+            }
+        };
+
+        let content_items = convert_content_blocks_to_items(content);
+
+        FunctionCallOutputPayload {
+            content: serialized_content,
+            content_items,
+            success: Some(is_success),
+        }
+    }
+}
+
+fn convert_content_blocks_to_items(
+    blocks: &[ContentBlock],
+) -> Option<Vec<FunctionCallOutputContentItem>> {
+    let mut saw_image = false;
+    let mut items = Vec::with_capacity(blocks.len());
+
+    for block in blocks {
+        match block {
+            ContentBlock::TextContent(text) => {
+                items.push(FunctionCallOutputContentItem::InputText {
+                    text: text.text.clone(),
+                });
+            }
+            ContentBlock::ImageContent(image) => {
+                saw_image = true;
+                // Just in case the content doesn't include a data URL, add it.
+                let image_url = if image.data.starts_with("data:") {
+                    image.data.clone()
+                } else {
+                    format!("data:{};base64,{}", image.mime_type, image.data)
+                };
+                items.push(FunctionCallOutputContentItem::InputImage { image_url });
+            }
+            // TODO: render audio, resource, and embedded resource content to the model.
+            _ => return None,
+        }
+    }
+
+    if saw_image { Some(items) } else { None }
+}
+
+// Implement Display so callers can treat the payload like a plain string when logging or doing
+// trivial substring checks in tests (existing tests call `.contains()` on the output). Display
+// returns the raw `content` field.
+
+impl std::fmt::Display for FunctionCallOutputPayload {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        f.write_str(&self.content)
+    }
+}
+
+impl std::ops::Deref for FunctionCallOutputPayload {
+    type Target = str;
+    fn deref(&self) -> &Self::Target {
+        &self.content
+    }
+}
+
+// (Moved event mapping logic into llmx-core to avoid coupling protocol to UI-facing events.)
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use anyhow::Result;
+    use mcp_types::ImageContent;
+    use mcp_types::TextContent;
+    use tempfile::tempdir;
+
+    #[test]
+    fn serializes_success_as_plain_string() -> Result<()> {
+        let item = ResponseInputItem::FunctionCallOutput {
+            call_id: "call1".into(),
+            output: FunctionCallOutputPayload {
+                content: "ok".into(),
+                ..Default::default()
+            },
+        };
+
+        let json = serde_json::to_string(&item)?;
+        let v: serde_json::Value = serde_json::from_str(&json)?;
+
+        // Success case -> output should be a plain string
+        assert_eq!(v.get("output").unwrap().as_str().unwrap(), "ok");
+        Ok(())
+    }
+
+    #[test]
+    fn serializes_failure_as_string() -> Result<()> {
+        let item = ResponseInputItem::FunctionCallOutput {
+            call_id: "call1".into(),
+            output: FunctionCallOutputPayload {
+                content: "bad".into(),
+                success: Some(false),
+                ..Default::default()
+            },
+        };
+
+        let json = serde_json::to_string(&item)?;
+        let v: serde_json::Value = serde_json::from_str(&json)?;
+
+        assert_eq!(v.get("output").unwrap().as_str().unwrap(), "bad");
+        Ok(())
+    }
+
+    #[test]
+    fn serializes_image_outputs_as_array() -> Result<()> {
+        let call_tool_result = CallToolResult {
+            content: vec![
+                ContentBlock::TextContent(TextContent {
+                    annotations: None,
+                    text: "caption".into(),
+                    r#type: "text".into(),
+                }),
+                ContentBlock::ImageContent(ImageContent {
+                    annotations: None,
+                    data: "BASE64".into(),
+                    mime_type: "image/png".into(),
+                    r#type: "image".into(),
+                }),
+            ],
+            is_error: None,
+            structured_content: None,
+        };
+
+        let payload = FunctionCallOutputPayload::from(&call_tool_result);
+        assert_eq!(payload.success, Some(true));
+        let items = payload.content_items.clone().expect("content items");
+        assert_eq!(
+            items,
+            vec![
+                FunctionCallOutputContentItem::InputText {
+                    text: "caption".into(),
+                },
+                FunctionCallOutputContentItem::InputImage {
+                    image_url: "data:image/png;base64,BASE64".into(),
+                },
+            ]
+        );
+
+        let item = ResponseInputItem::FunctionCallOutput {
+            call_id: "call1".into(),
+            output: payload,
+        };
+
+        let json = serde_json::to_string(&item)?;
+        let v: serde_json::Value = serde_json::from_str(&json)?;
+
+        let output = v.get("output").expect("output field");
+        assert!(output.is_array(), "expected array output");
+
+        Ok(())
+    }
+
+    #[test]
+    fn deserializes_array_payload_into_items() -> Result<()> {
+        let json = r#"[
+            {"type": "input_text", "text": "note"},
+            {"type": "input_image", "image_url": "data:image/png;base64,XYZ"}
+        ]"#;
+
+        let payload: FunctionCallOutputPayload = serde_json::from_str(json)?;
+
+        assert_eq!(payload.success, None);
+        let expected_items = vec![
+            FunctionCallOutputContentItem::InputText {
+                text: "note".into(),
+            },
+            FunctionCallOutputContentItem::InputImage {
+                image_url: "data:image/png;base64,XYZ".into(),
+            },
+        ];
+        assert_eq!(payload.content_items, Some(expected_items.clone()));
+
+        let expected_content = serde_json::to_string(&expected_items)?;
+        assert_eq!(payload.content, expected_content);
+
+        Ok(())
+    }
+
+    #[test]
+    fn deserialize_shell_tool_call_params() -> Result<()> {
+        let json = r#"{
+            "command": ["ls", "-l"],
+            "workdir": "/tmp",
+            "timeout": 1000
+        }"#;
+
+        let params: ShellToolCallParams = serde_json::from_str(json)?;
+        assert_eq!(
+            ShellToolCallParams {
+                command: vec!["ls".to_string(), "-l".to_string()],
+                workdir: Some("/tmp".to_string()),
+                timeout_ms: Some(1000),
+                with_escalated_permissions: None,
+                justification: None,
+            },
+            params
+        );
+        Ok(())
+    }
+
+    #[test]
+    fn local_image_read_error_adds_placeholder() -> Result<()> {
+        let dir = tempdir()?;
+        let missing_path = dir.path().join("missing-image.png");
+
+        let item = ResponseInputItem::from(vec![UserInput::LocalImage {
+            path: missing_path.clone(),
+        }]);
+
+        match item {
+            ResponseInputItem::Message { content, .. } => {
+                assert_eq!(content.len(), 1);
+                match &content[0] {
+                    ContentItem::InputText { text } => {
+                        let display_path = missing_path.display().to_string();
+                        assert!(
+                            text.contains(&display_path),
+                            "placeholder should mention missing path: {text}"
+                        );
+                        assert!(
+                            text.contains("could not read"),
+                            "placeholder should mention read issue: {text}"
+                        );
+                    }
+                    other => panic!("expected placeholder text but found {other:?}"),
+                }
+            }
+            other => panic!("expected message response but got {other:?}"),
+        }
+
+        Ok(())
+    }
+
+    #[test]
+    fn local_image_non_image_adds_placeholder() -> Result<()> {
+        let dir = tempdir()?;
+        let json_path = dir.path().join("example.json");
+        std::fs::write(&json_path, br#"{"hello":"world"}"#)?;
+
+        let item = ResponseInputItem::from(vec![UserInput::LocalImage {
+            path: json_path.clone(),
+        }]);
+
+        match item {
+            ResponseInputItem::Message { content, .. } => {
+                assert_eq!(content.len(), 1);
+                match &content[0] {
+                    ContentItem::InputText { text } => {
+                        assert!(
+                            text.contains("unsupported MIME type `application/json`"),
+                            "placeholder should mention unsupported MIME: {text}"
+                        );
+                        assert!(
+                            text.contains(&json_path.display().to_string()),
+                            "placeholder should mention path: {text}"
+                        );
+                    }
+                    other => panic!("expected placeholder text but found {other:?}"),
+                }
+            }
+            other => panic!("expected message response but got {other:?}"),
+        }
+
+        Ok(())
+    }
+}

llmx-rs/protocol/src/num_format.rs

@@ -0,0 +1,99 @@
+use std::sync::OnceLock;
+
+use icu_decimal::DecimalFormatter;
+use icu_decimal::input::Decimal;
+use icu_decimal::options::DecimalFormatterOptions;
+use icu_locale_core::Locale;
+
+fn make_local_formatter() -> Option<DecimalFormatter> {
+    let loc: Locale = sys_locale::get_locale()?.parse().ok()?;
+    DecimalFormatter::try_new(loc.into(), DecimalFormatterOptions::default()).ok()
+}
+
+fn make_en_us_formatter() -> DecimalFormatter {
+    #![allow(clippy::expect_used)]
+    let loc: Locale = "en-US".parse().expect("en-US wasn't a valid locale");
+    DecimalFormatter::try_new(loc.into(), DecimalFormatterOptions::default())
+        .expect("en-US wasn't a valid locale")
+}
+
+fn formatter() -> &'static DecimalFormatter {
+    static FORMATTER: OnceLock<DecimalFormatter> = OnceLock::new();
+    FORMATTER.get_or_init(|| make_local_formatter().unwrap_or_else(make_en_us_formatter))
+}
+
+/// Format an i64 with locale-aware digit separators (e.g. "12345" -> "12,345"
+/// for en-US).
+pub fn format_with_separators(n: i64) -> String {
+    formatter().format(&Decimal::from(n)).to_string()
+}
+
+fn format_si_suffix_with_formatter(n: i64, formatter: &DecimalFormatter) -> String {
+    let n = n.max(0);
+    if n < 1000 {
+        return formatter.format(&Decimal::from(n)).to_string();
+    }
+
+    // Format `n / scale` with the requested number of fractional digits.
+    let format_scaled = |n: i64, scale: i64, frac_digits: u32| -> String {
+        let value = n as f64 / scale as f64;
+        let scaled: i64 = (value * 10f64.powi(frac_digits as i32)).round() as i64;
+        let mut dec = Decimal::from(scaled);
+        dec.multiply_pow10(-(frac_digits as i16));
+        formatter.format(&dec).to_string()
+    };
+
+    const UNITS: [(i64, &str); 3] = [(1_000, "K"), (1_000_000, "M"), (1_000_000_000, "G")];
+    let f = n as f64;
+    for &(scale, suffix) in &UNITS {
+        if (100.0 * f / scale as f64).round() < 1000.0 {
+            return format!("{}{}", format_scaled(n, scale, 2), suffix);
+        } else if (10.0 * f / scale as f64).round() < 1000.0 {
+            return format!("{}{}", format_scaled(n, scale, 1), suffix);
+        } else if (f / scale as f64).round() < 1000.0 {
+            return format!("{}{}", format_scaled(n, scale, 0), suffix);
+        }
+    }
+
+    // Above 1000G, keep whole‑G precision.
+    format!(
+        "{}G",
+        format_with_separators(((n as f64) / 1e9).round() as i64)
+    )
+}
+
+/// Format token counts to 3 significant figures, using base-10 SI suffixes.
+///
+/// Examples (en-US):
+///   - 999 -> "999"
+///   - 1200 -> "1.20K"
+///   - 123456789 -> "123M"
+pub fn format_si_suffix(n: i64) -> String {
+    format_si_suffix_with_formatter(n, formatter())
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn kmg() {
+        let formatter = make_en_us_formatter();
+        let fmt = |n: i64| format_si_suffix_with_formatter(n, &formatter);
+        assert_eq!(fmt(0), "0");
+        assert_eq!(fmt(999), "999");
+        assert_eq!(fmt(1_000), "1.00K");
+        assert_eq!(fmt(1_200), "1.20K");
+        assert_eq!(fmt(10_000), "10.0K");
+        assert_eq!(fmt(100_000), "100K");
+        assert_eq!(fmt(999_500), "1.00M");
+        assert_eq!(fmt(1_000_000), "1.00M");
+        assert_eq!(fmt(1_234_000), "1.23M");
+        assert_eq!(fmt(12_345_678), "12.3M");
+        assert_eq!(fmt(999_950_000), "1.00G");
+        assert_eq!(fmt(1_000_000_000), "1.00G");
+        assert_eq!(fmt(1_234_000_000), "1.23G");
+        // Above 1000G we keep whole‑G precision (no higher unit supported here).
+        assert_eq!(fmt(1_234_000_000_000), "1,234G");
+    }
+}

llmx-rs/protocol/src/parse_command.rs

@@ -0,0 +1,31 @@
+use schemars::JsonSchema;
+use serde::Deserialize;
+use serde::Serialize;
+use std::path::PathBuf;
+use ts_rs::TS;
+
+#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize, JsonSchema, TS)]
+#[serde(tag = "type", rename_all = "snake_case")]
+pub enum ParsedCommand {
+    Read {
+        cmd: String,
+        name: String,
+        /// (Best effort) Path to the file being read by the command. When
+        /// possible, this is an absolute path, though when relative, it should
+        /// be resolved against the `cwd`` that will be used to run the command
+        /// to derive the absolute path.
+        path: PathBuf,
+    },
+    ListFiles {
+        cmd: String,
+        path: Option<String>,
+    },
+    Search {
+        cmd: String,
+        query: Option<String>,
+        path: Option<String>,
+    },
+    Unknown {
+        cmd: String,
+    },
+}

llmx-rs/protocol/src/plan_tool.rs

@@ -0,0 +1,28 @@
+use schemars::JsonSchema;
+use serde::Deserialize;
+use serde::Serialize;
+use ts_rs::TS;
+
+// Types for the TODO tool arguments matching llmx-vscode/todo-mcp/src/main.rs
+#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, TS)]
+#[serde(rename_all = "snake_case")]
+pub enum StepStatus {
+    Pending,
+    InProgress,
+    Completed,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, TS)]
+#[serde(deny_unknown_fields)]
+pub struct PlanItemArg {
+    pub step: String,
+    pub status: StepStatus,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize, JsonSchema, TS)]
+#[serde(deny_unknown_fields)]
+pub struct UpdatePlanArgs {
+    #[serde(default)]
+    pub explanation: Option<String>,
+    pub plan: Vec<PlanItemArg>,
+}

llmx-rs/protocol/src/user_input.rs

@@ -0,0 +1,24 @@
+use schemars::JsonSchema;
+use serde::Deserialize;
+use serde::Serialize;
+use ts_rs::TS;
+
+/// User input
+#[non_exhaustive]
+#[derive(Debug, Clone, Deserialize, Serialize, PartialEq, TS, JsonSchema)]
+#[serde(tag = "type", rename_all = "snake_case")]
+pub enum UserInput {
+    Text {
+        text: String,
+    },
+    /// Pre‑encoded data: URI image.
+    Image {
+        image_url: String,
+    },
+
+    /// Local image path provided by the user.  This will be converted to an
+    /// `Image` variant (base64 data URL) during request serialization.
+    LocalImage {
+        path: std::path::PathBuf,
+    },
+}