chore: refactor tool handling (#4510)
# Tool System Refactor
- Centralizes tool definitions and execution in `core/src/tools/*`:
specs (`spec.rs`), handlers (`handlers/*`), router (`router.rs`),
registry/dispatch (`registry.rs`), and shared context (`context.rs`).
One registry now builds the model-visible tool list and binds handlers.
- Router converts model responses to tool calls; Registry dispatches
with consistent telemetry via `codex-rs/otel` and unified error
handling. Function, Local Shell, MCP, and experimental `unified_exec`
all flow through this path; legacy shell aliases still work.
- Rationale: reduce per‑tool boilerplate, keep spec/handler in sync, and
make adding tools predictable and testable.
Example: `read_file`
- Spec: `core/src/tools/spec.rs` (see `create_read_file_tool`,
registered by `build_specs`).
- Handler: `core/src/tools/handlers/read_file.rs` (absolute `file_path`,
1‑indexed `offset`, `limit`, `L#: ` prefixes, safe truncation).
- E2E test: `core/tests/suite/read_file.rs` validates the tool returns
the requested lines.
## Next steps:
- Decompose `handle_container_exec_with_params`
- Add parallel tool calls
2025-10-03 13:21:06 +01:00
|
|
|
use std::collections::HashMap;
|
2025-10-05 17:10:49 +01:00
|
|
|
use std::sync::Arc;
|
chore: refactor tool handling (#4510)
# Tool System Refactor
- Centralizes tool definitions and execution in `core/src/tools/*`:
specs (`spec.rs`), handlers (`handlers/*`), router (`router.rs`),
registry/dispatch (`registry.rs`), and shared context (`context.rs`).
One registry now builds the model-visible tool list and binds handlers.
- Router converts model responses to tool calls; Registry dispatches
with consistent telemetry via `codex-rs/otel` and unified error
handling. Function, Local Shell, MCP, and experimental `unified_exec`
all flow through this path; legacy shell aliases still work.
- Rationale: reduce per‑tool boilerplate, keep spec/handler in sync, and
make adding tools predictable and testable.
Example: `read_file`
- Spec: `core/src/tools/spec.rs` (see `create_read_file_tool`,
registered by `build_specs`).
- Handler: `core/src/tools/handlers/read_file.rs` (absolute `file_path`,
1‑indexed `offset`, `limit`, `L#: ` prefixes, safe truncation).
- E2E test: `core/tests/suite/read_file.rs` validates the tool returns
the requested lines.
## Next steps:
- Decompose `handle_container_exec_with_params`
- Add parallel tool calls
2025-10-03 13:21:06 +01:00
|
|
|
|
|
|
|
|
use crate::client_common::tools::ToolSpec;
|
|
|
|
|
use crate::codex::Session;
|
|
|
|
|
use crate::codex::TurnContext;
|
|
|
|
|
use crate::function_tool::FunctionCallError;
|
2025-10-05 17:10:49 +01:00
|
|
|
use crate::tools::context::SharedTurnDiffTracker;
|
chore: refactor tool handling (#4510)
# Tool System Refactor
- Centralizes tool definitions and execution in `core/src/tools/*`:
specs (`spec.rs`), handlers (`handlers/*`), router (`router.rs`),
registry/dispatch (`registry.rs`), and shared context (`context.rs`).
One registry now builds the model-visible tool list and binds handlers.
- Router converts model responses to tool calls; Registry dispatches
with consistent telemetry via `codex-rs/otel` and unified error
handling. Function, Local Shell, MCP, and experimental `unified_exec`
all flow through this path; legacy shell aliases still work.
- Rationale: reduce per‑tool boilerplate, keep spec/handler in sync, and
make adding tools predictable and testable.
Example: `read_file`
- Spec: `core/src/tools/spec.rs` (see `create_read_file_tool`,
registered by `build_specs`).
- Handler: `core/src/tools/handlers/read_file.rs` (absolute `file_path`,
1‑indexed `offset`, `limit`, `L#: ` prefixes, safe truncation).
- E2E test: `core/tests/suite/read_file.rs` validates the tool returns
the requested lines.
## Next steps:
- Decompose `handle_container_exec_with_params`
- Add parallel tool calls
2025-10-03 13:21:06 +01:00
|
|
|
use crate::tools::context::ToolInvocation;
|
|
|
|
|
use crate::tools::context::ToolPayload;
|
2025-10-05 17:10:49 +01:00
|
|
|
use crate::tools::registry::ConfiguredToolSpec;
|
chore: refactor tool handling (#4510)
# Tool System Refactor
- Centralizes tool definitions and execution in `core/src/tools/*`:
specs (`spec.rs`), handlers (`handlers/*`), router (`router.rs`),
registry/dispatch (`registry.rs`), and shared context (`context.rs`).
One registry now builds the model-visible tool list and binds handlers.
- Router converts model responses to tool calls; Registry dispatches
with consistent telemetry via `codex-rs/otel` and unified error
handling. Function, Local Shell, MCP, and experimental `unified_exec`
all flow through this path; legacy shell aliases still work.
- Rationale: reduce per‑tool boilerplate, keep spec/handler in sync, and
make adding tools predictable and testable.
Example: `read_file`
- Spec: `core/src/tools/spec.rs` (see `create_read_file_tool`,
registered by `build_specs`).
- Handler: `core/src/tools/handlers/read_file.rs` (absolute `file_path`,
1‑indexed `offset`, `limit`, `L#: ` prefixes, safe truncation).
- E2E test: `core/tests/suite/read_file.rs` validates the tool returns
the requested lines.
## Next steps:
- Decompose `handle_container_exec_with_params`
- Add parallel tool calls
2025-10-03 13:21:06 +01:00
|
|
|
use crate::tools::registry::ToolRegistry;
|
|
|
|
|
use crate::tools::spec::ToolsConfig;
|
|
|
|
|
use crate::tools::spec::build_specs;
|
|
|
|
|
use codex_protocol::models::LocalShellAction;
|
|
|
|
|
use codex_protocol::models::ResponseInputItem;
|
|
|
|
|
use codex_protocol::models::ResponseItem;
|
|
|
|
|
use codex_protocol::models::ShellToolCallParams;
|
|
|
|
|
|
|
|
|
|
#[derive(Clone)]
|
|
|
|
|
pub struct ToolCall {
|
|
|
|
|
pub tool_name: String,
|
|
|
|
|
pub call_id: String,
|
|
|
|
|
pub payload: ToolPayload,
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pub struct ToolRouter {
|
|
|
|
|
registry: ToolRegistry,
|
2025-10-05 17:10:49 +01:00
|
|
|
specs: Vec<ConfiguredToolSpec>,
|
chore: refactor tool handling (#4510)
# Tool System Refactor
- Centralizes tool definitions and execution in `core/src/tools/*`:
specs (`spec.rs`), handlers (`handlers/*`), router (`router.rs`),
registry/dispatch (`registry.rs`), and shared context (`context.rs`).
One registry now builds the model-visible tool list and binds handlers.
- Router converts model responses to tool calls; Registry dispatches
with consistent telemetry via `codex-rs/otel` and unified error
handling. Function, Local Shell, MCP, and experimental `unified_exec`
all flow through this path; legacy shell aliases still work.
- Rationale: reduce per‑tool boilerplate, keep spec/handler in sync, and
make adding tools predictable and testable.
Example: `read_file`
- Spec: `core/src/tools/spec.rs` (see `create_read_file_tool`,
registered by `build_specs`).
- Handler: `core/src/tools/handlers/read_file.rs` (absolute `file_path`,
1‑indexed `offset`, `limit`, `L#: ` prefixes, safe truncation).
- E2E test: `core/tests/suite/read_file.rs` validates the tool returns
the requested lines.
## Next steps:
- Decompose `handle_container_exec_with_params`
- Add parallel tool calls
2025-10-03 13:21:06 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
impl ToolRouter {
|
|
|
|
|
pub fn from_config(
|
|
|
|
|
config: &ToolsConfig,
|
|
|
|
|
mcp_tools: Option<HashMap<String, mcp_types::Tool>>,
|
|
|
|
|
) -> Self {
|
|
|
|
|
let builder = build_specs(config, mcp_tools);
|
|
|
|
|
let (specs, registry) = builder.build();
|
2025-10-05 17:10:49 +01:00
|
|
|
|
chore: refactor tool handling (#4510)
# Tool System Refactor
- Centralizes tool definitions and execution in `core/src/tools/*`:
specs (`spec.rs`), handlers (`handlers/*`), router (`router.rs`),
registry/dispatch (`registry.rs`), and shared context (`context.rs`).
One registry now builds the model-visible tool list and binds handlers.
- Router converts model responses to tool calls; Registry dispatches
with consistent telemetry via `codex-rs/otel` and unified error
handling. Function, Local Shell, MCP, and experimental `unified_exec`
all flow through this path; legacy shell aliases still work.
- Rationale: reduce per‑tool boilerplate, keep spec/handler in sync, and
make adding tools predictable and testable.
Example: `read_file`
- Spec: `core/src/tools/spec.rs` (see `create_read_file_tool`,
registered by `build_specs`).
- Handler: `core/src/tools/handlers/read_file.rs` (absolute `file_path`,
1‑indexed `offset`, `limit`, `L#: ` prefixes, safe truncation).
- E2E test: `core/tests/suite/read_file.rs` validates the tool returns
the requested lines.
## Next steps:
- Decompose `handle_container_exec_with_params`
- Add parallel tool calls
2025-10-03 13:21:06 +01:00
|
|
|
Self { registry, specs }
|
|
|
|
|
}
|
|
|
|
|
|
2025-10-05 17:10:49 +01:00
|
|
|
pub fn specs(&self) -> Vec<ToolSpec> {
|
|
|
|
|
self.specs
|
|
|
|
|
.iter()
|
|
|
|
|
.map(|config| config.spec.clone())
|
|
|
|
|
.collect()
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pub fn tool_supports_parallel(&self, tool_name: &str) -> bool {
|
|
|
|
|
self.specs
|
|
|
|
|
.iter()
|
|
|
|
|
.filter(|config| config.supports_parallel_tool_calls)
|
|
|
|
|
.any(|config| config.spec.name() == tool_name)
|
chore: refactor tool handling (#4510)
# Tool System Refactor
- Centralizes tool definitions and execution in `core/src/tools/*`:
specs (`spec.rs`), handlers (`handlers/*`), router (`router.rs`),
registry/dispatch (`registry.rs`), and shared context (`context.rs`).
One registry now builds the model-visible tool list and binds handlers.
- Router converts model responses to tool calls; Registry dispatches
with consistent telemetry via `codex-rs/otel` and unified error
handling. Function, Local Shell, MCP, and experimental `unified_exec`
all flow through this path; legacy shell aliases still work.
- Rationale: reduce per‑tool boilerplate, keep spec/handler in sync, and
make adding tools predictable and testable.
Example: `read_file`
- Spec: `core/src/tools/spec.rs` (see `create_read_file_tool`,
registered by `build_specs`).
- Handler: `core/src/tools/handlers/read_file.rs` (absolute `file_path`,
1‑indexed `offset`, `limit`, `L#: ` prefixes, safe truncation).
- E2E test: `core/tests/suite/read_file.rs` validates the tool returns
the requested lines.
## Next steps:
- Decompose `handle_container_exec_with_params`
- Add parallel tool calls
2025-10-03 13:21:06 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pub fn build_tool_call(
|
|
|
|
|
session: &Session,
|
|
|
|
|
item: ResponseItem,
|
|
|
|
|
) -> Result<Option<ToolCall>, FunctionCallError> {
|
|
|
|
|
match item {
|
|
|
|
|
ResponseItem::FunctionCall {
|
|
|
|
|
name,
|
|
|
|
|
arguments,
|
|
|
|
|
call_id,
|
|
|
|
|
..
|
|
|
|
|
} => {
|
|
|
|
|
if let Some((server, tool)) = session.parse_mcp_tool_name(&name) {
|
|
|
|
|
Ok(Some(ToolCall {
|
|
|
|
|
tool_name: name,
|
|
|
|
|
call_id,
|
|
|
|
|
payload: ToolPayload::Mcp {
|
|
|
|
|
server,
|
|
|
|
|
tool,
|
|
|
|
|
raw_arguments: arguments,
|
|
|
|
|
},
|
|
|
|
|
}))
|
|
|
|
|
} else {
|
|
|
|
|
let payload = if name == "unified_exec" {
|
|
|
|
|
ToolPayload::UnifiedExec { arguments }
|
|
|
|
|
} else {
|
|
|
|
|
ToolPayload::Function { arguments }
|
|
|
|
|
};
|
|
|
|
|
Ok(Some(ToolCall {
|
|
|
|
|
tool_name: name,
|
|
|
|
|
call_id,
|
|
|
|
|
payload,
|
|
|
|
|
}))
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
ResponseItem::CustomToolCall {
|
|
|
|
|
name,
|
|
|
|
|
input,
|
|
|
|
|
call_id,
|
|
|
|
|
..
|
|
|
|
|
} => Ok(Some(ToolCall {
|
|
|
|
|
tool_name: name,
|
|
|
|
|
call_id,
|
|
|
|
|
payload: ToolPayload::Custom { input },
|
|
|
|
|
})),
|
|
|
|
|
ResponseItem::LocalShellCall {
|
|
|
|
|
id,
|
|
|
|
|
call_id,
|
|
|
|
|
action,
|
|
|
|
|
..
|
|
|
|
|
} => {
|
|
|
|
|
let call_id = call_id
|
|
|
|
|
.or(id)
|
|
|
|
|
.ok_or(FunctionCallError::MissingLocalShellCallId)?;
|
|
|
|
|
|
|
|
|
|
match action {
|
|
|
|
|
LocalShellAction::Exec(exec) => {
|
|
|
|
|
let params = ShellToolCallParams {
|
|
|
|
|
command: exec.command,
|
|
|
|
|
workdir: exec.working_directory,
|
|
|
|
|
timeout_ms: exec.timeout_ms,
|
|
|
|
|
with_escalated_permissions: None,
|
|
|
|
|
justification: None,
|
|
|
|
|
};
|
|
|
|
|
Ok(Some(ToolCall {
|
|
|
|
|
tool_name: "local_shell".to_string(),
|
|
|
|
|
call_id,
|
|
|
|
|
payload: ToolPayload::LocalShell { params },
|
|
|
|
|
}))
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
_ => Ok(None),
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
pub async fn dispatch_tool_call(
|
|
|
|
|
&self,
|
2025-10-05 17:10:49 +01:00
|
|
|
session: Arc<Session>,
|
|
|
|
|
turn: Arc<TurnContext>,
|
|
|
|
|
tracker: SharedTurnDiffTracker,
|
chore: refactor tool handling (#4510)
# Tool System Refactor
- Centralizes tool definitions and execution in `core/src/tools/*`:
specs (`spec.rs`), handlers (`handlers/*`), router (`router.rs`),
registry/dispatch (`registry.rs`), and shared context (`context.rs`).
One registry now builds the model-visible tool list and binds handlers.
- Router converts model responses to tool calls; Registry dispatches
with consistent telemetry via `codex-rs/otel` and unified error
handling. Function, Local Shell, MCP, and experimental `unified_exec`
all flow through this path; legacy shell aliases still work.
- Rationale: reduce per‑tool boilerplate, keep spec/handler in sync, and
make adding tools predictable and testable.
Example: `read_file`
- Spec: `core/src/tools/spec.rs` (see `create_read_file_tool`,
registered by `build_specs`).
- Handler: `core/src/tools/handlers/read_file.rs` (absolute `file_path`,
1‑indexed `offset`, `limit`, `L#: ` prefixes, safe truncation).
- E2E test: `core/tests/suite/read_file.rs` validates the tool returns
the requested lines.
## Next steps:
- Decompose `handle_container_exec_with_params`
- Add parallel tool calls
2025-10-03 13:21:06 +01:00
|
|
|
call: ToolCall,
|
|
|
|
|
) -> Result<ResponseInputItem, FunctionCallError> {
|
|
|
|
|
let ToolCall {
|
|
|
|
|
tool_name,
|
|
|
|
|
call_id,
|
|
|
|
|
payload,
|
|
|
|
|
} = call;
|
|
|
|
|
let payload_outputs_custom = matches!(payload, ToolPayload::Custom { .. });
|
|
|
|
|
let failure_call_id = call_id.clone();
|
|
|
|
|
|
|
|
|
|
let invocation = ToolInvocation {
|
|
|
|
|
session,
|
|
|
|
|
turn,
|
|
|
|
|
tracker,
|
|
|
|
|
call_id,
|
|
|
|
|
tool_name,
|
|
|
|
|
payload,
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
match self.registry.dispatch(invocation).await {
|
|
|
|
|
Ok(response) => Ok(response),
|
|
|
|
|
Err(FunctionCallError::Fatal(message)) => Err(FunctionCallError::Fatal(message)),
|
|
|
|
|
Err(err) => Ok(Self::failure_response(
|
|
|
|
|
failure_call_id,
|
|
|
|
|
payload_outputs_custom,
|
|
|
|
|
err,
|
|
|
|
|
)),
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
fn failure_response(
|
|
|
|
|
call_id: String,
|
|
|
|
|
payload_outputs_custom: bool,
|
|
|
|
|
err: FunctionCallError,
|
|
|
|
|
) -> ResponseInputItem {
|
|
|
|
|
let message = err.to_string();
|
|
|
|
|
if payload_outputs_custom {
|
|
|
|
|
ResponseInputItem::CustomToolCallOutput {
|
|
|
|
|
call_id,
|
|
|
|
|
output: message,
|
|
|
|
|
}
|
|
|
|
|
} else {
|
|
|
|
|
ResponseInputItem::FunctionCallOutput {
|
|
|
|
|
call_id,
|
|
|
|
|
output: codex_protocol::models::FunctionCallOutputPayload {
|
|
|
|
|
content: message,
|
|
|
|
|
success: Some(false),
|
[MCP] Render MCP tool call result images to the model (#5600)
It's pretty amazing we have gotten here without the ability for the
model to see image content from MCP tool calls.
This PR builds off of 4391 and fixes #4819. I would like @KKcorps to get
adequete credit here but I also want to get this fix in ASAP so I gave
him a week to update it and haven't gotten a response so I'm going to
take it across the finish line.
This test highlights how absured the current situation is. I asked the
model to read this image using the Chrome MCP
<img width="2378" height="674" alt="image"
src="https://github.com/user-attachments/assets/9ef52608-72a2-4423-9f5e-7ae36b2b56e0"
/>
After this change, it correctly outputs:
> Captured the page: image dhows a dark terminal-style UI labeled
`OpenAI Codex (v0.0.0)` with prompt `model: gpt-5-codex medium` and
working directory `/codex/codex-rs`
(and more)
Before this change, it said:
> Took the full-page screenshot you asked for. It shows a long,
horizontally repeating pattern of stylized people in orange, light-blue,
and mustard clothing, holding hands in alternating poses against a white
background. No text or other graphics-just rows of flat illustration
stretching off to the right.
Without this change, the Figma, Playwright, Chrome, and other visual MCP
servers are pretty much entirely useless.
I tested this change with the openai respones api as well as a third
party completions api
2025-10-27 14:55:57 -07:00
|
|
|
..Default::default()
|
chore: refactor tool handling (#4510)
# Tool System Refactor
- Centralizes tool definitions and execution in `core/src/tools/*`:
specs (`spec.rs`), handlers (`handlers/*`), router (`router.rs`),
registry/dispatch (`registry.rs`), and shared context (`context.rs`).
One registry now builds the model-visible tool list and binds handlers.
- Router converts model responses to tool calls; Registry dispatches
with consistent telemetry via `codex-rs/otel` and unified error
handling. Function, Local Shell, MCP, and experimental `unified_exec`
all flow through this path; legacy shell aliases still work.
- Rationale: reduce per‑tool boilerplate, keep spec/handler in sync, and
make adding tools predictable and testable.
Example: `read_file`
- Spec: `core/src/tools/spec.rs` (see `create_read_file_tool`,
registered by `build_specs`).
- Handler: `core/src/tools/handlers/read_file.rs` (absolute `file_path`,
1‑indexed `offset`, `limit`, `L#: ` prefixes, safe truncation).
- E2E test: `core/tests/suite/read_file.rs` validates the tool returns
the requested lines.
## Next steps:
- Decompose `handle_container_exec_with_params`
- Add parallel tool calls
2025-10-03 13:21:06 +01:00
|
|
|
},
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|