This is a large change to support a "history" feature like you would
expect in a shell like Bash.
History events are recorded in `$CODEX_HOME/history.jsonl`. Because it
is a JSONL file, it is straightforward to append new entries (as opposed
to the TypeScript file that uses `$CODEX_HOME/history.json`, so to be
valid JSON, each new entry entails rewriting the entire file). Because
it is possible for there to be multiple instances of Codex CLI writing
to `history.jsonl` at once, we use advisory file locking when working
with `history.jsonl` in `codex-rs/core/src/message_history.rs`.
Because we believe history is a sufficiently useful feature, we enable
it by default. Though to provide some safety, we set the file
permissions of `history.jsonl` to be `o600` so that other users on the
system cannot read the user's history. We do not yet support a default
list of `SENSITIVE_PATTERNS` as the TypeScript CLI does:
3fdf9df133/codex-cli/src/utils/storage/command-history.ts (L10-L17)
We are going to take a more conservative approach to this list in the
Rust CLI. For example, while `/\b[A-Za-z0-9-_]{20,}\b/` might exclude
sensitive information like API tokens, it would also exclude valuable
information such as references to Git commits.
As noted in the updated documentation, users can opt-out of history by
adding the following to `config.toml`:
```toml
[history]
persistence = "none"
```
Because `history.jsonl` could, in theory, be quite large, we take a[n
arguably overly pedantic] approach in reading history entries into
memory. Specifically, we start by telling the client the current number
of entries in the history file (`history_entry_count`) as well as the
inode (`history_log_id`) of `history.jsonl` (see the new fields on
`SessionConfiguredEvent`).
The client is responsible for keeping new entries in memory to create a
"local history," but if the user hits up enough times to go "past" the
end of local history, then the client should use the new
`GetHistoryEntryRequest` in the protocol to fetch older entries.
Specifically, it should pass the `history_log_id` it was given
originally and work backwards from `history_entry_count`. (It should
really fetch history in batches rather than one-at-a-time, but that is
something we can improve upon in subsequent PRs.)
The motivation behind this crazy scheme is that it is designed to defend
against:
* The `history.jsonl` being truncated during the session such that the
index into the history is no longer consistent with what had been read
up to that point. We do not yet have logic to enforce a `max_bytes` for
`history.jsonl`, but once we do, we will aspire to implement it in a way
that should result in a new inode for the file on most systems.
* New items from concurrent Codex CLI sessions amending to the history.
Because, in absence of truncation, `history.jsonl` is an append-only
log, so long as the client reads backwards from `history_entry_count`,
it should always get a consistent view of history. (That said, it will
not be able to read _new_ commands from concurrent sessions, but perhaps
we will introduce a `/` command to reload latest history or something
down the road.)
Admittedly, my testing of this feature thus far has been fairly light. I
expect we will find bugs and introduce enhancements/fixes going forward.
316 lines
11 KiB
Markdown
316 lines
11 KiB
Markdown
# codex-rs
|
|
|
|
April 24, 2025
|
|
|
|
Today, Codex CLI is written in TypeScript and requires Node.js 22+ to run it. For a number of users, this runtime requirement inhibits adoption: they would be better served by a standalone executable. As maintainers, we want Codex to run efficiently in a wide range of environments with minimal overhead. We also want to take advantage of operating system-specific APIs to provide better sandboxing, where possible.
|
|
|
|
To that end, we are moving forward with a Rust implementation of Codex CLI contained in this folder, which has the following benefits:
|
|
|
|
- The CLI compiles to small, standalone, platform-specific binaries.
|
|
- Can make direct, native calls to [seccomp](https://man7.org/linux/man-pages/man2/seccomp.2.html) and [landlock](https://man7.org/linux/man-pages/man7/landlock.7.html) in order to support sandboxing on Linux.
|
|
- No runtime garbage collection, resulting in lower memory consumption and better, more predictable performance.
|
|
|
|
Currently, the Rust implementation is materially behind the TypeScript implementation in functionality, so continue to use the TypeScript implementation for the time being. We will publish native executables via GitHub Releases as soon as we feel the Rust version is usable.
|
|
|
|
## Code Organization
|
|
|
|
This folder is the root of a Cargo workspace. It contains quite a bit of experimental code, but here are the key crates:
|
|
|
|
- [`core/`](./core) contains the business logic for Codex. Ultimately, we hope this to be a library crate that is generally useful for building other Rust/native applications that use Codex.
|
|
- [`exec/`](./exec) "headless" CLI for use in automation.
|
|
- [`tui/`](./tui) CLI that launches a fullscreen TUI built with [Ratatui](https://ratatui.rs/).
|
|
- [`cli/`](./cli) CLI multitool that provides the aforementioned CLIs via subcommands.
|
|
|
|
## Config
|
|
|
|
The CLI can be configured via a file named `config.toml`. By default, configuration is read from `~/.codex/config.toml`, though the `CODEX_HOME` environment variable can be used to specify a directory other than `~/.codex`.
|
|
|
|
The `config.toml` file supports the following options:
|
|
|
|
### model
|
|
|
|
The model that Codex should use.
|
|
|
|
```toml
|
|
model = "o3" # overrides the default of "o4-mini"
|
|
```
|
|
|
|
### model_provider
|
|
|
|
Codex comes bundled with a number of "model providers" predefined. This config value is a string that indicates which provider to use. You can also define your own providers via `model_providers`.
|
|
|
|
For example, if you are running ollama with Mistral locally, then you would need to add the following to your config:
|
|
|
|
```toml
|
|
model = "mistral"
|
|
model_provider = "ollama"
|
|
```
|
|
|
|
because the following definition for `ollama` is included in Codex:
|
|
|
|
```toml
|
|
[model_providers.ollama]
|
|
name = "Ollama"
|
|
base_url = "http://localhost:11434/v1"
|
|
wire_api = "chat"
|
|
```
|
|
|
|
This option defaults to `"openai"` and the corresponding provider is defined as follows:
|
|
|
|
```toml
|
|
[model_providers.openai]
|
|
name = "OpenAI"
|
|
base_url = "https://api.openai.com/v1"
|
|
env_key = "OPENAI_API_KEY"
|
|
wire_api = "responses"
|
|
```
|
|
|
|
### model_providers
|
|
|
|
This option lets you override and amend the default set of model providers bundled with Codex. This value is a map where the key is the value to use with `model_provider` to select the correspodning provider.
|
|
|
|
For example, if you wanted to add a provider that uses the OpenAI 4o model via the chat completions API, then you
|
|
|
|
```toml
|
|
# Recall that in TOML, root keys must be listed before tables.
|
|
model = "gpt-4o"
|
|
model_provider = "openai-chat-completions"
|
|
|
|
[model_providers.openai-chat-completions]
|
|
# Name of the provider that will be displayed in the Codex UI.
|
|
name = "OpenAI using Chat Completions"
|
|
# The path `/chat/completions` will be amended to this URL to make the POST
|
|
# request for the chat completions.
|
|
base_url = "https://api.openai.com/v1"
|
|
# If `env_key` is set, identifies an environment variable that must be set when
|
|
# using Codex with this provider. The value of the environment variable must be
|
|
# non-empty and will be used in the `Bearer TOKEN` HTTP header for the POST request.
|
|
env_key = "OPENAI_API_KEY"
|
|
# valid values for wire_api are "chat" and "responses".
|
|
wire_api = "chat"
|
|
```
|
|
|
|
### approval_policy
|
|
|
|
Determines when the user should be prompted to approve whether Codex can execute a command:
|
|
|
|
```toml
|
|
# This is analogous to --suggest in the TypeScript Codex CLI
|
|
approval_policy = "unless-allow-listed"
|
|
```
|
|
|
|
```toml
|
|
# If the command fails when run in the sandbox, Codex asks for permission to
|
|
# retry the command outside the sandbox.
|
|
approval_policy = "on-failure"
|
|
```
|
|
|
|
```toml
|
|
# User is never prompted: if the command fails, Codex will automatically try
|
|
# something out. Note the `exec` subcommand always uses this mode.
|
|
approval_policy = "never"
|
|
```
|
|
|
|
### profiles
|
|
|
|
A _profile_ is a collection of configuration values that can be set together. Multiple profiles can be defined in `config.toml` and you can specify the one you
|
|
want to use at runtime via the `--profile` flag.
|
|
|
|
Here is an example of a `config.toml` that defines multiple profiles:
|
|
|
|
```toml
|
|
model = "o3"
|
|
approval_policy = "unless-allow-listed"
|
|
sandbox_permissions = ["disk-full-read-access"]
|
|
disable_response_storage = false
|
|
|
|
# Setting `profile` is equivalent to specifying `--profile o3` on the command
|
|
# line, though the `--profile` flag can still be used to override this value.
|
|
profile = "o3"
|
|
|
|
[model_providers.openai-chat-completions]
|
|
name = "OpenAI using Chat Completions"
|
|
base_url = "https://api.openai.com/v1"
|
|
env_key = "OPENAI_API_KEY"
|
|
wire_api = "chat"
|
|
|
|
[profiles.o3]
|
|
model = "o3"
|
|
model_provider = "openai"
|
|
approval_policy = "never"
|
|
|
|
[profiles.gpt3]
|
|
model = "gpt-3.5-turbo"
|
|
model_provider = "openai-chat-completions"
|
|
|
|
[profiles.zdr]
|
|
model = "o3"
|
|
model_provider = "openai"
|
|
approval_policy = "on-failure"
|
|
disable_response_storage = true
|
|
```
|
|
|
|
Users can specify config values at multiple levels. Order of precedence is as follows:
|
|
|
|
1. custom command-line argument, e.g., `--model o3`
|
|
2. as part of a profile, where the `--profile` is specified via a CLI (or in the config file itself)
|
|
3. as an entry in `config.toml`, e.g., `model = "o3"`
|
|
4. the default value that comes with Codex CLI (i.e., Codex CLI defaults to `o4-mini`)
|
|
|
|
### sandbox_permissions
|
|
|
|
List of permissions to grant to the sandbox that Codex uses to execute untrusted commands:
|
|
|
|
```toml
|
|
# This is comparable to --full-auto in the TypeScript Codex CLI, though
|
|
# specifying `disk-write-platform-global-temp-folder` adds /tmp as a writable
|
|
# folder in addition to $TMPDIR.
|
|
sandbox_permissions = [
|
|
"disk-full-read-access",
|
|
"disk-write-platform-user-temp-folder",
|
|
"disk-write-platform-global-temp-folder",
|
|
"disk-write-cwd",
|
|
]
|
|
```
|
|
|
|
To add additional writable folders, use `disk-write-folder`, which takes a parameter (this can be specified multiple times):
|
|
|
|
```toml
|
|
sandbox_permissions = [
|
|
# ...
|
|
"disk-write-folder=/Users/mbolin/.pyenv/shims",
|
|
]
|
|
```
|
|
|
|
### mcp_servers
|
|
|
|
Defines the list of MCP servers that Codex can consult for tool use. Currently, only servers that are launched by executing a program that communicate over stdio are supported. For servers that use the SSE transport, consider an adapter like [mcp-proxy](https://github.com/sparfenyuk/mcp-proxy).
|
|
|
|
**Note:** Codex may cache the list of tools and resources from an MCP server so that Codex can include this information in context at startup without spawning all the servers. This is designed to save resources by loading MCP servers lazily.
|
|
|
|
This config option is comparable to how Claude and Cursor define `mcpServers` in their respective JSON config files, though because Codex uses TOML for its config language, the format is slightly different. For example, the following config in JSON:
|
|
|
|
```json
|
|
{
|
|
"mcpServers": {
|
|
"server-name": {
|
|
"command": "npx",
|
|
"args": ["-y", "mcp-server"],
|
|
"env": {
|
|
"API_KEY": "value"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Should be represented as follows in `~/.codex/config.toml`:
|
|
|
|
```toml
|
|
# IMPORTANT: the top-level key is `mcp_servers` rather than `mcpServers`.
|
|
[mcp_servers.server-name]
|
|
command = "npx"
|
|
args = ["-y", "mcp-server"]
|
|
env = { "API_KEY" = "value" }
|
|
```
|
|
|
|
### disable_response_storage
|
|
|
|
Currently, customers whose accounts are set to use Zero Data Retention (ZDR) must set `disable_response_storage` to `true` so that Codex uses an alternative to the Responses API that works with ZDR:
|
|
|
|
```toml
|
|
disable_response_storage = true
|
|
```
|
|
|
|
### notify
|
|
|
|
Specify a program that will be executed to get notified about events generated by Codex. Note that the program will receive the notification argument as a string of JSON, e.g.:
|
|
|
|
```json
|
|
{
|
|
"type": "agent-turn-complete",
|
|
"turn-id": "12345",
|
|
"input-messages": ["Rename `foo` to `bar` and update the callsites."],
|
|
"last-assistant-message": "Rename complete and verified `cargo build` succeeds."
|
|
}
|
|
```
|
|
|
|
The `"type"` property will always be set. Currently, `"agent-turn-complete"` is the only notification type that is supported.
|
|
|
|
As an example, here is a Python script that parses the JSON and decides whether to show a desktop push notification using [terminal-notifier](https://github.com/julienXX/terminal-notifier) on macOS:
|
|
|
|
```python
|
|
#!/usr/bin/env python3
|
|
|
|
import json
|
|
import subprocess
|
|
import sys
|
|
|
|
|
|
def main() -> int:
|
|
if len(sys.argv) != 2:
|
|
print("Usage: notify.py <NOTIFICATION_JSON>")
|
|
return 1
|
|
|
|
try:
|
|
notification = json.loads(sys.argv[1])
|
|
except json.JSONDecodeError:
|
|
return 1
|
|
|
|
match notification_type := notification.get("type"):
|
|
case "agent-turn-complete":
|
|
assistant_message = notification.get("last-assistant-message")
|
|
if assistant_message:
|
|
title = f"Codex: {assistant_message}"
|
|
else:
|
|
title = "Codex: Turn Complete!"
|
|
input_messages = notification.get("input_messages", [])
|
|
message = " ".join(input_messages)
|
|
title += message
|
|
case _:
|
|
print(f"not sending a push notification for: {notification_type}")
|
|
return 0
|
|
|
|
subprocess.check_output(
|
|
[
|
|
"terminal-notifier",
|
|
"-title",
|
|
title,
|
|
"-message",
|
|
message,
|
|
"-group",
|
|
"codex",
|
|
"-ignoreDnD",
|
|
"-activate",
|
|
"com.googlecode.iterm2",
|
|
]
|
|
)
|
|
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(main())
|
|
```
|
|
|
|
To have Codex use this script for notifications, you would configure it via `notify` in `~/.codex/config.toml` using the appropriate path to `notify.py` on your computer:
|
|
|
|
```toml
|
|
notify = ["python3", "/Users/mbolin/.codex/notify.py"]
|
|
```
|
|
|
|
### history
|
|
|
|
By default, Codex CLI records messages sent to the model in `$CODEX_HOME/history.jsonl`. Note that on UNIX, the file permissions are set to `o600`, so it should only be readable and writable by the owner.
|
|
|
|
To disable this behavior, configure `[history]` as follows:
|
|
|
|
```toml
|
|
[history]
|
|
persistence = "none" # "save-all" is the default value
|
|
```
|
|
|
|
### project_doc_max_bytes
|
|
|
|
Maximum number of bytes to read from an `AGENTS.md` file to include in the instructions sent with the first turn of a session. Defaults to 32 KiB.
|