To date, when handling `shell` and `local_shell` tool calls, we were
spawning new processes using the environment inherited from the Codex
process itself. This means that the sensitive `OPENAI_API_KEY` that
Codex needs to talk to OpenAI models was made available to everything
run by `shell` and `local_shell`. While there are cases where that might
be useful, it does not seem like a good default.
This PR introduces a complex `shell_environment_policy` config option to
control the `env` used with these tool calls. It is inevitably a bit
complex so that it is possible to override individual components of the
policy so without having to restate the entire thing.
Details are in the updated `README.md` in this PR, but here is the
relevant bit that explains the individual fields of
`shell_environment_policy`:
| Field | Type | Default | Description |
| ------------------------- | -------------------------- | ------- |
-----------------------------------------------------------------------------------------------------------------------------------------------
|
| `inherit` | string | `core` | Starting template for the
environment:<br>`core` (`HOME`, `PATH`, `USER`, …), `all` (clone full
parent env), or `none` (start empty). |
| `ignore_default_excludes` | boolean | `false` | When `false`, Codex
removes any var whose **name** contains `KEY`, `SECRET`, or `TOKEN`
(case-insensitive) before other rules run. |
| `exclude` | array<string> | `[]` | Case-insensitive glob
patterns to drop after the default filter.<br>Examples: `"AWS_*"`,
`"AZURE_*"`. |
| `set` | table<string,string> | `{}` | Explicit key/value
overrides or additions – always win over inherited values. |
| `include_only` | array<string> | `[]` | If non-empty, a
whitelist of patterns; only variables that match _one_ pattern survive
the final step. (Generally used with `inherit = "all"`.) |
In particular, note that the default is `inherit = "core"`, so:
* if you have extra env variables that you want to inherit from the
parent process, use `inherit = "all"` and then specify `include_only`
* if you have extra env variables where you want to hardcode the values,
the default `inherit = "core"` will work fine, but then you need to
specify `set`
This configuration is not battle-tested, so we will probably still have
to play with it a bit. `core/src/exec_env.rs` has the critical business
logic as well as unit tests.
Though if nothing else, previous to this change:
```
$ cargo run --bin codex -- debug seatbelt -- printenv OPENAI_API_KEY
# ...prints OPENAI_API_KEY...
```
But after this change it does not print anything (as desired).
One final thing to call out about this PR is that the
`configure_command!` macro we use in `core/src/exec.rs` has to do some
complex logic with respect to how it builds up the `env` for the process
being spawned under Landlock/seccomp. Specifically, doing
`cmd.env_clear()` followed by `cmd.envs(&$env_map)` (which is arguably
the most intuitive way to do it) caused the Landlock unit tests to fail
because the processes spawned by the unit tests started failing in
unexpected ways! If we forgo `env_clear()` in favor of updating env vars
one at a time, the tests still pass. The comment in the code talks about
this a bit, and while I would like to investigate this more, I need to
move on for the moment, but I do plan to come back to it to fully
understand what is going on. For example, this suggests that we might
not be able to spawn a C program that calls `env_clear()`, which would
be...weird. We may still have to fiddle with our Landlock config if that
is the case.
115 lines
3.5 KiB
YAML
115 lines
3.5 KiB
YAML
name: rust-ci
|
|
on:
|
|
pull_request:
|
|
branches:
|
|
- main
|
|
paths:
|
|
- "codex-rs/**"
|
|
- ".github/**"
|
|
push:
|
|
branches:
|
|
- main
|
|
|
|
workflow_dispatch:
|
|
|
|
# For CI, we build in debug (`--profile dev`) rather than release mode so we
|
|
# get signal faster.
|
|
|
|
jobs:
|
|
# CI that don't need specific targets
|
|
general:
|
|
name: Format / etc
|
|
runs-on: ubuntu-24.04
|
|
defaults:
|
|
run:
|
|
working-directory: codex-rs
|
|
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: dtolnay/rust-toolchain@1.87
|
|
with:
|
|
components: rustfmt
|
|
- name: cargo fmt
|
|
run: cargo fmt -- --config imports_granularity=Item --check
|
|
|
|
# CI to validate on different os/targets
|
|
lint_build_test:
|
|
name: ${{ matrix.runner }} - ${{ matrix.target }}
|
|
runs-on: ${{ matrix.runner }}
|
|
timeout-minutes: 30
|
|
defaults:
|
|
run:
|
|
working-directory: codex-rs
|
|
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
# Note: While Codex CLI does not support Windows today, we include
|
|
# Windows in CI to ensure the code at least builds there.
|
|
include:
|
|
- runner: macos-14
|
|
target: aarch64-apple-darwin
|
|
- runner: macos-14
|
|
target: x86_64-apple-darwin
|
|
- runner: ubuntu-24.04
|
|
target: x86_64-unknown-linux-musl
|
|
- runner: ubuntu-24.04
|
|
target: x86_64-unknown-linux-gnu
|
|
- runner: windows-latest
|
|
target: x86_64-pc-windows-msvc
|
|
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: dtolnay/rust-toolchain@1.87
|
|
with:
|
|
targets: ${{ matrix.target }}
|
|
components: clippy
|
|
|
|
- uses: actions/cache@v4
|
|
with:
|
|
path: |
|
|
~/.cargo/bin/
|
|
~/.cargo/registry/index/
|
|
~/.cargo/registry/cache/
|
|
~/.cargo/git/db/
|
|
${{ github.workspace }}/codex-rs/target/
|
|
key: cargo-${{ matrix.runner }}-${{ matrix.target }}-${{ hashFiles('**/Cargo.lock') }}
|
|
|
|
- if: ${{ matrix.target == 'x86_64-unknown-linux-musl' }}
|
|
name: Install musl build tools
|
|
run: |
|
|
sudo apt install -y musl-tools pkg-config
|
|
|
|
- name: cargo clippy
|
|
id: clippy
|
|
continue-on-error: true
|
|
run: cargo clippy --target ${{ matrix.target }} --all-features --tests -- -D warnings
|
|
|
|
# Running `cargo build` from the workspace root builds the workspace using
|
|
# the union of all features from third-party crates. This can mask errors
|
|
# where individual crates have underspecified features. To avoid this, we
|
|
# run `cargo build` for each crate individually, though because this is
|
|
# slower, we only do this for the x86_64-unknown-linux-gnu target.
|
|
- name: cargo build individual crates
|
|
id: build
|
|
if: ${{ matrix.target == 'x86_64-unknown-linux-gnu' }}
|
|
continue-on-error: true
|
|
run: find . -name Cargo.toml -mindepth 2 -maxdepth 2 -print0 | xargs -0 -n1 -I{} bash -c 'cd "$(dirname "{}")" && cargo build'
|
|
|
|
- name: cargo test
|
|
id: test
|
|
continue-on-error: true
|
|
run: cargo test --all-features --target ${{ matrix.target }}
|
|
env:
|
|
RUST_BACKTRACE: 1
|
|
|
|
# Fail the job if any of the previous steps failed.
|
|
- name: verify all steps passed
|
|
if: |
|
|
steps.clippy.outcome == 'failure' ||
|
|
steps.build.outcome == 'failure' ||
|
|
steps.test.outcome == 'failure'
|
|
run: |
|
|
echo "One or more checks failed (clippy, build, or test). See logs for details."
|
|
exit 1
|