Also: fixed the contents of the `APPLE_CERTIFICATE_P12` and
`APPLE_CERTIFICATE_PASSWORD` secrets, so the code-signing step will use
the right certificate now.
# External (non-OpenAI) Pull Request Requirements
Before opening this Pull Request, please read the dedicated
"Contributing" markdown file or your PR may be closed:
https://github.com/openai/codex/blob/main/docs/contributing.md
If your PR conforms to our contribution guidelines, replace this text
with a detailed and high quality description of your changes.
Apparently we were not running our `pnpm run prettier` check in CI, so
many files that were covered by the existing Prettier check were not
well-formatted.
This updates CI and formats the files.
# External (non-OpenAI) Pull Request Requirements
Before opening this Pull Request, please read the dedicated
"Contributing" markdown file or your PR may be closed:
https://github.com/openai/codex/blob/main/docs/contributing.md
If your PR conforms to our contribution guidelines, replace this text
with a detailed and high quality description of your changes.
There was a bit of copypasta I put up with when were publishing two
packages to npm, but now that it's three, I created some more scripts to
consolidate things.
With this change, I ran:
```shell
./scripts/stage_npm_packages.py --release-version 0.43.0-alpha.8 --package codex --package codex-responses-api-proxy --package codex-sdk
```
Indeed when it finished, I ended up with:
```shell
$ tree dist
dist
└── npm
├── codex-npm-0.43.0-alpha.8.tgz
├── codex-responses-api-proxy-npm-0.43.0-alpha.8.tgz
└── codex-sdk-npm-0.43.0-alpha.8.tgz
$ tar tzvf dist/npm/codex-sdk-npm-0.43.0-alpha.8.tgz
-rwxr-xr-x 0 0 0 25476720 Oct 26 1985 package/vendor/aarch64-apple-darwin/codex/codex
-rwxr-xr-x 0 0 0 29871400 Oct 26 1985 package/vendor/aarch64-unknown-linux-musl/codex/codex
-rwxr-xr-x 0 0 0 28368096 Oct 26 1985 package/vendor/x86_64-apple-darwin/codex/codex
-rwxr-xr-x 0 0 0 36029472 Oct 26 1985 package/vendor/x86_64-unknown-linux-musl/codex/codex
-rw-r--r-- 0 0 0 10926 Oct 26 1985 package/LICENSE
-rw-r--r-- 0 0 0 30187520 Oct 26 1985 package/vendor/aarch64-pc-windows-msvc/codex/codex.exe
-rw-r--r-- 0 0 0 35277824 Oct 26 1985 package/vendor/x86_64-pc-windows-msvc/codex/codex.exe
-rw-r--r-- 0 0 0 4842 Oct 26 1985 package/dist/index.js
-rw-r--r-- 0 0 0 1347 Oct 26 1985 package/package.json
-rw-r--r-- 0 0 0 9867 Oct 26 1985 package/dist/index.js.map
-rw-r--r-- 0 0 0 12 Oct 26 1985 package/README.md
-rw-r--r-- 0 0 0 4287 Oct 26 1985 package/dist/index.d.ts
```
This PR expands `.github/workflows/rust-release.yml` so that it also
builds and publishes the `npm` module for
`@openai/codex-responses-api-proxy` in addition to `@openai/codex`. Note
both `npm` modules are similar, in that they each contain a single `.js`
file that is a thin launcher around the appropriate native executable.
(Since we have a minimal dependency on Node.js, I also lowered the
minimum version from 20 to 16 and verified that works on my machine.)
As part of this change, we tighten up some of the docs around
`codex-responses-api-proxy` and ensure the details regarding protecting
the `OPENAI_API_KEY` in memory match the implementation.
To test the `npm` build process, I ran:
```
./codex-cli/scripts/build_npm_package.py --package codex-responses-api-proxy --version 0.43.0-alpha.3
```
which stages the `npm` module for `@openai/codex-responses-api-proxy` in
a temp directory, using the binary artifacts from
https://github.com/openai/codex/releases/tag/rust-v0.43.0-alpha.3.
This should make the `codex-responses-api-proxy` binaries available for
all platforms in a GitHub Release as well as a corresponding DotSlash
file.
Making `codex-responses-api-proxy` available as an `npm` module will be
done in a follow-up PR.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/4404).
* __->__ #4406
* #4404
* #4403
This updates our release process so that when we build an alpha of the
Codex CLI (as determined by pushing a tag of the format
`rust-v<cli-version>-alpha.<alpha-version>`), we will now publish the
corresponding npm module publicly, but under the `alpha` tag. As you can
see, this PR adds `--tag alpha` to the `npm publish` command, as
appropriate.
We try to ensure ripgrep (`rg`) is provided with Codex.
- For `brew`, we declare it as a dependency of our formula:
08d82d8b00/Formula/c/codex.rb (L24)
- For `npm`, we declare `@vscode/ripgrep` as a dependency, which
installs the platform-specific binary as part of a `postinstall` script:
fdb8dadcae/codex-cli/package.json (L22)
- Users who download the CLI directly from GitHub Releases are on their
own.
In practice, I have seen `@vscode/ripgrep` fail on occasion. Here is a
trace from a GitHub workflow:
```
npm error code 1
npm error path /Users/runner/hostedtoolcache/node/20.19.5/arm64/lib/node_modules/@openai/codex/node_modules/@vscode/ripgrep
npm error command failed
npm error command sh -c node ./lib/postinstall.js
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Download attempt 1 failed, retrying in 2 seconds...
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Download attempt 2 failed, retrying in 4 seconds...
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Download attempt 3 failed, retrying in 8 seconds...
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Download attempt 4 failed, retrying in 16 seconds...
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Error: Request failed: 403
```
To eliminate this error, this PR changes things so that we vendor the
`rg` binary into https://www.npmjs.com/package/@openai/codex so it is
guaranteed to be included when a user runs `npm i -g @openai/codex`.
The downside of this approach is the increase in package size: we
include the `rg` executable for six architectures (in addition to the
six copies of `codex` we already include). In a follow-up, I plan to add
support for "slices" of our npm module, so that soon users will be able
to do:
```
npm install -g @openai/codex@aarch64-apple-darwin
```
Admittedly, this is a sizable change and I tried to clean some things up
in the process:
- `install_native_deps.sh` has been replaced by `install_native_deps.py`
- `stage_release.sh` and `stage_rust_release.py` has been replaced by
`build_npm_package.py`
We now vendor in a DotSlash file for ripgrep (as a modest attempt to
facilitate local testing) and then build up the extension by:
- creating a temp directory and copying `package.json` over to it with
the target value for `"version"`
- finding the GitHub workflow that corresponds to the
`--release-version` and copying the various `codex` artifacts to
respective `vendor/TARGET_TRIPLE/codex` folder
- downloading the `rg` artifacts specified in the DotSlash file and
copying them over to the respective `vendor/TARGET_TRIPLE/path` folder
- if `--pack-output` is specified, runs `npm pack` on the temp directory
To test, I downloaded the artifact produced by this CI job:
https://github.com/openai/codex/actions/runs/17961595388/job/51085840022?pr=3660
and verified that `node ./bin/codex.js 'which -a rg'` worked as
intended.
Often, `gh` infers `--repo` when it is run from a Git clone, but our
`publish-npm` step is designed to avoid the overhead of cloning the
repo, so add the `--repo` option explicitly to fix things.
The build for `v0.37.0-alpha.3` failed on the `Create GitHub Release`
step:
https://github.com/openai/codex/actions/runs/17786866086/job/50556513221
with:
```
⚠️ GitHub release failed with status: 403
{"message":"Resource not accessible by integration","documentation_url":"https://docs.github.com/rest/releases/releases#create-a-release","status":"403"}
Skip retry — your GitHub token/PAT does not have the required permission to create a release
```
I believe I should have not introduced a top-level `permissions` for the
workflow in https://github.com/openai/codex/pull/3431 because that
affected the `permissions` for each job in the workflow.
This PR introduces `publish-npm` as its own job, which allows us to:
- consolidate all the Node.js-related steps required for publishing
- limit the reach of the `id-token: write` permission
- skip it altogether if is an alpha build
With this PR, each of `release`, `publish-npm`, and `update-branch` has
an explicit `permissions` block.
Apparently `-F` is the correct thing to use. From the code sample on
https://docs.github.com/en/rest/git/refs?apiVersion=2022-11-28#update-a-reference
```shell
gh api \
--method PATCH \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
/repos/OWNER/REPO/git/refs/REF \
-f 'sha=aa218f56b14c9653891f9e74264a383fa43fefbd' -F "force=true"
```
Also, I ran the following locally and verified it worked:
```shell
export GITHUB_REPOSITORY=openai/codex
export GITHUB_SHA=305252b2fb2d57bb40a9e4bad269db9a761f7099
gh api \
repos/${GITHUB_REPOSITORY}/git/refs/heads/latest-alpha-cli \
-X PATCH \
-f sha="${GITHUB_SHA}" \
-F force=true
```
`$GITHUB_REPOSITORY` and `$GITHUB_SHA` should already be available as
environment variables for the `run` step without having to be redeclared
in the `env` section.
This updates `rust-release.yml` so that the last step of creating a
release entails updating the `latest-alpha-cli` branch to point to the
tag used to create the latest release. This will facilitate building
automation to identify the most recent alpha release of Codex CLI
(though note this branch could also point to an official release, as it
is implemented today).
This introduces a new job, `update-branch`, which depends on the
`release` job. I made it separate from the `release` job because
`update-branch` needs the `contents: write` permission, so this limits
the amount of work we do with that permission.
Note I also created a branch protection rule for `latest-alpha-cli`
that:
- specifies repository admins as the only members of the bypass list
- only those with bypass permissions can create, update, or delete this
branch
- this branch requires a linear history
- note that force pushes _are_ allowed
This is the first step in fixing
https://github.com/openai/codex/issues/3098.
As a follow-up to https://github.com/openai/codex/pull/3439, this adds a
CI job to ensure the codegen script has to be updated in order to change
`codex-rs/mcp-types/src/lib.rs`.
I started looking at https://nexte.st/ because I was interested in a
test harness that lets a test dynamically declare itself "skipped,"
which would be a nice alternative to this pattern:
4c46490e53/codex-rs/core/tests/suite/cli_stream.rs (L22-L27)
ChatGPT pointed me at https://nexte.st/, which also claims to be "up to
3x as fast as cargo test." Locally, in `codex-rs`, I see
- `cargo nextest run` finishes in 19s
- `cargo test` finishes in 37s
Though looking at CI, the wins are quite as big, presumably because my
laptop has more cores than our GitHub runners (which is a separate
issue...). Comparing the [CI jobs from this
PR](https://github.com/openai/codex/actions/runs/17561325162/job/49878216246?pr=3323)
with that of a [recent open
PR](https://github.com/openai/codex/actions/runs/17561066581/job/49877342753?pr=3321):
| | `cargo test` | `cargo nextest` |
| ----------------------------------------------- | ------------ |
--------------- |
| `macos-14 - aarch64-apple-darwin` | 2m16s | 1m51s |
| `macos-14 - aarch64-apple-darwin` | 5m04s | 3m44s |
| `ubuntu-24.04 - x86_64-unknown-linux-musl` | 2m02s | 1m56s |
| `ubuntu-24.04-arm - aarch64-unknown-linux-musl` | 2m01s | 1m35s |
| `windows-latest - x86_64-pc-windows-msvc` | 3m07s | 2m53s |
| `windows-11-arm - aarch64-pc-windows-msvc` | 3m10s | 2m45s |
I thought that, to start, we would only make this change in CI before
declaring it the "official" way for the team to run the test suite.
Though unfortunately, I do not believe that `cargo nextest` _actually_
supports a dynamic skip feature, so I guess I'll have to keep looking?
Some related discussions:
- https://internals.rust-lang.org/t/pre-rfc-skippable-tests/14611
- https://internals.rust-lang.org/t/skippable-tests/21260
