af37785bca
We try to ensure ripgrep (`rg`) is provided with Codex. - For `brew`, we declare it as a dependency of our formula:08d82d8b00/Formula/c/codex.rb (L24)- For `npm`, we declare `@vscode/ripgrep` as a dependency, which installs the platform-specific binary as part of a `postinstall` script:fdb8dadcae/codex-cli/package.json (L22)- Users who download the CLI directly from GitHub Releases are on their own. In practice, I have seen `@vscode/ripgrep` fail on occasion. Here is a trace from a GitHub workflow: ``` npm error code 1 npm error path /Users/runner/hostedtoolcache/node/20.19.5/arm64/lib/node_modules/@openai/codex/node_modules/@vscode/ripgrep npm error command failed npm error command sh -c node ./lib/postinstall.js npm error Finding release for v13.0.0-13 npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13 npm error Deleting invalid download cache npm error Download attempt 1 failed, retrying in 2 seconds... npm error Finding release for v13.0.0-13 npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13 npm error Deleting invalid download cache npm error Download attempt 2 failed, retrying in 4 seconds... npm error Finding release for v13.0.0-13 npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13 npm error Deleting invalid download cache npm error Download attempt 3 failed, retrying in 8 seconds... npm error Finding release for v13.0.0-13 npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13 npm error Deleting invalid download cache npm error Download attempt 4 failed, retrying in 16 seconds... npm error Finding release for v13.0.0-13 npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13 npm error Deleting invalid download cache npm error Error: Request failed: 403 ``` To eliminate this error, this PR changes things so that we vendor the `rg` binary into https://www.npmjs.com/package/@openai/codex so it is guaranteed to be included when a user runs `npm i -g @openai/codex`. The downside of this approach is the increase in package size: we include the `rg` executable for six architectures (in addition to the six copies of `codex` we already include). In a follow-up, I plan to add support for "slices" of our npm module, so that soon users will be able to do: ``` npm install -g @openai/codex@aarch64-apple-darwin ``` Admittedly, this is a sizable change and I tried to clean some things up in the process: - `install_native_deps.sh` has been replaced by `install_native_deps.py` - `stage_release.sh` and `stage_rust_release.py` has been replaced by `build_npm_package.py` We now vendor in a DotSlash file for ripgrep (as a modest attempt to facilitate local testing) and then build up the extension by: - creating a temp directory and copying `package.json` over to it with the target value for `"version"` - finding the GitHub workflow that corresponds to the `--release-version` and copying the various `codex` artifacts to respective `vendor/TARGET_TRIPLE/codex` folder - downloading the `rg` artifacts specified in the DotSlash file and copying them over to the respective `vendor/TARGET_TRIPLE/path` folder - if `--pack-output` is specified, runs `npm pack` on the temp directory To test, I downloaded the artifact produced by this CI job: https://github.com/openai/codex/actions/runs/17961595388/job/51085840022?pr=3660 and verified that `node ./bin/codex.js 'which -a rg'` worked as intended.
60 lines
1.7 KiB
YAML
60 lines
1.7 KiB
YAML
name: ci
|
|
|
|
on:
|
|
pull_request: { branches: [main] }
|
|
push: { branches: [main] }
|
|
|
|
jobs:
|
|
build-test:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
env:
|
|
NODE_OPTIONS: --max-old-space-size=4096
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v5
|
|
|
|
- name: Setup pnpm
|
|
uses: pnpm/action-setup@v4
|
|
with:
|
|
run_install: false
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v5
|
|
with:
|
|
node-version: 22
|
|
|
|
- name: Install dependencies
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
# build_npm_package.py requires DotSlash when staging releases.
|
|
- uses: facebook/install-dotslash@v2
|
|
|
|
- name: Stage npm package
|
|
env:
|
|
GH_TOKEN: ${{ github.token }}
|
|
run: |
|
|
set -euo pipefail
|
|
CODEX_VERSION=0.40.0
|
|
PACK_OUTPUT="${RUNNER_TEMP}/codex-npm.tgz"
|
|
python3 ./codex-cli/scripts/build_npm_package.py \
|
|
--release-version "$CODEX_VERSION" \
|
|
--pack-output "$PACK_OUTPUT"
|
|
echo "PACK_OUTPUT=$PACK_OUTPUT" >> "$GITHUB_ENV"
|
|
|
|
- name: Upload staged npm package artifact
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: codex-npm-staging
|
|
path: ${{ env.PACK_OUTPUT }}
|
|
|
|
- name: Ensure root README.md contains only ASCII and certain Unicode code points
|
|
run: ./scripts/asciicheck.py README.md
|
|
- name: Check root README ToC
|
|
run: python3 scripts/readme_toc.py README.md
|
|
|
|
- name: Ensure codex-cli/README.md contains only ASCII and certain Unicode code points
|
|
run: ./scripts/asciicheck.py codex-cli/README.md
|
|
- name: Check codex-cli/README ToC
|
|
run: python3 scripts/readme_toc.py codex-cli/README.md
|