chore: migrate to pnpm for improved monorepo management (#287)

# Migrate to pnpm for improved monorepo management

## Summary
This PR migrates the Codex repository from npm to pnpm, providing faster
dependency installation, better disk space usage, and improved monorepo
management.

## Changes
- Added `pnpm-workspace.yaml` to define workspace packages
- Added `.npmrc` with optimal pnpm configuration
- Updated root package.json with workspace scripts
- Moved resolutions and overrides to the root package.json
- Updated scripts to use pnpm instead of npm
- Added documentation for the migration
- Updated GitHub Actions workflow for pnpm

## Benefits
- **Faster installations**: pnpm is significantly faster than npm
- **Disk space savings**: pnpm's content-addressable store avoids
duplication
- **Strict dependency management**: prevents phantom dependencies
- **Simplified monorepo management**: better workspace coordination
- **Preparation for Turborepo**: as discussed, this is the first step
before adding Turborepo

## Testing
- Verified that `pnpm install` works correctly
- Verified that `pnpm run build` completes successfully
- Ensured all existing functionality is preserved

## Documentation
Added a detailed migration guide in `PNPM_MIGRATION.md` explaining:
- Why we're migrating to pnpm
- How to use pnpm with this repository
- Common commands and workspace-specific commands
- Monorepo structure and configuration

## Next Steps
As discussed, once this change is stable, we can consider adding
Turborepo as a follow-up enhancement.
This commit is contained in:
Alpha Diop
2025-04-18 23:25:15 +00:00
committed by GitHub
parent 9a046dfcaa
commit e2fe2572ba
12 changed files with 152 additions and 7774 deletions

View File

@@ -19,40 +19,47 @@ jobs:
with:
node-version: 22
# Run codex-cli/ tasks first because they are higher signal.
- name: Setup pnpm
uses: pnpm/action-setup@v4
with:
version: 10.8.1
run_install: false
- name: Install dependencies (codex-cli)
working-directory: codex-cli
run: npm ci
- name: Check formatting (codex-cli)
working-directory: codex-cli
run: npm run format
- name: Run tests (codex-cli)
working-directory: codex-cli
run: npm run test
- name: Lint (codex-cli)
working-directory: codex-cli
- name: Get pnpm store directory
id: pnpm-cache
shell: bash
run: |
npm run lint -- \
echo "store_path=$(pnpm store path --silent)" >> $GITHUB_OUTPUT
- name: Setup pnpm cache
uses: actions/cache@v4
with:
path: ${{ steps.pnpm-cache.outputs.store_path }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: |
${{ runner.os }}-pnpm-store-
- name: Install dependencies
run: pnpm install
# Run all tasks using workspace filters
- name: Check formatting
run: pnpm run format
- name: Run tests
run: pnpm run test
- name: Lint
run: |
pnpm --filter @openai/codex exec -- eslint src tests --ext ts --ext tsx \
--report-unused-disable-directives \
--rule "no-console:error" \
--rule "no-debugger:error" \
--max-warnings=-1
- name: Typecheck (codex-cli)
working-directory: codex-cli
run: npm run typecheck
- name: Type-check
run: pnpm run typecheck
- name: Build (codex-cli)
working-directory: codex-cli
run: npm run build
# Run formatting checks in the root directory last.
- name: Install dependencies (root)
run: npm ci
- name: Check formatting (root)
run: npm run format
- name: Build
run: pnpm run build

6
.gitignore vendored
View File

@@ -1,5 +1,11 @@
# deps
# Node.js dependencies
node_modules
.pnpm-store
.pnpm-debug.log
# Keep pnpm-lock.yaml
!pnpm-lock.yaml
# build
dist/

4
.npmrc Normal file
View File

@@ -0,0 +1,4 @@
shamefully-hoist=true
strict-peer-dependencies=false
node-linker=hoisted
prefer-workspace-packages=true

View File

@@ -1,2 +1,3 @@
/codex-cli/dist
/codex-cli/node_modules
pnpm-lock.yaml

70
PNPM_MIGRATION.md Normal file
View File

@@ -0,0 +1,70 @@
# Migration to pnpm
This project has been migrated from npm to pnpm to improve dependency management and developer experience.
## Why pnpm?
- **Faster installation**: pnpm is significantly faster than npm and yarn
- **Disk space savings**: pnpm uses a content-addressable store to avoid duplication
- **Phantom dependency prevention**: pnpm creates a strict node_modules structure
- **Native workspaces support**: simplified monorepo management
## How to use pnpm
### Installation
```bash
# Global installation of pnpm
npm install -g pnpm@10.8.1
# Or with corepack (available with Node.js 22+)
corepack enable
corepack prepare pnpm@10.8.1 --activate
```
### Common commands
| npm command | pnpm equivalent |
| --------------- | ---------------- |
| `npm install` | `pnpm install` |
| `npm run build` | `pnpm run build` |
| `npm test` | `pnpm test` |
| `npm run lint` | `pnpm run lint` |
### Workspace-specific commands
| Action | Command |
| ------------------------------------------ | ---------------------------------------- |
| Run a command in a specific package | `pnpm --filter @openai/codex run build` |
| Install a dependency in a specific package | `pnpm --filter @openai/codex add lodash` |
| Run a command in all packages | `pnpm -r run test` |
## Monorepo structure
```
codex/
├── pnpm-workspace.yaml # Workspace configuration
├── .npmrc # pnpm configuration
├── package.json # Root dependencies and scripts
├── codex-cli/ # Main package
│ └── package.json # codex-cli specific dependencies
└── docs/ # Documentation (future package)
```
## Configuration files
- **pnpm-workspace.yaml**: Defines the packages included in the monorepo
- **.npmrc**: Configures pnpm behavior
- **Root package.json**: Contains shared scripts and dependencies
## CI/CD
CI/CD workflows have been updated to use pnpm instead of npm. Make sure your CI environments use pnpm 10.8.1 or higher.
## Known issues
If you encounter issues with pnpm, try the following solutions:
1. Remove the `node_modules` folder and `pnpm-lock.yaml` file, then run `pnpm install`
2. Make sure you're using pnpm 10.8.1 or higher
3. Verify that Node.js 22 or higher is installed

File diff suppressed because it is too large Load Diff

View File

@@ -22,8 +22,8 @@
"build:dev": "NODE_ENV=development node build.mjs --dev && NODE_OPTIONS=--enable-source-maps node dist/cli-dev.js",
"release:readme": "cp ../README.md ./README.md",
"release:version": "TS=$(date +%y%m%d%H%M) && sed -E -i'' -e \"s/\\\"0\\.1\\.[0-9]{10}\\\"/\\\"0.1.${TS}\\\"/g\" package.json src/utils/session.ts",
"release:build-and-publish": "npm run build && npm publish",
"release": "npm run release:readme && npm run release:version && npm install && npm run release:build-and-publish",
"release:build-and-publish": "pmpm run build && pmpm publish",
"release": "pnpm run release:readme && pnpm run release:version && pnpm install && pnpm run release:build-and-publish",
"prepare": "husky",
"pre-commit": "lint-staged"
},
@@ -79,14 +79,6 @@
"vitest": "^3.0.9",
"whatwg-url": "^14.2.0"
},
"resolutions": {
"braces": "^3.0.3",
"micromatch": "^4.0.8",
"semver": "^7.7.1"
},
"overrides": {
"punycode": "^2.3.1"
},
"repository": {
"type": "git",
"url": "https://github.com/openai/codex"

View File

@@ -496,7 +496,6 @@ export class AgentLoop {
if (this.model.startsWith("o")) {
reasoning = { effort: "high" };
if (this.model === "o3" || this.model === "o4-mini") {
// @ts-expect-error waiting for API type update
reasoning.summary = "auto";
}
}

4
lint-staged.config.mjs Normal file
View File

@@ -0,0 +1,4 @@
export default {
"*.{js,jsx,ts,tsx}": ["pnpm prettier --write", "pnpm eslint --fix"],
"*.{json,md,yml,yaml}": ["pnpm prettier --write"],
};

359
package-lock.json generated
View File

@@ -1,359 +0,0 @@
{
"name": "codex",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"devDependencies": {
"git-cliff": "^2.8.0",
"prettier": "^3.5.3"
}
},
"node_modules/cross-spawn": {
"version": "7.0.6",
"resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.6.tgz",
"integrity": "sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA==",
"dev": true,
"license": "MIT",
"dependencies": {
"path-key": "^3.1.0",
"shebang-command": "^2.0.0",
"which": "^2.0.1"
},
"engines": {
"node": ">= 8"
}
},
"node_modules/execa": {
"version": "8.0.1",
"resolved": "https://registry.npmjs.org/execa/-/execa-8.0.1.tgz",
"integrity": "sha512-VyhnebXciFV2DESc+p6B+y0LjSm0krU4OgJN44qFAhBY0TJ+1V61tYD2+wHusZ6F9n5K+vl8k0sTy7PEfV4qpg==",
"dev": true,
"license": "MIT",
"dependencies": {
"cross-spawn": "^7.0.3",
"get-stream": "^8.0.1",
"human-signals": "^5.0.0",
"is-stream": "^3.0.0",
"merge-stream": "^2.0.0",
"npm-run-path": "^5.1.0",
"onetime": "^6.0.0",
"signal-exit": "^4.1.0",
"strip-final-newline": "^3.0.0"
},
"engines": {
"node": ">=16.17"
},
"funding": {
"url": "https://github.com/sindresorhus/execa?sponsor=1"
}
},
"node_modules/get-stream": {
"version": "8.0.1",
"resolved": "https://registry.npmjs.org/get-stream/-/get-stream-8.0.1.tgz",
"integrity": "sha512-VaUJspBffn/LMCJVoMvSAdmscJyS1auj5Zulnn5UoYcY531UWmdwhRWkcGKnGU93m5HSXP9LP2usOryrBtQowA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=16"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/git-cliff": {
"version": "2.8.0",
"resolved": "https://registry.npmjs.org/git-cliff/-/git-cliff-2.8.0.tgz",
"integrity": "sha512-iKF5QTXAb9+iVvmu5HpnMPWYw7fs74xkpAaRbSf29+dZaMTTNRIUST/y+Ir2S1bDUWWJNjXlwT9ZT62JuYLQnA==",
"dev": true,
"license": "MIT OR Apache-2.0",
"dependencies": {
"execa": "^8.0.1"
},
"bin": {
"git-cliff": "lib/cli/cli.js"
},
"engines": {
"node": ">=18.19 || >=20.6 || >=21"
},
"optionalDependencies": {
"git-cliff-darwin-arm64": "2.8.0",
"git-cliff-darwin-x64": "2.8.0",
"git-cliff-linux-arm64": "2.8.0",
"git-cliff-linux-x64": "2.8.0",
"git-cliff-windows-arm64": "2.8.0",
"git-cliff-windows-x64": "2.8.0"
}
},
"node_modules/git-cliff-darwin-arm64": {
"version": "2.8.0",
"resolved": "https://registry.npmjs.org/git-cliff-darwin-arm64/-/git-cliff-darwin-arm64-2.8.0.tgz",
"integrity": "sha512-rurUV2d1Z2n+c2+wUrO0gZaFb3c1G+ej0bPfKTPfde/CblxiysMkh+4dz23NrVbc8IlS5rSYv/JFGVaVSBNJRw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT OR Apache-2.0",
"optional": true,
"os": [
"darwin"
]
},
"node_modules/git-cliff-darwin-x64": {
"version": "2.8.0",
"resolved": "https://registry.npmjs.org/git-cliff-darwin-x64/-/git-cliff-darwin-x64-2.8.0.tgz",
"integrity": "sha512-Wtj+FGWZBWmeYUAGlkfz7QPz4+VVxxDPMhQ/7iwKVA3iryIX0slGfzYpqMurEFnTAMr0r+4IU3Q4O/ib7iUscg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT OR Apache-2.0",
"optional": true,
"os": [
"darwin"
]
},
"node_modules/git-cliff-linux-arm64": {
"version": "2.8.0",
"resolved": "https://registry.npmjs.org/git-cliff-linux-arm64/-/git-cliff-linux-arm64-2.8.0.tgz",
"integrity": "sha512-k4RdfMdORXyefznWlQb+7wDgo7XgQF9qg8hJC34bwyJK2sODirrGau3uTx1/9Fi37g+pAOM7wM+LYppHCTZ2bQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT OR Apache-2.0",
"optional": true,
"os": [
"linux"
]
},
"node_modules/git-cliff-linux-x64": {
"version": "2.8.0",
"resolved": "https://registry.npmjs.org/git-cliff-linux-x64/-/git-cliff-linux-x64-2.8.0.tgz",
"integrity": "sha512-FcWX4GHgodYrQlZR03fzooanStgR03JNWvyaMQB1asplQ18nlziK2UyA+PESCIxOQmeLXauqoCApfzmdtp5myg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT OR Apache-2.0",
"optional": true,
"os": [
"linux"
]
},
"node_modules/git-cliff-windows-arm64": {
"version": "2.8.0",
"resolved": "https://registry.npmjs.org/git-cliff-windows-arm64/-/git-cliff-windows-arm64-2.8.0.tgz",
"integrity": "sha512-GJSrqmBVTbMtBJI3/YCDxLviZZDgYgnKqYgquBk2u2AELAnnuWFnVFQ7ZEBUqgFF2UJu9EdV2Nv6MV8d/wnP0g==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT OR Apache-2.0",
"optional": true,
"os": [
"win32"
]
},
"node_modules/git-cliff-windows-x64": {
"version": "2.8.0",
"resolved": "https://registry.npmjs.org/git-cliff-windows-x64/-/git-cliff-windows-x64-2.8.0.tgz",
"integrity": "sha512-8jl0YMXPYjUmVygUEeQ4wf1zte3Rv8LPq1sIklUKl80XE4g2Gm/8EIWbKpUPLQH6IncRwepY6VuMgpVpPXbwNw==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT OR Apache-2.0",
"optional": true,
"os": [
"win32"
]
},
"node_modules/human-signals": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/human-signals/-/human-signals-5.0.0.tgz",
"integrity": "sha512-AXcZb6vzzrFAUE61HnN4mpLqd/cSIwNQjtNWR0euPm6y0iqx3G4gOXaIDdtdDwZmhwe82LA6+zinmW4UBWVePQ==",
"dev": true,
"license": "Apache-2.0",
"engines": {
"node": ">=16.17.0"
}
},
"node_modules/is-stream": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/is-stream/-/is-stream-3.0.0.tgz",
"integrity": "sha512-LnQR4bZ9IADDRSkvpqMGvt/tEJWclzklNgSw48V5EAaAeDd6qGvN8ei6k5p0tvxSR171VmGyHuTiAOfxAbr8kA==",
"dev": true,
"license": "MIT",
"engines": {
"node": "^12.20.0 || ^14.13.1 || >=16.0.0"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/isexe": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/isexe/-/isexe-2.0.0.tgz",
"integrity": "sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw==",
"dev": true,
"license": "ISC"
},
"node_modules/merge-stream": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/merge-stream/-/merge-stream-2.0.0.tgz",
"integrity": "sha512-abv/qOcuPfk3URPfDzmZU1LKmuw8kT+0nIHvKrKgFrwifol/doWcdA4ZqsWQ8ENrFKkd67Mfpo/LovbIUsbt3w==",
"dev": true,
"license": "MIT"
},
"node_modules/mimic-fn": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/mimic-fn/-/mimic-fn-4.0.0.tgz",
"integrity": "sha512-vqiC06CuhBTUdZH+RYl8sFrL096vA45Ok5ISO6sE/Mr1jRbGH4Csnhi8f3wKVl7x8mO4Au7Ir9D3Oyv1VYMFJw==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/npm-run-path": {
"version": "5.3.0",
"resolved": "https://registry.npmjs.org/npm-run-path/-/npm-run-path-5.3.0.tgz",
"integrity": "sha512-ppwTtiJZq0O/ai0z7yfudtBpWIoxM8yE6nHi1X47eFR2EWORqfbu6CnPlNsjeN683eT0qG6H/Pyf9fCcvjnnnQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"path-key": "^4.0.0"
},
"engines": {
"node": "^12.20.0 || ^14.13.1 || >=16.0.0"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/npm-run-path/node_modules/path-key": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/path-key/-/path-key-4.0.0.tgz",
"integrity": "sha512-haREypq7xkM7ErfgIyA0z+Bj4AGKlMSdlQE2jvJo6huWD1EdkKYV+G/T4nq0YEF2vgTT8kqMFKo1uHn950r4SQ==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/onetime": {
"version": "6.0.0",
"resolved": "https://registry.npmjs.org/onetime/-/onetime-6.0.0.tgz",
"integrity": "sha512-1FlR+gjXK7X+AsAHso35MnyN5KqGwJRi/31ft6x0M194ht7S+rWAvd7PHss9xSKMzE0asv1pyIHaJYq+BbacAQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"mimic-fn": "^4.0.0"
},
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/path-key": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/path-key/-/path-key-3.1.1.tgz",
"integrity": "sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/prettier": {
"version": "3.5.3",
"resolved": "https://registry.npmjs.org/prettier/-/prettier-3.5.3.tgz",
"integrity": "sha512-QQtaxnoDJeAkDvDKWCLiwIXkTgRhwYDEQCghU9Z6q03iyek/rxRh/2lC3HB7P8sWT2xC/y5JDctPLBIGzHKbhw==",
"dev": true,
"license": "MIT",
"bin": {
"prettier": "bin/prettier.cjs"
},
"engines": {
"node": ">=14"
},
"funding": {
"url": "https://github.com/prettier/prettier?sponsor=1"
}
},
"node_modules/shebang-command": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/shebang-command/-/shebang-command-2.0.0.tgz",
"integrity": "sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA==",
"dev": true,
"license": "MIT",
"dependencies": {
"shebang-regex": "^3.0.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/shebang-regex": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/shebang-regex/-/shebang-regex-3.0.0.tgz",
"integrity": "sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/signal-exit": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/signal-exit/-/signal-exit-4.1.0.tgz",
"integrity": "sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw==",
"dev": true,
"license": "ISC",
"engines": {
"node": ">=14"
},
"funding": {
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/strip-final-newline": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/strip-final-newline/-/strip-final-newline-3.0.0.tgz",
"integrity": "sha512-dOESqjYr96iWYylGObzd39EuNTa5VJxyvVAEm5Jnh7KGo75V43Hk1odPQkNDyXNmUR6k+gEiDVXnjB8HJ3crXw==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/which": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/which/-/which-2.0.2.tgz",
"integrity": "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA==",
"dev": true,
"license": "ISC",
"dependencies": {
"isexe": "^2.0.0"
},
"bin": {
"node-which": "bin/node-which"
},
"engines": {
"node": ">= 8"
}
}
}
}

View File

@@ -1,13 +1,34 @@
{
"name": "codex-monorepo",
"private": true,
"description": "Tools for repo-wide maintenance.",
"scripts": {
"release": "cd codex-cli && npm run release",
"release": "pnpm --filter @openai/codex run release",
"format": "prettier --check *.json *.md .github/workflows/*.yml",
"format:fix": "prettier --write *.json *.md .github/workflows/*.yml",
"build": "pnpm --filter @openai/codex run build",
"test": "pnpm --filter @openai/codex run test",
"lint": "pnpm --filter @openai/codex run lint",
"typecheck": "pnpm --filter @openai/codex run typecheck",
"prepare": "husky",
"changelog": "git-cliff --config cliff.toml --output CHANGELOG.ignore.md $LAST_RELEASE_TAG..HEAD"
},
"devDependencies": {
"git-cliff": "^2.8.0",
"husky": "^9.1.7",
"prettier": "^3.5.3"
}
},
"resolutions": {
"braces": "^3.0.3",
"micromatch": "^4.0.8",
"semver": "^7.7.1"
},
"overrides": {
"punycode": "^2.3.1"
},
"engines": {
"node": ">=22",
"pnpm": ">=9.0.0"
},
"packageManager": "pnpm@10.8.1"
}

5
pnpm-workspace.yaml Normal file
View File

@@ -0,0 +1,5 @@
packages:
- 'codex-cli'
- 'docs'
# For future packages
- 'packages/*'