feat: make incremental indexing configurable via CLI and workflow
Added full control over incremental indexing behavior: **CLI Changes:** - Added `--incremental` flag (default: true) - Added `--full` flag to disable incremental mode - Updated interactive prompt to ask about incremental mode **Function Changes:** - Updated buildIndex(force, mode, incremental) signature - Added incremental parameter with default value true - Conditional logic: if incremental=true, skip unchanged repos; else re-index all - Added console logging to show incremental mode status **Workflow Changes:** - Added `incremental` input (boolean, default: true) - Passes incremental setting to buildIndex via environment variable - Defaults to true for scheduled (cron) runs **Usage Examples:** ```bash # CLI - incremental mode (default) ./awesome index # CLI - force full re-index ./awesome index --full # CLI - explicit incremental ./awesome index --incremental # Workflow - incremental (default) gh workflow run build-database.yml # Workflow - full re-index gh workflow run build-database.yml -f incremental=false ``` This makes incremental indexing opt-out instead of hardcoded, giving users full control over indexing behavior. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
19
.github/workflows/build-database.yml
vendored
19
.github/workflows/build-database.yml
vendored
@@ -14,6 +14,11 @@ on:
|
||||
options:
|
||||
- full
|
||||
- sample
|
||||
incremental:
|
||||
description: 'Incremental mode (skip unchanged repos)'
|
||||
required: false
|
||||
default: true
|
||||
type: boolean
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
@@ -49,6 +54,7 @@ jobs:
|
||||
env:
|
||||
CI: true
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
INCREMENTAL: ${{ github.event.inputs.incremental || 'true' }}
|
||||
run: |
|
||||
# Pass env vars!
|
||||
CI=${CI:-false}
|
||||
@@ -56,9 +62,15 @@ jobs:
|
||||
START_TIME=$(date -u +"%Y-%m-%d %H:%M:%S UTC")
|
||||
echo "start_time=$START_TIME" >> $GITHUB_OUTPUT
|
||||
|
||||
# Determine index mode
|
||||
# Determine index mode and incremental setting
|
||||
INDEX_MODE="${{ github.event.inputs.index_mode || 'full' }}"
|
||||
INCREMENTAL="${{ github.event.inputs.incremental }}"
|
||||
# Default to true if not specified (for scheduled runs)
|
||||
if [ -z "$INCREMENTAL" ]; then
|
||||
INCREMENTAL="true"
|
||||
fi
|
||||
echo "Index mode: $INDEX_MODE"
|
||||
echo "Incremental: $INCREMENTAL"
|
||||
|
||||
# Build the index in non-interactive mode (350m timeout, job timeout is 360m)
|
||||
timeout 350m node -e "
|
||||
@@ -79,8 +91,9 @@ jobs:
|
||||
console.warn('⚠️ WARNING: No GitHub token found! Rate limit will be 60/hour instead of 5000/hour');
|
||||
}
|
||||
|
||||
// Build index
|
||||
await indexer.buildIndex(false, '${INDEX_MODE}');
|
||||
// Build index with incremental flag
|
||||
const incremental = process.env.INCREMENTAL === 'true';
|
||||
await indexer.buildIndex(false, '${INDEX_MODE}', incremental);
|
||||
|
||||
// Close database
|
||||
db.close();
|
||||
|
||||
Reference in New Issue
Block a user