feat: make incremental indexing configurable via CLI and workflow

Added full control over incremental indexing behavior:

**CLI Changes:**
- Added `--incremental` flag (default: true)
- Added `--full` flag to disable incremental mode
- Updated interactive prompt to ask about incremental mode

**Function Changes:**
- Updated buildIndex(force, mode, incremental) signature
- Added incremental parameter with default value true
- Conditional logic: if incremental=true, skip unchanged repos; else re-index all
- Added console logging to show incremental mode status

**Workflow Changes:**
- Added `incremental` input (boolean, default: true)
- Passes incremental setting to buildIndex via environment variable
- Defaults to true for scheduled (cron) runs

**Usage Examples:**
```bash
# CLI - incremental mode (default)
./awesome index

# CLI - force full re-index
./awesome index --full

# CLI - explicit incremental
./awesome index --incremental

# Workflow - incremental (default)
gh workflow run build-database.yml

# Workflow - full re-index
gh workflow run build-database.yml -f incremental=false
```

This makes incremental indexing opt-out instead of hardcoded, giving users full control over indexing behavior.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
valknarness
2025-10-28 09:59:47 +01:00
parent 98ddac97e8
commit 99cf83330c
3 changed files with 51 additions and 12 deletions

View File

@@ -14,6 +14,11 @@ on:
options:
- full
- sample
incremental:
description: 'Incremental mode (skip unchanged repos)'
required: false
default: true
type: boolean
permissions:
contents: read
@@ -49,6 +54,7 @@ jobs:
env:
CI: true
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
INCREMENTAL: ${{ github.event.inputs.incremental || 'true' }}
run: |
# Pass env vars!
CI=${CI:-false}
@@ -56,9 +62,15 @@ jobs:
START_TIME=$(date -u +"%Y-%m-%d %H:%M:%S UTC")
echo "start_time=$START_TIME" >> $GITHUB_OUTPUT
# Determine index mode
# Determine index mode and incremental setting
INDEX_MODE="${{ github.event.inputs.index_mode || 'full' }}"
INCREMENTAL="${{ github.event.inputs.incremental }}"
# Default to true if not specified (for scheduled runs)
if [ -z "$INCREMENTAL" ]; then
INCREMENTAL="true"
fi
echo "Index mode: $INDEX_MODE"
echo "Incremental: $INCREMENTAL"
# Build the index in non-interactive mode (350m timeout, job timeout is 360m)
timeout 350m node -e "
@@ -79,8 +91,9 @@ jobs:
console.warn('⚠️ WARNING: No GitHub token found! Rate limit will be 60/hour instead of 5000/hour');
}
// Build index
await indexer.buildIndex(false, '${INDEX_MODE}');
// Build index with incremental flag
const incremental = process.env.INCREMENTAL === 'true';
await indexer.buildIndex(false, '${INDEX_MODE}', incremental);
// Close database
db.close();