feat: add VPS deployment automation with systemd templates

Add automated VPS deployment system for remote database indexing: - Interactive setup script with SSH deployment - Systemd service/timer templates for daily scheduling - Indexing script with Docker container support - Comprehensive README documentation Features: - Automated daily indexing on remote servers - Optional Docker container deployment - Configurable paths, schedules, and incremental mode - Full monitoring and management commands - Troubleshooting guide for common issues Usage: cd deploy && ./setup-vps.sh 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-29 00:20:54 +01:00
parent 99cf83330c
commit 1b62269116
5 changed files with 498 additions and 0 deletions
--- a/README.md
+++ b/README.md
@@ -349,6 +349,209 @@ The `scripts/download-db.sh` script provides an interactive interface to:

 ---

+## 🖥️ VPS Deployment (Automated Remote Indexing!)
+
+Want to run automated database builds on YOUR OWN server? We got you COVERED! Deploy to any VPS with systemd and optionally deploy to Docker containers!
+
+### Quick Start (The EASY Way!)
+
+```bash
+# From your local machine, run the interactive setup script
+cd deploy
+./setup-vps.sh
+```
+
+The script will prompt you for configuration and handle EVERYTHING:
+- ✅ Process templates with YOUR configuration
+- ✅ Copy files to remote server via SSH
+- ✅ Install and enable systemd service
+- ✅ Start the timer for daily indexing
+- ✅ Show status and helpful commands
+
+### Prerequisites
+
+**On your VPS:**
+- SSH access (root or sudo user)
+- Systemd installed (standard on modern Linux)
+- awesome CLI installed and configured
+- Docker (optional, for container deployment)
+- GitHub token configured via `awesome settings`
+
+**On your local machine:**
+- SSH key-based authentication to VPS
+- Bash shell (Linux, macOS, Windows/Git Bash)
+
+### Configuration Options
+
+The setup script will interactively prompt for:
+
+| Setting | Default | Description |
+|---------|---------|-------------|
+| SSH Host | `root@vps` | SSH connection string (user@hostname) |
+| User | `root` | User to run service as |
+| Working Directory | `/root` | Directory for scripts |
+| Awesome Command | `/usr/local/bin/awesome` | Path to awesome CLI |
+| Log File | `/var/log/awesome-index.log` | Log file location |
+| Database Source | `/root/.awesome/awesome.db` | Database file path |
+| Database Staging | `/tmp/awesome-database` | Temporary staging directory |
+| Docker Container | `awesome_app` | Container name (empty to skip) |
+| Container Path | `/home/node/.awesome` | Database path in container |
+| Schedule Time | `02:00` | Daily run time (HH:MM format) |
+| Incremental | `true` | Use incremental indexing by default |
+
+### What Gets Deployed
+
+The deployment creates:
+
+1. **`/root/scripts/awesome-index.sh`** - Main indexing script that:
+   - Runs `awesome index --incremental` (or `--full`)
+   - Logs output to configured log file
+   - Stages database in temporary directory
+   - Optionally deploys to Docker container
+   - Verifies deployment and cleans up
+
+2. **`/etc/systemd/system/awesome-index.service`** - Systemd service that:
+   - Runs as configured user
+   - Has restart-on-failure handling
+   - Logs to systemd journal
+   - Can be triggered manually
+
+3. **`/etc/systemd/system/awesome-index.timer`** - Systemd timer that:
+   - Schedules daily runs at configured time
+   - Persistent (runs missed executions on boot)
+   - Integrates with timers.target
+
+### Monitoring & Management
+
+**View logs:**
+```bash
+# Real-time service logs
+ssh root@vps 'journalctl -u awesome-index.service -f'
+
+# View log file
+ssh root@vps 'tail -f /var/log/awesome-index.log'
+```
+
+**Check status:**
+```bash
+# Timer status
+ssh root@vps 'systemctl status awesome-index.timer'
+
+# Next scheduled run
+ssh root@vps 'systemctl list-timers awesome-index.timer'
+
+# Service status
+ssh root@vps 'systemctl status awesome-index.service'
+```
+
+**Manual trigger:**
+```bash
+# Run indexing immediately
+ssh root@vps 'systemctl start awesome-index.service'
+```
+
+**Manage timer:**
+```bash
+# Stop automatic runs
+ssh root@vps 'systemctl stop awesome-index.timer'
+
+# Disable automatic runs
+ssh root@vps 'systemctl disable awesome-index.timer'
+
+# Re-enable
+ssh root@vps 'systemctl enable --now awesome-index.timer'
+```
+
+### Docker Container Deployment
+
+If you provide a container name during setup, the script will automatically:
+1. Copy database to staging directory
+2. Check if container exists
+3. Deploy database via `docker cp`
+4. Verify deployment with `ls -lh` in container
+5. Clean up staging directory
+
+**Optional container restart** (commented by default):
+```bash
+# Edit /root/scripts/awesome-index.sh on VPS and uncomment:
+docker restart "$CONTAINER"
+```
+
+### Manual Deployment (Advanced)
+
+If you prefer manual control, you can deploy the templates yourself:
+
+```bash
+# On your local machine
+cd deploy
+
+# 1. Process templates manually
+sed -e "s|{{LOG_FILE}}|/var/log/awesome-index.log|g" \
+    -e "s|{{DB_SOURCE}}|/root/.awesome/awesome.db|g" \
+    # ... (substitute all variables)
+    awesome-index.sh.template > awesome-index.sh
+
+# 2. Copy to VPS
+scp awesome-index.sh root@vps:/root/scripts/
+scp awesome-index.service root@vps:/etc/systemd/system/
+scp awesome-index.timer root@vps:/etc/systemd/system/
+
+# 3. Enable on VPS
+ssh root@vps 'chmod +x /root/scripts/awesome-index.sh'
+ssh root@vps 'systemctl daemon-reload'
+ssh root@vps 'systemctl enable --now awesome-index.timer'
+```
+
+### Troubleshooting
+
+**Service fails to start:**
+```bash
+# Check service logs
+ssh root@vps 'journalctl -u awesome-index.service -n 50'
+
+# Verify script is executable
+ssh root@vps 'ls -l /root/scripts/awesome-index.sh'
+
+# Test script manually
+ssh root@vps '/root/scripts/awesome-index.sh'
+```
+
+**Docker deployment fails:**
+```bash
+# Verify container exists and is running
+ssh root@vps 'docker ps -a | grep awesome_app'
+
+# Check container path exists
+ssh root@vps 'docker exec awesome_app ls -la /home/node/.awesome'
+
+# Test docker cp manually
+ssh root@vps 'docker cp /root/.awesome/awesome.db awesome_app:/home/node/.awesome/'
+```
+
+**Indexing takes too long / times out:**
+```bash
+# Switch to incremental mode (edit service or run with flag)
+ssh root@vps 'sed -i "s/--full/--incremental/g" /root/scripts/awesome-index.sh'
+
+# Verify GitHub token is configured
+ssh root@vps 'awesome settings'
+```
+
+**Timer not running:**
+```bash
+# Verify timer is enabled and active
+ssh root@vps 'systemctl is-enabled awesome-index.timer'
+ssh root@vps 'systemctl is-active awesome-index.timer'
+
+# Check timer configuration
+ssh root@vps 'systemctl cat awesome-index.timer'
+
+# Reload systemd if you made changes
+ssh root@vps 'systemctl daemon-reload'
+```
+
+---
+
 ## 🌟 What Makes This Cosmically AWESOME?

 1. **🎨 Funkadelic Aesthetics** - Purple/pink/gold theme that'll make your EYES happy!
--- a/deploy/awesome-index.service.template
+++ b/deploy/awesome-index.service.template
@@ -0,0 +1,19 @@
+[Unit]
+Description=Awesome Database Indexing
+After=network.target docker.service
+
+[Service]
+Type=oneshot
+User={{USER}}
+WorkingDirectory={{WORK_DIR}}
+ExecStart={{SCRIPT_PATH}}
+StandardOutput=journal
+StandardError=journal
+
+# Restart on failure
+Restart=on-failure
+RestartSec=300
+
+# Resource limits (optional)
+# MemoryMax=2G
+# CPUQuota=50%
--- a/deploy/awesome-index.sh.template
+++ b/deploy/awesome-index.sh.template
@@ -0,0 +1,84 @@
+#!/bin/bash
+set -e
+
+# Configuration (filled by setup script)
+LOG_FILE="{{LOG_FILE}}"
+DB_SOURCE="{{DB_SOURCE}}"
+DB_STAGING="{{DB_STAGING}}"
+CONTAINER="{{CONTAINER}}"
+CONTAINER_PATH="{{CONTAINER_PATH}}"
+AWESOME_CMD="{{AWESOME_CMD}}"
+INCREMENTAL="{{INCREMENTAL}}"
+
+log() {
+    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1" | tee -a "$LOG_FILE"
+}
+
+log "============================================"
+log "Starting awesome database indexing..."
+log "Incremental mode: $INCREMENTAL"
+
+# Run indexing
+cd "$(dirname "$DB_SOURCE")"
+if [ "$INCREMENTAL" = "true" ]; then
+    INDEX_CMD="$AWESOME_CMD index --incremental"
+else
+    INDEX_CMD="$AWESOME_CMD index --full"
+fi
+
+log "Running: $INDEX_CMD"
+if $INDEX_CMD 2>&1 | tee -a "$LOG_FILE"; then
+    log "✓ Indexing completed successfully"
+else
+    EXIT_CODE=$?
+    log "✗ Indexing failed with exit code $EXIT_CODE"
+    exit $EXIT_CODE
+fi
+
+# Verify database exists
+if [ ! -f "$DB_SOURCE" ]; then
+    log "✗ Database file not found at $DB_SOURCE"
+    exit 1
+fi
+
+# Get database stats
+DB_SIZE=$(du -h "$DB_SOURCE" | cut -f1)
+log "Database size: $DB_SIZE"
+
+# Prepare staging directory
+log "Preparing staging directory..."
+rm -rf "$DB_STAGING"
+mkdir -p "$DB_STAGING"
+cp "$DB_SOURCE" "$DB_STAGING/"
+log "✓ Database copied to staging"
+
+# Check if Docker container exists
+if ! docker inspect "$CONTAINER" &>/dev/null; then
+    log "⚠️  Docker container '$CONTAINER' not found, skipping deployment"
+    log "✓ Process completed (indexing only)"
+    exit 0
+fi
+
+# Copy to Docker container
+log "Deploying to Docker container '$CONTAINER'..."
+if docker cp "$DB_STAGING/." "$CONTAINER:$CONTAINER_PATH/"; then
+    log "✓ Database deployed successfully"
+
+    # Verify in container
+    docker exec "$CONTAINER" ls -lh "$CONTAINER_PATH/awesome.db" 2>&1 | tee -a "$LOG_FILE" || true
+
+    # Optional: restart container to pick up new database
+    # Uncomment if your app needs a restart
+    # log "Restarting container..."
+    # docker restart "$CONTAINER"
+else
+    log "✗ Docker copy failed"
+    exit 1
+fi
+
+# Cleanup
+rm -rf "$DB_STAGING"
+log "✓ Cleanup completed"
+
+log "✓ Process completed successfully"
+log "============================================"
--- a/deploy/awesome-index.timer.template
+++ b/deploy/awesome-index.timer.template
@@ -0,0 +1,15 @@
+[Unit]
+Description=Daily Awesome Database Indexing Timer
+Requires=awesome-index.service
+
+[Timer]
+# Run daily at {{SCHEDULE_TIME}}
+OnCalendar=daily
+OnCalendar={{SCHEDULE_TIME}}
+Persistent=true
+
+# Random delay to avoid peak times (optional)
+# RandomizedDelaySec=30min
+
+[Install]
+WantedBy=timers.target
--- a/deploy/setup-vps.sh
+++ b/deploy/setup-vps.sh
@@ -0,0 +1,177 @@
+#!/bin/bash
+set -e
+
+# Colors for output
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+BLUE='\033[0;34m'
+NC='\033[0m' # No Color
+
+echo -e "${BLUE}╔════════════════════════════════════════════════════════════╗${NC}"
+echo -e "${BLUE}║  AWESOME Database VPS Deployment Setup                    ║${NC}"
+echo -e "${BLUE}╚════════════════════════════════════════════════════════════╝${NC}"
+echo ""
+
+# Get script directory
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+# Default values
+DEFAULT_SSH_HOST="root@vps"
+DEFAULT_USER="root"
+DEFAULT_WORK_DIR="/root"
+DEFAULT_AWESOME_CMD="/usr/local/bin/awesome"
+DEFAULT_LOG_FILE="/var/log/awesome-index.log"
+DEFAULT_DB_SOURCE="/root/.awesome/awesome.db"
+DEFAULT_DB_STAGING="/tmp/awesome-database"
+DEFAULT_CONTAINER="awesome_app"
+DEFAULT_CONTAINER_PATH="/home/node/.awesome"
+DEFAULT_SCHEDULE_TIME="02:00"
+DEFAULT_INCREMENTAL="true"
+
+# Prompt for values
+echo -e "${YELLOW}Please provide the following configuration:${NC}"
+echo ""
+
+read -p "SSH host [${DEFAULT_SSH_HOST}]: " SSH_HOST
+SSH_HOST="${SSH_HOST:-$DEFAULT_SSH_HOST}"
+
+read -p "User to run service as [${DEFAULT_USER}]: " USER
+USER="${USER:-$DEFAULT_USER}"
+
+read -p "Working directory [${DEFAULT_WORK_DIR}]: " WORK_DIR
+WORK_DIR="${WORK_DIR:-$DEFAULT_WORK_DIR}"
+
+read -p "Awesome command path [${DEFAULT_AWESOME_CMD}]: " AWESOME_CMD
+AWESOME_CMD="${AWESOME_CMD:-$DEFAULT_AWESOME_CMD}"
+
+read -p "Log file path [${DEFAULT_LOG_FILE}]: " LOG_FILE
+LOG_FILE="${LOG_FILE:-$DEFAULT_LOG_FILE}"
+
+read -p "Database source path [${DEFAULT_DB_SOURCE}]: " DB_SOURCE
+DB_SOURCE="${DB_SOURCE:-$DEFAULT_DB_SOURCE}"
+
+read -p "Database staging directory [${DEFAULT_DB_STAGING}]: " DB_STAGING
+DB_STAGING="${DB_STAGING:-$DEFAULT_DB_STAGING}"
+
+read -p "Docker container name (leave empty to skip Docker deployment) [${DEFAULT_CONTAINER}]: " CONTAINER
+CONTAINER="${CONTAINER:-$DEFAULT_CONTAINER}"
+
+if [ -n "$CONTAINER" ]; then
+    read -p "Container database path [${DEFAULT_CONTAINER_PATH}]: " CONTAINER_PATH
+    CONTAINER_PATH="${CONTAINER_PATH:-$DEFAULT_CONTAINER_PATH}"
+fi
+
+read -p "Schedule time (HH:MM format) [${DEFAULT_SCHEDULE_TIME}]: " SCHEDULE_TIME
+SCHEDULE_TIME="${SCHEDULE_TIME:-$DEFAULT_SCHEDULE_TIME}"
+
+read -p "Use incremental indexing by default? (true/false) [${DEFAULT_INCREMENTAL}]: " INCREMENTAL
+INCREMENTAL="${INCREMENTAL:-$DEFAULT_INCREMENTAL}"
+
+# Confirmation
+echo ""
+echo -e "${YELLOW}Configuration Summary:${NC}"
+echo "  SSH Host:           $SSH_HOST"
+echo "  User:               $USER"
+echo "  Working Directory:  $WORK_DIR"
+echo "  Awesome Command:    $AWESOME_CMD"
+echo "  Log File:           $LOG_FILE"
+echo "  Database Source:    $DB_SOURCE"
+echo "  Database Staging:   $DB_STAGING"
+echo "  Docker Container:   ${CONTAINER:-<not configured>}"
+if [ -n "$CONTAINER" ]; then
+    echo "  Container Path:     $CONTAINER_PATH"
+fi
+echo "  Schedule Time:      $SCHEDULE_TIME"
+echo "  Incremental:        $INCREMENTAL"
+echo ""
+
+read -p "Continue with deployment? (y/n): " CONFIRM
+if [ "$CONFIRM" != "y" ] && [ "$CONFIRM" != "Y" ]; then
+    echo -e "${RED}Deployment cancelled.${NC}"
+    exit 0
+fi
+
+echo ""
+echo -e "${BLUE}Starting deployment...${NC}"
+
+# Create temporary directory for processed files
+TEMP_DIR=$(mktemp -d)
+trap "rm -rf $TEMP_DIR" EXIT
+
+# Process templates
+echo -e "${YELLOW}→${NC} Processing templates..."
+
+SCRIPT_PATH="$WORK_DIR/scripts/awesome-index.sh"
+
+# Process awesome-index.sh
+sed -e "s|{{LOG_FILE}}|$LOG_FILE|g" \
+    -e "s|{{DB_SOURCE}}|$DB_SOURCE|g" \
+    -e "s|{{DB_STAGING}}|$DB_STAGING|g" \
+    -e "s|{{CONTAINER}}|$CONTAINER|g" \
+    -e "s|{{CONTAINER_PATH}}|$CONTAINER_PATH|g" \
+    -e "s|{{AWESOME_CMD}}|$AWESOME_CMD|g" \
+    -e "s|{{INCREMENTAL}}|$INCREMENTAL|g" \
+    "$SCRIPT_DIR/awesome-index.sh.template" > "$TEMP_DIR/awesome-index.sh"
+
+# Process awesome-index.service
+sed -e "s|{{USER}}|$USER|g" \
+    -e "s|{{WORK_DIR}}|$WORK_DIR|g" \
+    -e "s|{{SCRIPT_PATH}}|$SCRIPT_PATH|g" \
+    "$SCRIPT_DIR/awesome-index.service.template" > "$TEMP_DIR/awesome-index.service"
+
+# Process awesome-index.timer
+sed -e "s|{{SCHEDULE_TIME}}|$SCHEDULE_TIME|g" \
+    "$SCRIPT_DIR/awesome-index.timer.template" > "$TEMP_DIR/awesome-index.timer"
+
+echo -e "${GREEN}✓${NC} Templates processed"
+
+# Test SSH connection
+echo -e "${YELLOW}→${NC} Testing SSH connection..."
+if ssh "$SSH_HOST" "echo 'Connection successful'" &>/dev/null; then
+    echo -e "${GREEN}✓${NC} SSH connection successful"
+else
+    echo -e "${RED}✗${NC} SSH connection failed"
+    exit 1
+fi
+
+# Create directories on remote server
+echo -e "${YELLOW}→${NC} Creating directories on remote server..."
+ssh "$SSH_HOST" "mkdir -p $WORK_DIR/scripts /etc/systemd/system"
+echo -e "${GREEN}✓${NC} Directories created"
+
+# Copy script to remote server
+echo -e "${YELLOW}→${NC} Copying indexing script..."
+scp "$TEMP_DIR/awesome-index.sh" "$SSH_HOST:$SCRIPT_PATH"
+ssh "$SSH_HOST" "chmod +x $SCRIPT_PATH"
+echo -e "${GREEN}✓${NC} Script copied and made executable"
+
+# Copy systemd files
+echo -e "${YELLOW}→${NC} Installing systemd service and timer..."
+scp "$TEMP_DIR/awesome-index.service" "$SSH_HOST:/etc/systemd/system/"
+scp "$TEMP_DIR/awesome-index.timer" "$SSH_HOST:/etc/systemd/system/"
+echo -e "${GREEN}✓${NC} Systemd files installed"
+
+# Reload systemd and enable timer
+echo -e "${YELLOW}→${NC} Enabling systemd timer..."
+ssh "$SSH_HOST" "systemctl daemon-reload && systemctl enable awesome-index.timer && systemctl start awesome-index.timer"
+echo -e "${GREEN}✓${NC} Timer enabled and started"
+
+# Show status
+echo ""
+echo -e "${GREEN}╔════════════════════════════════════════════════════════════╗${NC}"
+echo -e "${GREEN}║  Deployment completed successfully!                        ║${NC}"
+echo -e "${GREEN}╚════════════════════════════════════════════════════════════╝${NC}"
+echo ""
+echo -e "${YELLOW}Status on remote server:${NC}"
+ssh "$SSH_HOST" "systemctl status awesome-index.timer --no-pager" || true
+echo ""
+echo -e "${YELLOW}Next scheduled run:${NC}"
+ssh "$SSH_HOST" "systemctl list-timers awesome-index.timer --no-pager" || true
+echo ""
+echo -e "${YELLOW}Useful commands:${NC}"
+echo "  View logs:          ssh $SSH_HOST 'journalctl -u awesome-index.service -f'"
+echo "  Manual trigger:     ssh $SSH_HOST 'systemctl start awesome-index.service'"
+echo "  Check timer status: ssh $SSH_HOST 'systemctl status awesome-index.timer'"
+echo "  View log file:      ssh $SSH_HOST 'tail -f $LOG_FILE'"
+echo ""