Compare commits

...

9 Commits

Author SHA1 Message Date
valknar d7829cb5ae docs: fix README backup/update/notifications sections
- Source .env before restic init so RESTIC_REPOSITORY is available
- Remove stale references to deleted _backup/.env and _update/.env
- Update Notifications section to point at root .env

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 21:13:00 +02:00
valknar 8ae9c9e878 fix(update): compare local image store IDs, not running container IDs
docker compose images -q reports the image IDs of currently running
containers, which don't change after a pull — so before == after always
and containers were never recreated.

Fix: resolve each service's image tag to its local SHA256 ID via
docker image inspect, which reads the local image store and correctly
reflects the newly pulled image. Falls back from 'config --images'
(compose v2.19+) to parsing 'config' yaml for older versions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 21:09:56 +02:00
valknar fcff6f3298 refactor: absorb _backup and _update into stacks.sh
- Inline update logic (pull → compare digests → up -d → prune → notify)
- Inline backup logic with dynamic Postgres detection: any running
  <stack>_db container is dumped using the <stack>/<stack> convention
- Systemd unit files are now generated on `install` from embedded
  heredocs pointing at stacks.sh itself — no external scripts needed
- Root .env (WEBHOOK_URL, RESTIC_REPOSITORY, RESTIC_PASSWORD) replaces
  the per-service .env files in _backup/ and _update/
- Remove _backup/ and _update/ directories entirely
- Update README accordingly

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 20:56:04 +02:00
valknar e3cd2df372 docs: document stacks.sh in README
Replace manual docker compose / systemctl snippets with stacks.sh
equivalents and add a dedicated section covering all commands.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 19:55:05 +02:00
valknar 067d017ea6 feat(stacks): add --static flag to completion command
Bakes the current stack list into the generated completion script instead
of using runtime directory discovery. Useful for remote hosts where the
stacks dir path differs from the local repo.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 19:52:55 +02:00
valknar 3251f27edb feat: add stacks.sh stack manager CLI
Bash script for managing Docker Compose stacks with glob filtering,
parallel execution, service management, scaffolding, and shell completion.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 19:13:43 +02:00
valknar cf32e669cb fix(gitea): use .RunModeIsProd and hardcode Umami values in header template
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 10:34:21 +02:00
valknar e613d766ec fix(gitea): revert to UMAMI_ID/UMAMI_SRC and use .UmamiSrc/.UmamiId template vars
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 10:26:39 +02:00
valknar 53608a7195 fix(gitea): use .UmamiSrc/.UmamiScript template vars for Umami analytics
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 10:24:57 +02:00
14 changed files with 1444 additions and 236 deletions
+6
View File
@@ -0,0 +1,6 @@
# Webhook for update/backup notifications (n8n → Telegram)
WEBHOOK_URL=https://n8n.example.com/webhook/your-webhook-id
# Restic backup repository and password
RESTIC_REPOSITORY=/mnt/hidrive/users/valknar/Backup/stacks
RESTIC_PASSWORD=changeme
+80 -43
View File
@@ -20,10 +20,71 @@ Each stack is independently deployable with its own `compose.yml` and `.env`. Al
## Tools ## Tools
| Directory | Description | | File | Description |
|---|---| |---|---|
| `_backup` | Daily restic backups to HiDrive (host script + systemd timer) | | `stacks.sh` | CLI to manage stacks, services, scaffolding, updates, and backups |
| `_update` | Nightly image update check + prune (host script + systemd timer) | | `.env` | Root config: `WEBHOOK_URL`, `RESTIC_REPOSITORY`, `RESTIC_PASSWORD` (gitignored) |
| `.env.example` | Template for the root `.env` |
## stacks.sh
`stacks.sh` is the primary management CLI. It wraps `docker compose` with glob-based multi-stack targeting, manages the update and backup systemd services, scaffolds new stacks, and generates shell completions.
```bash
./stacks.sh help
```
**Stack commands** — all accept one or more stack names or glob patterns (omit for all stacks):
```bash
./stacks.sh ls # list all stacks with live container status
./stacks.sh ps gitea # container status table
./stacks.sh up # start all stacks
./stacks.sh up gitea traefik # start specific stacks
./stacks.sh down 'g*' # stop stacks matching glob
./stacks.sh restart 'g*,traefik' # glob + exact name, comma-separated
./stacks.sh pull --parallel # pull all images in parallel
./stacks.sh logs -f gitea # follow logs
./stacks.sh logs -n 100 gitea n8n # tail multiple stacks
./stacks.sh exec gitea gitea gitea admin user list # exec in container
./stacks.sh run passbolt passbolt bin/cake passbolt healthcheck
```
**Service management** (reads `WEBHOOK_URL`, `RESTIC_REPOSITORY`, `RESTIC_PASSWORD` from root `.env`):
```bash
./stacks.sh update install # write & enable systemd update timer
./stacks.sh update run # run update now
./stacks.sh update status # show timer/service status
./stacks.sh update logs # show journal logs
./stacks.sh backup install # write & enable systemd backup timer
./stacks.sh backup run # run backup now (auto-detects <stack>_db containers)
./stacks.sh backup snapshots # list restic snapshots
```
**Scaffold a new stack:**
```bash
./stacks.sh new myapp # basic stack with Traefik labels
./stacks.sh new myapp --db postgres # with Postgres service
./stacks.sh new myapp --db postgres --redis # with Postgres + Redis
./stacks.sh new myapp --no-traefik # expose port instead of Traefik
```
Generates `compose.yml` (with healthchecks, `../.data/` volumes, Traefik labels) and `.env.example`.
**Shell completion:**
```bash
# Dynamic — discovers stacks at tab-complete time (default)
./stacks.sh completion zsh --install
# Static — bakes current stack list in; useful on the VPS
./stacks.sh completion zsh --static --install
```
**Global flags:** `--dry-run`, `--parallel`, `--verbose`, `--quiet`
## Deployment ## Deployment
@@ -44,60 +105,36 @@ All stacks share the external `falcon_network` Docker network for inter-service
## Backup ## Backup
The `_backup` stack runs a daily restic backup at 3:00 AM. It dumps all Postgres databases, then backs up the entire `.data/` directory to HiDrive. Retention: 7 daily, 4 weekly, 6 monthly snapshots. Notifications go to Telegram via n8n. Runs daily at 3:00 AM via a systemd timer. Detects Postgres databases automatically by convention (`<stack>_db` container, user `<stack>`, database `<stack>`), dumps each one, then runs a full restic backup of `.data/`. Retention: 7 daily, 4 weekly, 6 monthly. Notifications go to Telegram via n8n.
```bash ```bash
# Deploy backup stack # First-time setup on VPS
rsync -avz _backup/ vps:~/stacks/_backup/ cp .env.example .env && $EDITOR .env # set RESTIC_REPOSITORY, RESTIC_PASSWORD, WEBHOOK_URL
source .env && restic init # initialise restic repo
# Initialize restic repo (first time only) ./stacks.sh backup install # write & enable systemd unit + timer
ssh vps 'source ~/stacks/_backup/.env && restic init -r /mnt/hidrive/users/valknar/Backup/stacks' ./stacks.sh backup run # test run
./stacks.sh backup snapshots # list snapshots
# Install systemd units ./stacks.sh backup status # timer/service status
ssh vps 'ln -sf ~/stacks/_backup/stacks-backup.service /etc/systemd/system/ && \ ./stacks.sh backup logs # journald logs
ln -sf ~/stacks/_backup/stacks-backup.timer /etc/systemd/system/ && \
systemctl daemon-reload && systemctl enable --now stacks-backup.timer'
# Manual test run
ssh vps '~/stacks/_backup/backup.sh'
# Check timer status
ssh vps 'systemctl status stacks-backup.timer'
# View snapshots
ssh vps 'source ~/stacks/_backup/.env && restic -r /mnt/hidrive/users/valknar/Backup/stacks snapshots'
``` ```
## Updates ## Updates
The `_update` script runs nightly at 2:00 AM. It pulls the latest image for every stack, recreates any containers whose image changed, prunes dangling images, and sends a Telegram notification via n8n. Runs nightly at 2:00 AM via a systemd timer. Pulls the latest image for every stack, recreates any container whose image changed, prunes dangling images, and sends a Telegram notification via n8n.
```bash ```bash
# Deploy update stack ./stacks.sh update install # write & enable systemd unit + timer
rsync -avz _update/ vps:~/stacks/_update/ ./stacks.sh update run # test run
ssh vps 'chmod +x ~/stacks/_update/update.sh' ./stacks.sh update status # timer/service status
./stacks.sh update logs # journald logs
# Install systemd units
ssh vps 'ln -sf ~/stacks/_update/stacks-update.service /etc/systemd/system/ && \
ln -sf ~/stacks/_update/stacks-update.timer /etc/systemd/system/ && \
systemctl daemon-reload && systemctl enable --now stacks-update.timer'
# Manual test run
ssh vps '~/stacks/_update/update.sh'
# Check timer status
ssh vps 'systemctl status stacks-update.timer'
``` ```
## Notifications ## Notifications
The update script and the backup script both POST to an n8n webhook, which forwards messages to Telegram. Both update and backup POST to an n8n webhook on completion, which forwards the message to Telegram.
The webhook URL is set in: The webhook URL is set via `WEBHOOK_URL` in the root `.env`. Both services point to the same n8n workflow at `https://n8n.pivoine.art`, which accepts `{ "message": "...", "color": "..." }` and forwards it to Telegram.
- `_backup/.env``WEBHOOK_URL`
- `_update/.env``WEBHOOK_URL`
Both point to the same n8n workflow at `https://n8n.pivoine.art`. The workflow accepts `{ "message": "..." }` and forwards it to Telegram.
## Data ## Data
-3
View File
@@ -1,3 +0,0 @@
RESTIC_REPOSITORY=/mnt/hidrive/users/valknar/Backup/stacks
RESTIC_PASSWORD=change_me
WEBHOOK_URL=https://n8n.example.com/webhook/change_me
-83
View File
@@ -1,83 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
STACKS_DIR="$(dirname "$SCRIPT_DIR")"
DATA_DIR="$STACKS_DIR/.data"
DUMP_DIR="$DATA_DIR/backup/dumps"
LOG_FILE="$SCRIPT_DIR/backup.log"
# Load environment
set -a
source "$SCRIPT_DIR/.env"
set +a
export RESTIC_REPOSITORY RESTIC_PASSWORD
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "$LOG_FILE"; }
notify() {
local color="$1" text="$2"
curl -sf -o /dev/null -X POST "$WEBHOOK_URL" \
-H 'Content-Type: application/json' \
-d "{\"message\":\"$text\",\"color\":\"$color\"}"
}
# Truncate log on each run
: > "$LOG_FILE"
log "Starting backup"
# --- Postgres dumps ---
mkdir -p "$DUMP_DIR"
declare -A DATABASES=(
[umami_db]="umami:umami"
[gitea_db]="gitea:gitea"
[n8n_db]="n8n:n8n"
[immich_db]="immich:immich"
[coolify_db]="coolify:coolify"
[passbolt_db]="passbolt:passbolt"
)
dump_errors=()
for container in "${!DATABASES[@]}"; do
IFS=: read -r user db <<< "${DATABASES[$container]}"
log "Dumping $db from $container"
if docker exec "$container" pg_dump -U "$user" "$db" > "$DUMP_DIR/$db.sql" 2>>"$LOG_FILE"; then
log " OK ($(du -h "$DUMP_DIR/$db.sql" | cut -f1))"
else
log " FAILED: $container"
dump_errors+=("$db")
fi
done
if [ ${#dump_errors[@]} -gt 0 ]; then
log "WARNING: Failed dumps: ${dump_errors[*]}"
fi
# --- Restic backup ---
log "Running restic backup"
if ! restic backup "$DATA_DIR" --tag stacks 2>&1 | tee -a "$LOG_FILE"; then
log "FATAL: restic backup failed"
notify "#cc0000" "❌ **Backup failed** — restic backup error. Check \`$LOG_FILE\`."
exit 1
fi
# --- Restic prune ---
log "Pruning old snapshots"
if ! restic forget --prune --keep-daily 7 --keep-weekly 4 --keep-monthly 6 2>&1 | tee -a "$LOG_FILE"; then
log "WARNING: restic forget failed"
fi
# --- Summary ---
snapshot_info=$(restic snapshots --latest 1 --compact 2>/dev/null | tail -3 | head -1)
repo_stats=$(restic stats 2>/dev/null | grep "Total Size" || true)
summary="✅ **Backup complete**"
[ ${#dump_errors[@]} -gt 0 ] && summary+="\n⚠️ Failed dumps: ${dump_errors[*]}"
summary+="\nLatest: \`$snapshot_info\`"
[ -n "$repo_stats" ] && summary+="\nRepo: $repo_stats"
notify "#36a64f" "$summary"
log "Backup complete"
-10
View File
@@ -1,10 +0,0 @@
[Unit]
Description=Backup stacks data to HiDrive via restic
After=network-online.target docker.service
Wants=network-online.target
[Service]
Type=oneshot
User=root
ExecStart=/root/stacks/_backup/backup.sh
Environment=HOME=/root
-9
View File
@@ -1,9 +0,0 @@
[Unit]
Description=Daily stacks backup at 3:00 AM
[Timer]
OnCalendar=*-*-* 03:00:00 Europe/Amsterdam
Persistent=true
[Install]
WantedBy=timers.target
-1
View File
@@ -1 +0,0 @@
WEBHOOK_URL=https://n8n.example.com/webhook/change_me
-10
View File
@@ -1,10 +0,0 @@
[Unit]
Description=Docker stacks image update
After=network-online.target docker.service
Wants=network-online.target
[Service]
Type=oneshot
User=root
ExecStart=/root/stacks/_update/update.sh
Environment=HOME=/root
-9
View File
@@ -1,9 +0,0 @@
[Unit]
Description=Daily Docker stacks image update at 2:00 AM
[Timer]
OnCalendar=*-*-* 02:00:00 Europe/Amsterdam
Persistent=true
[Install]
WantedBy=timers.target
-62
View File
@@ -1,62 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
STACKS_DIR="$(dirname "$SCRIPT_DIR")"
LOG_FILE="$SCRIPT_DIR/update.log"
set -a; source "$SCRIPT_DIR/.env"; set +a
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "$LOG_FILE"; }
notify() {
local color="$1" text="$2"
curl -sf -o /dev/null -X POST "$WEBHOOK_URL" \
-H 'Content-Type: application/json' \
-d "{\"message\":\"$text\",\"color\":\"$color\"}"
}
: > "$LOG_FILE"
log "Starting image update check"
updated=(); failed=(); checked=0
for stack_dir in "$STACKS_DIR"/*/; do
stack=$(basename "$stack_dir")
[[ "$stack" == _* ]] && continue
[[ ! -f "$stack_dir/compose.yml" ]] && continue
log "Checking $stack"
cd "$stack_dir"
checked=$((checked + 1))
before=$(docker compose images -q 2>/dev/null | sort)
docker compose pull 2>&1 | tee -a "$LOG_FILE"
after=$(docker compose images -q 2>/dev/null | sort)
if [ "$before" != "$after" ]; then
log " Updates found — recreating containers"
if docker compose up -d 2>&1 | tee -a "$LOG_FILE"; then
updated+=("$stack")
else
log " FAILED to recreate $stack"
failed+=("$stack")
fi
else
log " Up to date"
fi
done
log "Pruning dangling images"
docker image prune -f 2>&1 | tee -a "$LOG_FILE"
date_str=$(date '+%Y-%m-%d %H:%M')
if [ ${#failed[@]} -gt 0 ]; then
notify "#cc0000" "❌ **Update failed** — $date_str\nUpdated: ${updated[*]:-none}\nFailed: ${failed[*]}"
elif [ ${#updated[@]} -gt 0 ]; then
notify "#36a64f" "✅ **Updates applied** — $date_str\nStacks updated (${#updated[@]}/$checked): ${updated[*]}"
else
notify "#36a64f" "✅ **All up to date** — $date_str ($checked stacks checked)"
fi
log "Update check complete"
-2
View File
@@ -3,5 +3,3 @@ NETWORK_NAME=falcon_network
RUNNER_TOKEN=change_me RUNNER_TOKEN=change_me
EMAIL_FROM=gitea@pivoine.art EMAIL_FROM=gitea@pivoine.art
EMAIL_USER=Gitea EMAIL_USER=Gitea
UMAMI_SRC=https://umami.example.com/script.js
UMAMI_ID=
-2
View File
@@ -36,8 +36,6 @@ services:
GITEA__ui__THEMES: gitea-auto,gitea-light,gitea-dark,arc-green,edge-auto,edge-dark,edge-light,everforest-auto,everforest-dark,everforest-light,gruvbox-auto,gruvbox-dark,gruvbox-light,gruvbox-material-auto,gruvbox-material-dark,gruvbox-material-light,nord,palenight,soft-era,sonokai,sonokai-andromeda,sonokai-atlantis,sonokai-espresso,sonokai-maia,sonokai-shusia GITEA__ui__THEMES: gitea-auto,gitea-light,gitea-dark,arc-green,edge-auto,edge-dark,edge-light,everforest-auto,everforest-dark,everforest-light,gruvbox-auto,gruvbox-dark,gruvbox-light,gruvbox-material-auto,gruvbox-material-dark,gruvbox-material-light,nord,palenight,soft-era,sonokai,sonokai-andromeda,sonokai-atlantis,sonokai-espresso,sonokai-maia,sonokai-shusia
GITEA__ui__DEFAULT_THEME: nord GITEA__ui__DEFAULT_THEME: nord
GITEA__ui__ENABLE_FEED: "true" GITEA__ui__ENABLE_FEED: "true"
UMAMI_SRC: ${UMAMI_SRC}
UMAMI_ID: ${UMAMI_ID}
ports: ports:
- "2222:2222" - "2222:2222"
volumes: volumes:
+2 -2
View File
@@ -1,3 +1,3 @@
{{if and (eq .RunMode "prod") (not .IsSigned)}} {{if and .RunModeIsProd (not .IsSigned)}}
<script defer src="{{env "UMAMI_SRC"}}" data-website-id="{{env "UMAMI_ID"}}"></script> <script defer src="https://umami.pivoine.art/script.js" data-website-id="eee7e810-116b-408a-a46d-f51dcec217c2"></script>
{{end}} {{end}}
Executable
+1356
View File
File diff suppressed because it is too large Load Diff