valknar/docker-compose
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
8bdcde4b904ba7fd9039efab834a346dac870cb1
docker-compose/README.md
Sebastian Krüger ef0309838c refactor(ai): remove crawl4ai service, add backrest config to repo 
- Remove crawl4ai service from ai/compose.yaml (will use local MCP instead)
- Remove crawl4ai backup volume from core/compose.yaml
- Add core/backrest/config.json (infrastructure as code)
- Change backrest from volume to bind-mounted config
- Update CLAUDE.md and README.md documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-25 06:20:22 +01:00

576 lines
22 KiB
Markdown
Raw Blame History

<div align="center">
```
___ ___ ___ ___ ___ ___
/\ \ /\ \ /\__\ /\ \ /\ \ /\__\
/::\ \ /::\ \ /:/ / /::\ \ /::\ \ /:| _|_
/::\:\__\ /::\:\__\ /:/__/ /:/\:\__\ /:/\:\__\ /::|/\__\
\/\::/ / \/\::/ / \:\ \ \:\ \/__/ \:\/:/ / \/|::/ /
/:/ / /:/ / \:\__\ \:\__\ \::/ / |:/ /
\/__/ \/__/ \/__/ \/__/ \/__/ \/__/
```
# ⚡ THE FALCON ⚡
**Captain Valknar's Legendary Starship**
[![Status](https://img.shields.io/badge/STATUS-DEEP_SPACE-00d4ff?style=for-the-badge&logo=spacex&logoColor=white)](https://pivoine.art)
[![Mission](https://img.shields.io/badge/MISSION-ALIEN_ENCOUNTERS-4169e1?style=for-the-badge&logo=rocket&logoColor=white)](https://sexy.pivoine.art)
[![Crew](https://img.shields.io/badge/CAPTAIN-VALKNAR-silver?style=for-the-badge&logo=linux&logoColor=white)](mailto:valknar@pivoine.art)
[![Network](https://img.shields.io/badge/NETWORK-FALCON__NETWORK-0077b6?style=for-the-badge&logo=docker&logoColor=white)](#)
[![Location](https://img.shields.io/badge/SECTOR-PIVOINE.ART-00b4d8?style=for-the-badge&logo=cloudflare&logoColor=white)](https://pivoine.art)
---
</div>
## 🌌 SHIP'S LOG
**STARDATE:** 2025.11.15
**LOCATION:** Deep Space, Uncharted Territories
**STATUS:** Captain currently engaged in... diplomatic relations with alien civilizations
**SYSTEMS:** All green, automated deployment active, CI/CD pipeline operational
> *"The Falcon doesn't just traverse the stars — it commands them."*
> — Captain Valknar, moments before jumping to hyperspace
---
## 🛸 VESSEL SPECIFICATIONS
The **Falcon** is a state-of-the-art containerized starship, powered by Docker's quantum drive engines and orchestrated through the legendary Arty navigation system.
### 🎯 MISSION CRITICAL SYSTEMS
**29 Services** organized across **7 Specialized Stacks**
#### 🛠️ CORE Infrastructure (3 services)
| Service | Purpose | Access |
|---------|---------|--------|
| **PostgreSQL 16** | Central database vault | Internal: 5432 |
| **Redis 7** | Hyperspeed cache drive | Internal |
| **Backrest** | Automated backup system | [restic.pivoine.art](https://restic.pivoine.art) |
#### 🎨 SEXY Portfolio (2 services)
| Service | Purpose | Access |
|---------|---------|--------|
| **Directus API** | Headless CMS backend | [sexy.pivoine.art/api](https://sexy.pivoine.art/api) |
| **SvelteKit Frontend** | Art portfolio interface | [sexy.pivoine.art](https://sexy.pivoine.art) |
#### 🧰 UTIL Productivity (7 services)
| Service | Purpose | Access |
|---------|---------|--------|
| **PairDrop** | P2P file sharing | [drop.pivoine.art](https://drop.pivoine.art) |
| **Joplin Server** | Note-taking sync hub | [joplin.pivoine.art](https://joplin.pivoine.art) |
| **Linkwarden** | Bookmark manager | [links.pivoine.art](https://links.pivoine.art) |
| **Mattermost** | Team collaboration | [mattermost.pivoine.art](https://mattermost.pivoine.art) |
| **Vaultwarden** | Password manager | [vault.pivoine.art](https://vault.pivoine.art) |
| **Tandoor** | Recipe management | [tandoor.pivoine.art](https://tandoor.pivoine.art) |
| **Meilisearch** | Search engine | Internal |
#### 🤖 AI Intelligence (5 services)
| Service | Purpose | Access |
|---------|---------|--------|
| **Open WebUI** | Claude AI interface | [ai.pivoine.art](https://ai.pivoine.art) |
| **LiteLLM** | API proxy | [llm.ai.pivoine.art](https://llm.ai.pivoine.art) |
| **Crawl4AI** | Web scraping | Internal: 11235 |
| **FaceFusion** | AI face swapping | [facefusion.ai.pivoine.art](https://facefusion.ai.pivoine.art) |
| **PostgreSQL+pgvector** | Vector database | Internal |
#### 🛡️ NET Infrastructure (4 services)
| Service | Purpose | Access |
|---------|---------|--------|
| **Traefik** | Reverse proxy & SSL | [proxy.pivoine.art](https://proxy.pivoine.art) |
| **Netdata** | Real-time monitoring | [netdata.pivoine.art](https://netdata.pivoine.art) |
| **Watchtower** | Auto-update agent | Background |
| **Umami** | Analytics platform | [umami.pivoine.art](https://umami.pivoine.art) |
#### 📺 MEDIA Streaming (2 services)
| Service | Purpose | Access |
|---------|---------|--------|
| **Jellyfin** | Media server | [jelly.pivoine.art](https://jelly.pivoine.art) |
| **Filestash** | File manager | [filestash.pivoine.art](https://filestash.pivoine.art) |
#### 🚀 DEV Platform (6 services)
| Service | Purpose | Access |
|---------|---------|--------|
| **Gitea** | Git & CI/CD | [dev.pivoine.art](https://dev.pivoine.art) |
| **Act Runner** | Workflow executor | Background |
| **Coolify** | Deployment platform | [coolify.dev.pivoine.art](https://coolify.dev.pivoine.art) |
| **Soketi** | WebSocket server | Internal |
| **n8n** | Workflow automation | [n8n.dev.pivoine.art](https://n8n.dev.pivoine.art) |
| **Asciinema** | Terminal recorder | [asciinema.dev.pivoine.art](https://asciinema.dev.pivoine.art) |
### ⚙️ INFRASTRUCTURE
```
┌──────────────────────────────────────────────────────┐
│ 🛡️ TRAEFIK REVERSE PROXY (NET Stack) │
│ ├─ Auto-SSL via Let's Encrypt │
│ ├─ HTTP → HTTPS Redirects │
│ ├─ Load Balancing & Routing │
│ ├─ Dashboard at proxy.pivoine.art │
│ ├─ Dynamic Security Headers │
│ └─ Multi-network Support (falcon + coolify) │
├──────────────────────────────────────────────────────┤
│ 💾 CORE POSTGRESQL 16 (CORE Stack) │
│ ├─ sexy (Directus CMS) │
│ ├─ umami (Analytics) │
│ ├─ n8n (Workflows) │
│ ├─ linkwarden (Bookmarks) │
│ ├─ joplin (Notes) │
│ ├─ mattermost (Chat) │
│ ├─ tandoor (Recipes) │
│ ├─ gitea (Git & CI/CD) │
│ ├─ coolify (Deployment) │
│ └─ asciinema (Terminal Recording) │
├──────────────────────────────────────────────────────┤
│ 🤖 AI POSTGRESQL+PGVECTOR (AI Stack) │
│ ├─ Vector similarity search │
│ ├─ Open WebUI document embeddings │
│ └─ RAG knowledge base │
├──────────────────────────────────────────────────────┤
│ ⚡ REDIS 7 CACHE (CORE Stack) │
│ ├─ Directus caching │
│ ├─ Coolify queue management │
│ └─ Umami session storage │
├──────────────────────────────────────────────────────┤
│ 🔐 BACKREST BACKUP SYSTEM (CORE Stack) │
│ ├─ 17 automated backup plans │
│ ├─ Daily incremental snapshots │
│ ├─ Weekly maintenance (prune & check) │
│ ├─ Retention: 7d/4w/3-12m/2-3y │
│ └─ HiDrive remote storage │
├──────────────────────────────────────────────────────┤
│ 📊 MONITORING & UPDATES (NET Stack) │
│ ├─ Netdata: Real-time metrics & alerts │
│ ├─ Watchtower: Automatic container updates │
│ └─ Mattermost webhooks for notifications │
└──────────────────────────────────────────────────────┘
```
---
## 🚀 LAUNCH SEQUENCE
### Prerequisites
- Docker Engine v20+ installed
- Docker Compose v2.20+ installed
- Arty navigation system (`npm install -g arty` or `pnpm add -g arty`)
- Clearance level: **Captain**
### 🔧 Initialize Ship Systems
```bash
# Create the ship's neural network
arty net/create
# Launch all systems
arty up
# Monitor system status
arty ps
# Access ship's logs (real-time)
arty logs
```
### 📡 Individual System Control
```bash
# Power down specific systems
arty down
# Restart malfunctioning modules
arty restart
# Pull latest system updates from the mothership
arty pull
# Diagnostic report
arty config
```
---
## 💫 NAVIGATION COMMANDS
### SEXY Database Operations
```bash
# Export Directus database + schema snapshot
arty sexy/export/all
# Export only database
arty sexy/db/export
# Export only schema
arty sexy/schema/export
# Import database + schema (⚠️ replaces existing data)
arty sexy/import/all
# Export uploads directory
arty sexy/uploads/export
# Import uploads directory
arty sexy/uploads/import
# Update frontend bundle from registry image
arty sexy/bundle/update
```
### Deployment & Sync
```bash
# Synchronize .env to remote VPS
arty env/sync
```
### Backup Operations (Backrest)
```bash
# Access backup web interface
# URL: https://restic.pivoine.art
# Username: valknar
# Password: Set on first access
# View backup status in logs
docker logs core_backrest | grep scheduled
# Manually trigger backup for a specific plan
docker exec core_backrest /backrest backup --plan postgres-backup
# List all snapshots in repository
docker exec core_backrest restic -r /repos snapshots
# Restore via web UI (recommended)
# Navigate to restic.pivoine.art → Browse snapshots → Restore files
```
**Automated Backup Schedule:**
- **17 backup plans** running daily (2 AM - 11 AM, staggered)
- **Weekly maintenance**: Sundays at 2 AM (prune) and 3 AM (integrity check)
- **Destination**: `/mnt/hidrive/users/valknar/Backup` (HiDrive remote storage)
- **Retention policies**: 7 daily, 4 weekly, 3-12 monthly, 2-3 yearly (varies by service)
### Quick Service Access
```bash
# UTIL Stack
https://drop.pivoine.art # PairDrop file sharing
https://joplin.pivoine.art # Note-taking
https://links.pivoine.art # Bookmarks
https://mattermost.pivoine.art # Team chat
https://vault.pivoine.art # Passwords
https://tandoor.pivoine.art # Recipes
# AI Stack
https://ai.pivoine.art # Open WebUI (Claude)
https://llm.ai.pivoine.art # LiteLLM proxy
https://facefusion.ai.pivoine.art # Face swapping
# MEDIA Stack
https://jelly.pivoine.art # Jellyfin media server
https://filestash.pivoine.art # File manager
# DEV Stack
https://dev.pivoine.art # Gitea (Git + CI/CD)
https://coolify.dev.pivoine.art # Deployment platform
https://n8n.dev.pivoine.art # Workflow automation
https://asciinema.dev.pivoine.art # Terminal recording
# NET Stack
https://proxy.pivoine.art # Traefik dashboard
https://netdata.pivoine.art # Real-time monitoring
https://umami.pivoine.art # Analytics
```
---
## 🔄 CI/CD PIPELINE (GITEA ACTIONS)
The **SEXY** mission uses an automated build and deployment pipeline powered by Gitea Actions.
### 📦 Container Registry
**Image Source:** `dev.pivoine.art/valknar/sexy:latest`
**Registry:** Gitea Container Registry (self-hosted)
### ⚙️ Automated Workflow
```bash
# Workflow triggers on:
├─ Push to main/develop branches
├─ Git tags (v*.*.*)
├─ Pull requests (build only, no push)
└─ Manual workflow dispatch
# Build process:
1. Checkout repository
2. Set up Docker Buildx
3. Login to Gitea Container Registry
4. Extract metadata (tags, labels)
5. Build multi-platform image (linux/amd64)
6. Push to registry with cache optimization
7. Generate deployment summary
```
### 🏷️ Image Tagging Strategy
```yaml
# Automatic tags:
- latest # Main branch builds
- develop # Develop branch builds
- v1.2.3 # Semantic version tags
- v1.2 # Major.minor tags
- v1 # Major version tags
- main-abc123 # Branch + commit SHA
```
### 🚀 Auto-Deployment
**Watchtower** monitors the registry and automatically updates containers when new images are pushed:
```bash
# Check interval: Every 5 minutes
# Update strategy: Rolling restart
# Label-based: Only updates containers with watchtower.enable=true
# Manual pull and restart:
ssh -A root@vps "cd ~/Projects/docker-compose && \
docker pull dev.pivoine.art/valknar/sexy:latest && \
arty up -d sexy_frontend"
```
### 🔑 Required Secrets
Configure in Gitea repository settings:
```bash
# Repository → Settings → Secrets
REGISTRY_TOKEN=<gitea_access_token_with_package_write_scope>
```
### 📊 Build Cache
Uses **registry cache** for faster builds:
```bash
# Cache location:
dev.pivoine.art/valknar/sexy:buildcache
# Benefits:
- Reuses Docker layers between builds
- Significantly faster rebuild times
- No GitHub Actions cache dependency
```
### 🛠️ Runner Configuration
**Gitea Runner:** `docker-runner`
**Labels:** ubuntu-latest, ubuntu-22.04, ubuntu-20.04
**Images:** catthehacker/ubuntu:act-* (with Docker pre-installed)
**Privileged Mode:** Enabled for Docker-in-Docker support
```bash
# View runner status:
ssh -A root@vps "docker logs dev_gitea_runner"
# Runner restart:
ssh -A root@vps "cd ~/Projects/docker-compose && arty restart gitea_runner"
```
---
## 🌠 SHIP ARCHITECTURE
```
THE FALCON (falcon_network)
│
├─ 🛠️ CORE STACK (3 services)
│ ├─ postgres [5432] → PostgreSQL 16 Data Vault
│ ├─ redis [Internal] → Redis 7 Cache Drive
│ └─ backrest [restic.pivoine.art] → Backup System
│
├─ 🎨 SEXY STACK (2 services)
│ ├─ sexy_api [sexy.pivoine.art/api] → Directus CMS
│ └─ sexy_frontend [sexy.pivoine.art] → SvelteKit App
│
├─ 🧰 UTIL STACK (7 services)
│ ├─ pairdrop [drop.pivoine.art] → P2P File Sharing
│ ├─ joplin [joplin.pivoine.art] → Note-Taking Sync
│ ├─ linkwarden [links.pivoine.art] → Bookmark Manager
│ ├─ linkwarden_meili [Internal] → Search Engine
│ ├─ mattermost [mattermost.pivoine.art] → Team Chat
│ ├─ vaultwarden [vault.pivoine.art] → Password Manager
│ └─ tandoor [tandoor.pivoine.art] → Recipe Manager
│
├─ 🤖 AI STACK (4 services)
│ ├─ ai_postgres [Internal] → pgvector Database
│ ├─ webui [ai.pivoine.art] → Open WebUI (Claude)
│ ├─ litellm [llm.ai.pivoine.art] → API Proxy
│ └─ facefusion [facefusion.ai.pivoine.art] → Face AI
│
├─ 🛡️ NET STACK (4 services)
│ ├─ traefik [80/443, proxy.pivoine.art] → Reverse Proxy
│ ├─ netdata [netdata.pivoine.art] → Monitoring
│ ├─ watchtower [Background] → Auto-Updater
│ └─ umami [umami.pivoine.art] → Analytics
│
├─ 📺 MEDIA STACK (2 services)
│ ├─ jellyfin [jelly.pivoine.art] → Media Streaming
│ └─ filestash [filestash.pivoine.art] → File Manager
│
├─ 🚀 DEV STACK (6 services)
│ ├─ gitea [dev.pivoine.art, SSH:2222] → Git + CI/CD
│ ├─ gitea_runner [Background] → Actions Runner
│ ├─ coolify [coolify.dev.pivoine.art] → Deploy Platform
│ ├─ coolify_soketi [coolify-realtime...] → WebSocket Server
│ ├─ n8n [n8n.dev.pivoine.art] → Workflows
│ └─ asciinema [asciinema.dev.pivoine.art] → Terminal Recorder
│
└─ 💾 PERSISTENT VOLUMES (29 services = 40+ volumes)
├─ Core: postgres_data, redis_data, backrest_*
├─ Sexy: directus_uploads, directus_bundle
├─ Util: pairdrop_*, joplin_data, linkwarden_*, mattermost_*, vaultwarden_data, tandoor_*
├─ AI: ai_postgres_data, ai_webui_data, facefusion_*
├─ Net: letsencrypt_data, netdata_*
├─ Media: jelly_config, jelly_cache, filestash_data
└─ Dev: gitea_*, coolify_data, n8n_data, asciinema_data
```
**Network Architecture:**
- **falcon_network**: Main external network connecting all 29 services
- **coolify network**: Separate network for Coolify-deployed applications
- **Traefik multi-network**: Connected to both networks for unified routing
---
## 🎨 TECHNOLOGY STACK
<div align="center">
![Docker](https://img.shields.io/badge/DOCKER-2496ED?style=for-the-badge&logo=docker&logoColor=white)
![Traefik](https://img.shields.io/badge/TRAEFIK-00ADD8?style=for-the-badge&logo=traefikproxy&logoColor=white)
![PostgreSQL](https://img.shields.io/badge/POSTGRESQL-336791?style=for-the-badge&logo=postgresql&logoColor=white)
![Redis](https://img.shields.io/badge/REDIS-DC382D?style=for-the-badge&logo=redis&logoColor=white)
![Directus](https://img.shields.io/badge/DIRECTUS-6644FF?style=for-the-badge&logo=directus&logoColor=white)
![Svelte](https://img.shields.io/badge/SVELTE-FF3E00?style=for-the-badge&logo=svelte&logoColor=white)
![Next.js](https://img.shields.io/badge/NEXT.JS-000000?style=for-the-badge&logo=nextdotjs&logoColor=white)
</div>
---
## ⚠️ PROTOCOLS & SECURITY
```
🔐 ENCRYPTION STANDARD
├─ All transmissions encrypted via HTTPS
├─ Let's Encrypt quantum certificates
├─ TLS 1.2+ with strong cipher suites only
├─ HSTS enabled (1-year, preload ready)
└─ SNI strict mode enforced
🛡️ SECURITY HEADERS
├─ X-Frame-Options: SAMEORIGIN
├─ X-XSS-Protection enabled
├─ Content-Type-Options: nosniff
├─ Referrer-Policy configured
└─ Permissions-Policy restrictions
🔒 ACCESS CONTROL
├─ Admin credentials in .env vault
├─ Database authentication: scram-sha-256
├─ HTTP Basic Auth on sensitive endpoints
├─ Rate limiting available (100 req/s)
└─ VPN cloaking device enabled
💾 BACKUP PROTOCOL
├─ Automated daily backups (2-10 AM)
├─ 16 backup plans covering all volumes
├─ Retention: 7 daily, 4 weekly, 3-12 monthly
├─ Encrypted restic repositories
├─ Weekly maintenance (prune & integrity check)
├─ Web UI for monitoring & restore
└─ HiDrive remote storage
```
---
## 📊 MISSION STATUS
```
╔══════════════════════════════════════════════════════════╗
║ SHIP'S VITAL SIGNS ║
╠══════════════════════════════════════════════════════════╣
║ ✅ CORE Stack (3) → OPERATIONAL ║
║ ✅ SEXY Stack (2) → ONLINE ║
║ ✅ UTIL Stack (7) → ACTIVE ║
║ ✅ AI Stack (5) → INTELLIGENT ║
║ ✅ NET Stack (4) → SECURED ║
║ ✅ MEDIA Stack (2) → STREAMING ║
║ ✅ DEV Stack (6) → DEPLOYING ║
║ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ║
║ 📦 Total Services: 29 ║
║ 🗄️ Database Servers: 2 (PostgreSQL 16 + AI pgvector) ║
║ 💾 Backup Plans: 17 automated (daily 2-11 AM) ║
║ 🔐 SSL Certificates: Auto-renewed (Let's Encrypt) ║
║ 📡 Monitoring: Netdata + Mattermost webhooks ║
║ 🔄 Auto-Updates: Watchtower (5-min interval) ║
║ 🤖 CI/CD: Gitea Actions (docker-runner active) ║
║ 🌟 Captain Status: ON ADVENTURE ║
╚══════════════════════════════════════════════════════════╝
Next Backup: Tomorrow 2:00 AM (postgres-backup)
Backup Destination: /mnt/hidrive/users/valknar/Backup
Repository: Initialized & Healthy
Weekly Maintenance: Sundays 2 AM (prune), 3 AM (check)
```
---
## 👽 CAPTAIN'S NOTES
*Currently out exploring the cosmos and making friends with alien species. You know how it is — one minute you're charting a nebula, the next you're at an intergalactic party.*
*If systems malfunction, check the logs. If things are really bad, I left a backup captain AI (it's called documentation).*
*Stay shiny, crew. Valknar out.*
---
## 📡 TRANSMISSION CHANNELS
- 🌐 **Flagship:** [pivoine.art](https://pivoine.art)
- 📧 **Subspace Mail:** valknar@pivoine.art
- 🎨 **Art Portfolio:** [sexy.pivoine.art](https://sexy.pivoine.art)
- 🤖 **AI Interface:** [ai.pivoine.art](https://ai.pivoine.art)
- 🚀 **Git Operations:** [dev.pivoine.art](https://dev.pivoine.art)
- 💬 **Team Chat:** [mattermost.pivoine.art](https://mattermost.pivoine.art)
- 📊 **Analytics:** [umami.pivoine.art](https://umami.pivoine.art)
- 🛡️ **Monitoring:** [netdata.pivoine.art](https://netdata.pivoine.art)
---
<div align="center">
```
╔═══════════════════════════════════════════════════════════╗
║ ║
║ "In space, no one can hear you `docker compose up`" ║
║ ║
║ — Captain Valknar, The Falcon ║
║ ║
╚═══════════════════════════════════════════════════════════╝
```
![Made with Docker](https://img.shields.io/badge/POWERED_BY-DOCKER_COMPOSE-0db7ed?style=for-the-badge&logo=docker&logoColor=white)
![Arty](https://img.shields.io/badge/NAVIGATED_BY-ARTY-4169e1?style=for-the-badge&logo=npm&logoColor=white)
![Captain](https://img.shields.io/badge/COMMANDED_BY-VALKNAR-silver?style=for-the-badge&logo=linux&logoColor=white)
**THE FALCON** • *Fastest ship in the Docker registry* • **EST. 2025**
</div>
Reference in New Issue View Git Blame Copy Permalink