From 3fea58578260a55e4ba6d9ac302122cf4de92f7d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Sebastian=20Kr=C3=BCger?= Date: Tue, 4 Nov 2025 23:24:00 +0100 Subject: [PATCH] docs: update documentation with security enhancements MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Updated CLAUDE.md and README.md to document security features: **CLAUDE.md updates:** - Expanded Traefik section with security architecture details - Added Security Configuration section with detailed guides - Documented HTTP Basic Auth setup and credential management - Added security testing commands and procedures - Included TLS/header configuration instructions **README.md updates:** - Enhanced PROTOCOLS & SECURITY section - Added TLS 1.2+ and cipher suite information - Listed security headers (HSTS, X-Frame-Options, etc.) - Documented HTTP Basic Auth and rate limiting - Updated access control list Documentation now reflects all security hardening applied to Traefik reverse proxy and service authentication. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- CLAUDE.md | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++++- README.md | 19 +++++++++++++----- 2 files changed, 72 insertions(+), 6 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 6672899..4ef8cc1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -31,7 +31,7 @@ Configuration is centralized in `arty.yml`: Sensitive values (passwords, secrets) live in `.env` and override arty.yml defaults. -### Traefik Routing Architecture +### Traefik Routing & Security Architecture Services expose themselves via Docker labels: - HTTP → HTTPS redirect on `web` entrypoint (port 80) - SSL termination on `web-secure` entrypoint (port 443) @@ -40,6 +40,13 @@ Services expose themselves via Docker labels: - Compression middleware applied via labels - All routers scoped to `$NETWORK_NAME` network +**Security Features:** +- **TLS Security**: Minimum TLS 1.2, strong cipher suites (ECDHE, AES-GCM, ChaCha20), SNI strict mode +- **Security Headers**: HSTS (1-year), X-Frame-Options, X-XSS-Protection, Content-Type-Options, Referrer-Policy, Permissions-Policy +- **Dynamic Configuration**: Security settings in `proxy/dynamic/security.yaml` with auto-reload +- **Rate Limiting**: Available middlewares (100 req/s general, 30 req/s API) +- **HTTP Basic Auth**: Scrapyd protected with username/password authentication + ### Database Initialization `core/postgres/init/01-init-databases.sh` runs on first PostgreSQL startup: - Creates `directus` database for Sexy CMS @@ -142,12 +149,15 @@ Next.js app with embedded SQLite: Web scraping cluster with three services: - **scrapyd**: Scrapyd daemon exposed at `scrapy.pivoine.art:6800` - Web interface for deploying and managing spiders + - Protected by HTTP Basic Auth (credentials in `.env`) - Data persisted in `scrapyd_data` volume - **scrapy**: Development container for running Scrapy commands - Shared `scrapy_code` volume for spider projects - **scrapyrt**: Scrapyd Real-Time API on port 9080 - Run spiders via HTTP API without scheduling +**Authentication**: Access requires username/password (stored as `SCRAPY_AUTH_USERS` in `.env` using htpasswd format) + ### n8n (n8n/compose.yaml) Workflow automation platform: - **n8n**: n8n application exposed at `n8n.pivoine.art:5678` @@ -195,6 +205,53 @@ docker volume ls | grep falcon docker volume inspect ``` +## Security Configuration + +### HTTP Basic Authentication +Scrapyd is protected with HTTP Basic Auth via Traefik middleware: +- Credentials stored in `.env` as `SCRAPY_AUTH_USERS` +- Format: `username:$apr1$hash` (Apache htpasswd format) +- Generate new hash: `openssl passwd -apr1 'your_password'` +- Remember to escape `$` signs with `$$` in `.env` files + +**To update credentials:** +```bash +# Generate hash +echo "username:$(openssl passwd -apr1 'new_password')" + +# Update .env +SCRAPY_AUTH_USERS=username:$$apr1$$hash$$here + +# Sync to VPS +rsync -avzhe ssh .env root@vps:~/Projects/docker-compose/ + +# Restart services +ssh -A root@vps "cd ~/Projects/docker-compose && arty restart" +``` + +### Security Headers & TLS +Global security settings applied via `proxy/dynamic/security.yaml`: +- **TLS**: Minimum TLS 1.2, strong ciphers only, SNI strict mode +- **Headers**: HSTS, X-Frame-Options, CSP, Referrer-Policy, etc. +- **Rate Limiting**: Available middlewares for DDoS protection + +Test security: +```bash +# Check headers +curl -I https://scrapy.pivoine.art + +# SSL Labs test +# Visit: https://www.ssllabs.com/ssltest/analyze.html?d=scrapy.pivoine.art +``` + +### Modifying Security Settings +Edit `proxy/dynamic/security.yaml` to customize: +- TLS versions and cipher suites +- Security header values +- Rate limiting thresholds + +Traefik automatically reloads changes (no restart needed). + ## Troubleshooting ### Services won't start diff --git a/README.md b/README.md index 83841fe..a321758 100644 --- a/README.md +++ b/README.md @@ -202,14 +202,23 @@ THE FALCON (falcon_network) 🔐 ENCRYPTION STANDARD ├─ All transmissions encrypted via HTTPS ├─ Let's Encrypt quantum certificates -├─ Traefik middleware compression active -└─ CORS shields configured per sector +├─ TLS 1.2+ with strong cipher suites only +├─ HSTS enabled (1-year, preload ready) +└─ SNI strict mode enforced -🛡️ ACCESS CONTROL +🛡️ SECURITY HEADERS +├─ X-Frame-Options: SAMEORIGIN +├─ X-XSS-Protection enabled +├─ Content-Type-Options: nosniff +├─ Referrer-Policy configured +└─ Permissions-Policy restrictions + +🔒 ACCESS CONTROL ├─ Admin credentials in .env vault ├─ Database authentication: scram-sha-256 -├─ VPN cloaking device enabled -└─ Email verification for new recruits +├─ HTTP Basic Auth on sensitive endpoints +├─ Rate limiting available (100 req/s) +└─ VPN cloaking device enabled ``` ---