Commit Graph

10 Commits

Author SHA1 Message Date
17512d111e feat: vert auth 2025-11-06 18:05:01 +01:00
0b839edd72 fix: update SCRAPYD_URL to point to /scrapyd path
Updated default SCRAPYD_URL from https://scrapy.pivoine.art to
https://scrapy.pivoine.art/scrapyd to match new routing
2025-11-05 06:43:42 +01:00
00739893ef feat: route scrapy-ui to root and scrapyd to /scrapyd
- scrapy.pivoine.art/ → scrapy-ui (main interface)
- scrapy.pivoine.art/scrapyd → scrapyd daemon
- Added router priorities to avoid conflicts
- Removed /ui prefix from scrapy-ui
2025-11-05 06:41:03 +01:00
321277de45 fix: restore stripprefix middleware for scrapy-ui
Next.js app doesn't have basePath configured, needs path stripping
2025-11-05 06:34:06 +01:00
16645f3ac0 test: remove stripprefix middleware from scrapy-ui 2025-11-05 06:26:29 +01:00
d94daedeef fix: use correct scrapy-ui image tag :main
Changed image tag from :latest to :main for ghcr.io/valknarness/scrapy-ui

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-05 06:16:49 +01:00
6df2de5bb2 feat: add Scrapy UI service with path-based routing
Added scrapy-ui service to scrapy stack:
- Image: ghcr.io/valknarness/scrapy-ui:latest
- Path-based routing: https://scrapy.pivoine.art/ui
- HTTP Basic Auth protection (same credentials as scrapyd)
- StripPrefix middleware to remove /ui from requests
- Auto-connects to scrapyd using SCRAPYD_URL, USERNAME, PASSWORD
- Watchtower auto-updates enabled

Traefik configuration:
- Middleware chain: auth -> stripprefix -> compress
- Routes /ui prefix to port 3000
- Shares authentication with main scrapyd interface

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-05 06:14:18 +01:00
b026878c38 feat: add HTTP Basic Auth to Scrapyd for security
Added Traefik Basic Auth middleware to secure Scrapyd web interface:
- Added basicauth middleware to scrapyd service labels
- Middleware chains auth with compression for HTTPS routes
- Added SCRAPY_AUTH_USERS environment variable to arty.yml
- Credentials stored in .env (htpasswd format with escaped $)

Access to scrapy.pivoine.art now requires username/password.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-04 23:17:13 +01:00
ef646eccfa fix: add scrapy_code named volume for scrapy and scrapyrt services
Both scrapy and scrapyrt services now use the shared scrapy_code named
volume instead of bind mounts. Code directory will be added later.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-04 22:43:55 +01:00
785942da61 feat: add Scrapy, n8n, and Filestash stacks to Falcon
Added three new service stacks to the docker-compose infrastructure:

- **Scrapy stack** (scrapy.pivoine.art):
  - scrapyd: Web scraping daemon with web interface (port 6800)
  - scrapy: Development container for spider commands
  - scrapyrt: Real-time API for running spiders (port 9080)

- **n8n stack** (n8n.pivoine.art):
  - Workflow automation platform with PostgreSQL backend
  - 200+ integrations for automated tasks
  - Runners enabled for task execution
  - Webhook support for external triggers

- **Filestash stack** (stash.pivoine.art):
  - Web-based file manager with multi-backend support
  - Supports SFTP, S3, Dropbox, Google Drive, FTP, WebDAV
  - In-browser file viewing and media playback

Infrastructure updates:
- Updated PostgreSQL init script to create n8n database
- Added environment variables to arty.yml for all three stacks
- Updated compose.yaml include list
- Updated CLAUDE.md and README.md documentation
- Normalized service names in existing stacks (gotify, proxy, umami, vpn)

All services integrated with Traefik for SSL termination and include
Watchtower auto-update labels.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-04 22:36:13 +01:00