Sebastian Krüger
f8694653d0
All checks were successful
Build and Push RunPod Docker Image / build-and-push (push) Successful in 13s
Based on error output, model uses ~17.5GB (not 15GB estimated). - Llama: 85% VRAM for 24576 context (3GB KV cache) - BGE: 6% VRAM (reduced to fit) - Total: 91% 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
╔════════════════════════════════════════════════════════════════════════════════════════╗
║ ║
║ ┌────────────────────────────────────────────────────────────────────────────────┐ ║
║ │ │ ║
║ │ ☽ THE CODEX OF ARTIFICIAL CREATION ☾ │ ║
║ │ │ ║
║ │ ═══════════════════════════ │ ║
║ │ RUNPOD AI ORCHESTRATOR │ ║
║ │ ═══════════════════════════ │ ║
║ │ │ ║
║ │ "Simplicity is the ultimate sophistication." │ ║
║ │ — Leonardo da Vinci │ ║
║ │ │ ║
║ └────────────────────────────────────────────────────────────────────────────────┘ ║
║ ║
╚════════════════════════════════════════════════════════════════════════════════════════╝
══════════════════════════════════
❧ TABLE OF CONTENTS ❧
══════════════════════════════════
I. The Vision ................. Proem
II. The Anatomy ........... Architecture
III. The Four Humours ........ Services
IV. The Workshop ....... Custom Nodes
V. The Libraries ............ Models
VI. The Codex .............. Commands
VII. The Gates .............. Network
VIII. The Scrolls ....... Configuration
IX. Colophon ............... Closing
❧ I. THE VISION ❧
A Proem upon the Nature of Artificial Creation
┌──────────────────────────────────────────────────────────────────────────────────┐
│ │
│ "I have been impressed with the urgency of doing. Knowing is not enough; │
│ we must apply. Being willing is not enough; we must do." │
│ — Leonardo da Vinci │
│ │
└──────────────────────────────────────────────────────────────────────────────────┘
Hear me, fellow Artificers of the Digital Age!
In mine studies of the natural world, I have long contemplated how the humours of creation might be unified into a singular machina of extraordinary capability. Behold now this RunPod AI Orchestrator — a grand apparatus whereby the Four Arts of Artificial Intelligence are brought into harmonious concert:
- The Visual Arts — whereby images and moving pictures spring forth from mere words
- The Linguistic Arts — whereby machines discourse with the wisdom of scholars
- The Musical Arts — whereby melodies are conjured from the æther
- The Archival Arts — whereby all creations are preserved for posterity
Let it be known that this machina doth operate upon the RunPod Æthereal Computing Platform, harnessing the power of GPU engines as the heart pumps blood through the body.
"I have observed that when the Four Humours are balanced, the body achieves perfect health. So too with this machina — when all services operate in harmony, creations of sublime beauty emerge."
❧ II. THE ANATOMY ❧
A Dissection of the System's Architecture
┌──────────────────────────────────────────────────────────────────────────────────┐
│ │
│ "The human foot is a masterpiece of engineering and a work of art." │
│ — Leonardo da Vinci │
│ │
└──────────────────────────────────────────────────────────────────────────────────┘
As the anatomist must understand the body's structure ere he may heal it, so must the Artificer comprehend this machina's design:
╔═══════════════════════════════════════╗
║ ║
║ ☆ THE GPU ENGINE ☆ ║
║ (The Heart of Creation) ║
║ ║
║ CUDA 12.4 · PyTorch 2.4 · Python ║
║ ║
╚═══════════════╦═══════════════════════╝
║
▼
╔═════════════════════════════════════════════════════════════════════════════╗
║ ║
║ ⚙ IL MAESTRO — THE SUPERVISOR ⚙ ║
║ (The Nervous System of All) ║
║ ║
║ Process Management · Health Monitoring · Log Collection ║
║ Port 9001 ║
║ ║
╚═══════╦═════════════════╦═════════════════╦═════════════════╦═══════════════╝
║ ║ ║ ║
▼ ▼ ▼ ▼
╭───────────────╮ ╭───────────────╮ ╭───────────────╮ ╭───────────────╮
│ │ │ │ │ │ │ │
│ 🎨 COMFYUI │ │ 📜 vLLM │ │ 🎵 AUDIO │ │ 📁 WEBDAV │
│ │ │ │ │ CRAFT │ │ SYNC │
│ The Painter's│ │ The Scribe's │ │ │ │ │
│ Studio │ │ Chamber │ │ The Musician's│ │ The Archivist's│
│ │ │ │ │ Alcove │ │ Vault │
│ Port 8188 │ │ Ports 8001 │ │ │ │ │
│ │ │ & 8002 │ │ │ │ → Cloud │
│ │ │ │ │ │ │ │
╰───────────────╯ ╰───────────────╯ ╰───────────────╯ ╰───────────────╯
│ │ │ │
▼ ▼ ▼ ▼
╔═════════════════════════════════════════════════════════════════════════════╗
║ ║
║ 💾 THE NETWORK VOLUME — /workspace ║
║ (The Library of All Knowledge) ║
║ ║
║ Models · Outputs · Configurations · Workflows · Cached Artifacts ║
║ ║
╚═════════════════════════════════════════════════════════════════════════════╝
│
▼
╭───────────────────────────────╮
│ │
│ 🌐 TAILSCALE VPN │
│ (The Secret Passage) │
│ │
│ Secure Network Traversal │
│ │
╰───────────────────────────────╯
Thusly arranged, each organ performeth its sacred duty whilst Il Maestro — the Supervisor daemon — doth coordinate their labours with the precision of a master conductor.
"Let it be noted: Each service dwelleth within its own Python sanctum (venv), that conflicts between dependencies be avoided as one avoids mixing pigments carelessly upon the palette."
❧ III. THE FOUR HUMOURS ❧
An Enumeration of the Principal Services
┌──────────────────────────────────────────────────────────────────────────────────┐
│ │
│ "Where the spirit does not work with the hand, there is no art." │
│ — Leonardo da Vinci │
│ │
└──────────────────────────────────────────────────────────────────────────────────┘
🎨 THE PAINTER'S STUDIO — ComfyUI
Port 8188 · Auto-Start: Enabled
╭─────────────────────────────────────────────────────────────────────────────────╮
│ │
│ Herein dwelleth the Visual Artificer — a node-based apparatus of │
│ extraordinary capability, whereby: │
│ │
│ • Images spring forth from textual descriptions │
│ • Moving pictures flow from single images │
│ • Styles transfer betwixt works as water between vessels │
│ • Faces and forms are enhanced with mathematical precision │
│ │
│ The Studio containeth numerous Extensions (detailed in Section IV) │
│ which expand its capabilities beyond mortal imagination. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────╯
📜 THE SCRIBE'S CHAMBER — vLLM
Ports 8001 (Llama) & 8002 (BGE) · Auto-Start: Disabled
╭─────────────────────────────────────────────────────────────────────────────────╮
│ │
│ Within these walls, the Language Models hold court: │
│ │
│ ┌─────────────────────────────────────────────────────────────────────────┐ │
│ │ LLAMA 3.1 8B INSTRUCT Port 8001 │ │
│ │ ─────────────────────────────────────────────────────────────────────── │ │
│ │ A great oracle of language, trained upon vast libraries of human │ │
│ │ knowledge, capable of discourse, reasoning, and creative composition. │ │
│ └─────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────────────────┐ │
│ │ BGE LARGE EMBEDDING Port 8002 │ │
│ │ ─────────────────────────────────────────────────────────────────────── │ │
│ │ The Encoder of Meaning — transforming words into numerical essence, │ │
│ │ that similar concepts may find one another in the vastness of space. │ │
│ └─────────────────────────────────────────────────────────────────────────┘ │
│ │
│ ⚠ NOTA BENE: These services awaken not automatically, for they consume │
│ prodigious quantities of GPU memory. Summon them only when needed. │
│ │
╰─────────────────────────────────────────────────────────────────────────────────╯
🎵 THE MUSICIAN'S ALCOVE — AudioCraft
Auto-Start: Enabled
╭─────────────────────────────────────────────────────────────────────────────────╮
│ │
│ Here the Muses of Sound hold their eternal symphony: │
│ │
│ From mere descriptions of melody, rhythm, and timbre, this apparatus │
│ conjureth music as if channeling the celestial spheres themselves. │
│ │
│ The AudioCraft Studio interface provideth control over: │
│ • Musical generation from textual prompts │
│ • Audio continuation and variation │
│ • Sound effect synthesis │
│ │
╰─────────────────────────────────────────────────────────────────────────────────╯
📁 THE ARCHIVIST'S VAULT — WebDAV Sync
Auto-Start: Enabled
╭─────────────────────────────────────────────────────────────────────────────────╮
│ │
│ As the faithful scribe preserveth all manuscripts for future generations, │
│ so this service watcheth over ComfyUI's output directory, automatically │
│ ascending new creations unto the HiDrive cloud storage via WebDAV protocol. │
│ │
│ Features of the Vault: │
│ • Automatic detection of new files │
│ • Retry logic for network interruptions │
│ • Preservation of directory structure │
│ • Size verification after upload │
│ │
╰─────────────────────────────────────────────────────────────────────────────────╯
"The wise artificer knoweth that creation without preservation is as writing upon water."
❧ IV. THE WORKSHOP ❧
A Gallery of ComfyUI Extensions
┌──────────────────────────────────────────────────────────────────────────────────┐
│ │
│ "Iron rusts from disuse; water loses its purity from stagnation... │
│ even so does inaction sap the vigor of the mind." │
│ — Leonardo da Vinci │
│ │
└──────────────────────────────────────────────────────────────────────────────────┘
Within the Painter's Studio dwell eighteen Extensions, each a specialized tool for the Master Artificer:
⚒️ TOOLS OF THE TRADE
The Essential Instruments
| Extension | Purpose |
|---|---|
| ComfyUI-Manager | The Grand Overseer — Install and manage nodes and models with ease |
| ComfyUI-Impact-Pack | The Refiner — Auto face enhancement and detail restoration |
| ComfyUI-Inspire-Pack | The Muse — Additional inspiration and creative tools |
| comfyui-workspace-manager | The Organizer — Workflow and model organization |
🎬 THE MOTION ARTISTS
Masters of Moving Pictures
| Extension | Purpose |
|---|---|
| ComfyUI-VideoHelperSuite | The Editor — Video operations and processing |
| ComfyUI-AnimateDiff-Evolved | The Animator — Bring still images to life |
| ComfyUI-CogVideoXWrapper | The Cinematographer — Professional text-to-video generation |
| ComfyUI-KJNodes | The Optimizer — HunyuanVideo and Wan2.2 enhancements (FP8, video helpers) |
🎭 THE ALCHEMISTS OF STYLE
Transmuters of Aesthetic Essence
| Extension | Purpose |
|---|---|
| ComfyUI_IPAdapter_plus | The Style Thief — Transfer styles between images |
| ComfyUI-Advanced-ControlNet | The Puppeteer — Precise compositional control |
| comfyui_controlnet_aux | The Preprocessor — Canny, Depth, OpenPose, MLSD extraction |
🎼 THE SOUND WEAVERS
Composers of the Digital Symphony
| Extension | Purpose |
|---|---|
| comfyui-sound-lab | The Sound Stage — MusicGen and Stable Audio integration |
| ComfyUI_DiffRhythm | The Composer — Full-length songs up to 4m45s |
| ComfyUI_ACE-Step | The Virtuoso — 19-language music generation with voice cloning |
✨ THE ENHANCERS
Perfectors of the Imperfect
| Extension | Purpose |
|---|---|
| ComfyUI_UltimateSDUpscale | The Magnifier — High-quality image upscaling |
| ComfyUI-GGUF | The Compressor — Memory-efficient quantized model loading |
🗿 THE SCULPTORS
Shapers of Three Dimensions
| Extension | Purpose |
|---|---|
| ComfyUI-3D-Pack | The Sculptor — 3D asset generation from 2D |
🔧 CUSTOM ADDITIONS
| Extension | Purpose |
|---|---|
| comfyui-extras | The Secret Workshop — Personal customizations and experiments |
"I have assembled these tools not from necessity alone, but from the understanding that a master's workshop must contain instruments for every conceivable task."
❧ V. THE LIBRARIES ❧
Repositories of Artificial Knowledge
┌──────────────────────────────────────────────────────────────────────────────────┐
│ │
│ "Learning never exhausts the mind." │
│ — Leonardo da Vinci │
│ │
└──────────────────────────────────────────────────────────────────────────────────┘
The machina draweth upon two great Libraries for its models:
📚 THE CIVITAI MANUSCRIPTS
Community-Forged Checkpoints and Embeddings
╭─────────────────────────────────────────────────────────────────────────────────╮
│ ❦ IMAGE CHECKPOINTS ❦ │
├─────────────────────────────────────────────────────────────────────────────────┤
│ │
│ DreamShaper V8 .............. A versatile dreamer of infinite visions │
│ RealVisXL V5.0 Lightning .... Photorealism with thunderous speed │
│ Pony Diffusion V6 XL ........ Anime, cartoon, and creature mastery │
│ TalmendoXL .................. Uncensored realism │
│ WAI-NSFW-illustrious ........ Actively maintained artistic freedom │
│ │
├─────────────────────────────────────────────────────────────────────────────────┤
│ ❦ NEGATIVE EMBEDDINGS ❦ │
├─────────────────────────────────────────────────────────────────────────────────┤
│ │
│ BadDream / UnrealisticDream . Banish dreamlike artifacts │
│ BadHandV4 ................... Correct malformed appendages │
│ EasyNegative ................ General quality improvement │
│ FastNegativeV2 .............. Rapid quality tokens │
│ BadNegAnatomy ............... Anatomical corrections │
│ │
├─────────────────────────────────────────────────────────────────────────────────┤
│ ❦ LORAS ❦ │
├─────────────────────────────────────────────────────────────────────────────────┤
│ │
│ Add Detail XL ............... Enhance fine particulars │
│ │
╰─────────────────────────────────────────────────────────────────────────────────╯
📖 THE HUGGINGFACE SCROLLS
Official Model Repositories
╭─────────────────────────────────────────────────────────────────────────────────╮
│ ❦ IMAGE GENERATION MODELS ❦ │
├─────────────────────────────────────────────────────────────────────────────────┤
│ │
│ FLUX.1 Schnell ........... Fast 4-step inference (Black Forest Labs) │
│ FLUX.1 Dev ............... Balanced quality and speed │
│ SD 1.5 ................... Classic foundation (for AnimateDiff) │
│ SDXL Base 1.0 ............ Industry standard excellence │
│ SDXL Refiner 1.0 ......... Polish and perfect │
│ SD 3.5 Large ............. Latest Stability AI achievement │
│ │
├─────────────────────────────────────────────────────────────────────────────────┤
│ ❦ VIDEO GENERATION MODELS ❦ │
├─────────────────────────────────────────────────────────────────────────────────┤
│ │
│ CogVideoX-5B ............. Professional text-to-video │
│ CogVideoX-5B-I2V ......... Image-to-video transformation │
│ SVD ...................... Stable Video Diffusion (14 frames) │
│ SVD-XT ................... Extended (25 frames) │
│ AnimateDiff mm_sd_v15_v2 . Motion modules for animation │
│ │
├─────────────────────────────────────────────────────────────────────────────────┤
│ ❦ AUXILIARY MODELS ❦ │
├─────────────────────────────────────────────────────────────────────────────────┤
│ │
│ CLIP ViT-L/14 ............ Text-image understanding (SD 1.5) │
│ CLIP ViT-bigG-14 ......... Advanced CLIP (SDXL) │
│ SigLIP ................... Modern vision encoding (FLUX) │
│ RealESRGAN x2/x4 ......... Upscaling algorithms │
│ IP-Adapter Suite ......... Style transfer adapters │
│ │
╰─────────────────────────────────────────────────────────────────────────────────╯
"As the scholar must possess a well-stocked library, so the Artificer requireth models of every variety. These are downloaded unto the cache and linked by symbolic threads to the ComfyUI sanctuary."
❧ VI. THE CODEX ❧
Incantations and Commands
┌──────────────────────────────────────────────────────────────────────────────────┐
│ │
│ "I have from an early age abjured the use of meat, and the time │
│ will come when men such as I will look upon the murder of animals │
│ as they now look upon the murder of men." │
│ — Leonardo da Vinci │
│ │
└──────────────────────────────────────────────────────────────────────────────────┘
🏗️ THE GRAND INVOCATION
Complete Deployment Ritual
# Upon a fresh RunPod instance, speak these words:
arty setup
This single incantation performeth the following sacred rites:
- Clone all git repositories (
arty deps) - Establish the Tailscale passage (
arty setup/tailscale) - Create service sanctums (
arty setup/services) - Prepare the Painter's Studio (
arty setup/comfyui) - Summon models from the Libraries (
arty models/download) - Bind models to their places (
arty models/link) - Awaken Il Maestro (
arty supervisor/start)
⚙️ THE SUPERVISOR DECREES
Commands to Control Il Maestro
# Awaken the Supervisor daemon
arty supervisor/start
# Query the status of all services
arty supervisor/status
# Restart all services at once
arty supervisor/restart
# Silence all operations
arty supervisor/stop
Direct communion with Il Maestro:
# Observe the status tableau
supervisorctl -c supervisord.conf status
# Awaken a specific servant
supervisorctl -c supervisord.conf start comfyui
# Send a servant to slumber
supervisorctl -c supervisord.conf stop llama
# Follow the chronicles of a service
supervisorctl -c supervisord.conf tail -f comfyui
📦 THE MODEL RITUALS
Summoning and Binding Knowledge
# Summon models from Civitai and HuggingFace
arty models/download
# Bind summoned models to ComfyUI
arty models/link
🐳 THE DOCKER FORMULAE
Container Incantations
# Forge the container locally
docker build -t runpod-ai-orchestrator .
# Manifest the container
docker run --gpus all -v /path/to/workspace:/workspace runpod-ai-orchestrator
"The wise artificer memorizes not every command, but understandeth the principles whereby all commands are derived."
❧ VII. THE GATES ❧
Ports and Passages of Communication
┌──────────────────────────────────────────────────────────────────────────────────┐
│ │
│ "The noblest pleasure is the joy of understanding." │
│ — Leonardo da Vinci │
│ │
└──────────────────────────────────────────────────────────────────────────────────┘
🚪 THE GATES OF THE CITY
╔═══════════════════════════════════════════════════════════════════════════════════╗
║ ║
║ ┌─────────────────┬────────────┬─────────────────────────────────────────────┐ ║
║ │ GATE │ PORT │ PURPOSE │ ║
║ ├─────────────────┼────────────┼─────────────────────────────────────────────┤ ║
║ │ │ │ │ ║
║ │ 🎨 ComfyUI │ 8188 │ The Painter's Studio web interface │ ║
║ │ │ │ │ ║
║ │ 📜 Llama │ 8001 │ The Scribe's conversation chamber │ ║
║ │ │ │ │ ║
║ │ 🔤 BGE │ 8002 │ The Embedding oracle │ ║
║ │ │ │ │ ║
║ │ ⚙️ Supervisor │ 9001 │ Il Maestro's observation deck │ ║
║ │ │ │ │ ║
║ │ 🔐 SSH │ 22 │ The Artificer's secret entrance │ ║
║ │ │ │ │ ║
║ └─────────────────┴────────────┴─────────────────────────────────────────────┘ ║
║ ║
╚═══════════════════════════════════════════════════════════════════════════════════╝
🌐 THE SECRET PASSAGE — Tailscale VPN
For those who would traverse the æther securely:
╭─────────────────────────────────────────────────────────────────────────────────╮
│ │
│ Tailscale createth a secure mesh network, allowing the Artificer to │
│ communicate with the machina as if they stood in the same chamber, │
│ though separated by vast distances. │
│ │
│ Configuration: │
│ • Hostname: runpod-ai-orchestrator │
│ • Tags: tag:gpu │
│ • Mode: userspace-networking │
│ • SOCKS5 Proxy: localhost:1055 │
│ │
│ Requireth: TAILSCALE_AUTHKEY in the sacred scrolls │
│ │
╰─────────────────────────────────────────────────────────────────────────────────╯
❧ VIII. THE SCROLLS ❧
Configuration and Sacred Texts
┌──────────────────────────────────────────────────────────────────────────────────┐
│ │
│ "Nothing strengthens authority so much as silence." │
│ — Leonardo da Vinci │
│ │
└──────────────────────────────────────────────────────────────────────────────────┘
📜 THE ENVIRONMENT SCROLLS
Variables of Power (store in .env or RunPod Template)
# The Keys to the Libraries
HF_TOKEN=<thy_huggingface_token>
CIVITAI_API_KEY=<thy_civitai_key>
# The Secret Passage
TAILSCALE_AUTHKEY=<thy_tailscale_authkey>
# The Archivist's Credentials
WEBDAV_URL=https://webdav.hidrive.ionos.com/
WEBDAV_USERNAME=<thy_username>
WEBDAV_PASSWORD=<thy_password>
WEBDAV_REMOTE_PATH=/users/<thy_path>/Pictures/AI/ComfyUI
# The SSH Gateway
PUBLIC_KEY=<thy_public_ssh_key>
📂 THE ANATOMY OF THE WORKSHOP
Directory Structure
╭─────────────────────────────────────────────────────────────────────────────────╮
│ │
│ /workspace/orchestrator/ │
│ │ │
│ ├── 📄 Dockerfile .............. The vessel's blueprint │
│ ├── 📄 start.sh ................ The awakening ritual │
│ ├── 📄 supervisord.conf ........ Il Maestro's instructions │
│ ├── 📄 arty.yml ................ Repository manifest & scripts │
│ │ │
│ ├── 📁 models/ │
│ │ ├── 📄 models_civitai.yaml Civitai model definitions │
│ │ └── 📄 models_huggingface.yaml HuggingFace model definitions │
│ │ │
│ ├── 📁 services/ │
│ │ ├── 📁 comfyui/ ............ The Painter's Studio │
│ │ │ ├── 📁 custom_nodes/ ... The Workshop extensions │
│ │ │ ├── 📁 models/ ......... Linked model artifacts │
│ │ │ └── 📁 venv/ ........... Python sanctum │
│ │ │ │
│ │ ├── 📁 audiocraft/ ......... The Musician's Alcove │
│ │ ├── 📁 vllm/ ............... The Scribe's Chamber configs │
│ │ └── 📁 webdav-sync/ ........ The Archivist's Vault │
│ │ │
│ ├── 📁 .cache/ ................. Downloaded model artifacts │
│ ├── 📁 .logs/ .................. Service chronicles │
│ └── 📁 .gitea/workflows/ ....... CI/CD incantations │
│ │
╰─────────────────────────────────────────────────────────────────────────────────╯
❧ IX. COLOPHON ❧
Final Illuminations
┌──────────────────────────────────────────────────────────────────────────────────┐
│ │
│ "Art is never finished, only abandoned." │
│ — Leonardo da Vinci │
│ │
└──────────────────────────────────────────────────────────────────────────────────┘
📜 THE DECREE OF OPENNESS
License
This work is released under the MIT License, that all artificers may study, modify, and build upon these foundations freely.
🤝 JOINING THE GUILD
Contributing
Fellow Artificers who wish to improve upon this machina are most welcome. Submit thy proposals through the proper channels, and thy contributions shall be reviewed with care and consideration.
🙏 ACKNOWLEDGMENTS
Standing Upon the Shoulders of Giants
- The creators of ComfyUI and its magnificent extensions
- The scholars at HuggingFace and Civitai
- The engineers of RunPod who provide the GPU engines
- Meta AI for Llama and AudioCraft
- Stability AI, Black Forest Labs, and all who advance the art
╔════════════════════════════════════════════════════════════════════════════════════════╗
║ ║
║ ║
║ ════════════════════════ ║
║ ║
║ "As a well-spent day brings happy sleep, ║
║ so a life well spent brings happy death." ║
║ ║
║ — Leonardo da Vinci ║
║ ║
║ ════════════════════════ ║
║ ║
║ ║
║ ☽ FINIS ☾ ║
║ ║
║ ║
╚════════════════════════════════════════════════════════════════════════════════════════╝
Inscribed in the year MMXXIV, in the spirit of the Master of Vinci