valknar/runpod
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b370c16eb9203e3107508729b68ff5d9679ed873
runpod/arty.yml
Sebastian Krüger b370c16eb9 feat: add workflow linking script to arty.yml 
Add new arty script `workflows/link-comfyui` to create symlinks for all
production workflows from git repository to ComfyUI workflows directory.

Script creates symlinks for:
- All 6 workflow category directories (text-to-image, image-to-image,
  image-to-video, text-to-music, upscaling, advanced)
- templates/ directory for future workflow templates
- README.md and WORKFLOW_STANDARDS.md documentation

Usage: arty run workflows/link-comfyui

Updates:
- Added workflows/link-comfyui script with informative output
- Updated Fresh Deployment guide to include workflow linking step
- Added section 6 "ComfyUI Workflows" with complete workflow listing
- Updated Storage section to document workflow symlinks
- Added workflow README to documentation list

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-22 12:12:06 +01:00

259 lines
12 KiB
YAML
Raw Blame History

name: "RunPod AI Model Orchestrator"
version: "2.0.0"
description: "Process-based AI model orchestrator for RunPod GPU instances with ComfyUI integration"
author: "valknar@pivoine.art"
license: "MIT"
# Git repositories to clone for a fresh RunPod deployment
references:
# ComfyUI base installation
- url: https://github.com/comfyanonymous/ComfyUI.git
into: /workspace/ComfyUI
description: "ComfyUI - Node-based interface for image/video/audio generation"
# ComfyUI Essential Custom Nodes
- url: https://github.com/ltdrdata/ComfyUI-Manager.git
into: /workspace/ComfyUI/custom_nodes/ComfyUI-Manager
description: "ComfyUI Manager - Install/manage custom nodes and models"
essential: true
- url: https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite.git
into: /workspace/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite
description: "Video operations and processing"
essential: true
- url: https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved.git
into: /workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved
description: "AnimateDiff for video generation"
essential: true
- url: https://github.com/cubiq/ComfyUI_IPAdapter_plus.git
into: /workspace/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus
description: "IP-Adapter for style transfer"
essential: true
- url: https://github.com/ltdrdata/ComfyUI-Impact-Pack.git
into: /workspace/ComfyUI/custom_nodes/ComfyUI-Impact-Pack
description: "Auto face enhancement and detailer"
essential: true
# ComfyUI Optional Custom Nodes
- url: https://github.com/kijai/ComfyUI-CogVideoXWrapper.git
into: /workspace/ComfyUI/custom_nodes/ComfyUI-CogVideoXWrapper
description: "CogVideoX integration for text-to-video"
essential: false
- url: https://github.com/ltdrdata/ComfyUI-Inspire-Pack.git
into: /workspace/ComfyUI/custom_nodes/ComfyUI-Inspire-Pack
description: "Additional inspiration tools"
essential: false
- url: https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet.git
into: /workspace/ComfyUI/custom_nodes/ComfyUI-Advanced-ControlNet
description: "Advanced ControlNet features"
essential: false
- url: https://github.com/MrForExample/ComfyUI-3D-Pack.git
into: /workspace/ComfyUI/custom_nodes/ComfyUI-3D-Pack
description: "3D asset generation"
essential: false
- url: https://github.com/MixLabPro/comfyui-sound-lab.git
into: /workspace/ComfyUI/custom_nodes/comfyui-sound-lab
description: "MusicGen and Stable Audio integration"
essential: false
# Environment profiles for selective repository management
envs:
# Production: Only essential components
prod:
- /workspace/ai
- /workspace/ComfyUI
- /workspace/ComfyUI/custom_nodes/ComfyUI-Manager
- /workspace/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite
- /workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved
- /workspace/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus
- /workspace/ComfyUI/custom_nodes/ComfyUI-Impact-Pack
# Development: All repositories including optional nodes
dev:
- /workspace/ai
- /workspace/ComfyUI
- /workspace/ComfyUI/custom_nodes/ComfyUI-Manager
- /workspace/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite
- /workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved
- /workspace/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus
- /workspace/ComfyUI/custom_nodes/ComfyUI-Impact-Pack
- /workspace/ComfyUI/custom_nodes/ComfyUI-CogVideoXWrapper
- /workspace/ComfyUI/custom_nodes/ComfyUI-Inspire-Pack
- /workspace/ComfyUI/custom_nodes/ComfyUI-Advanced-ControlNet
- /workspace/ComfyUI/custom_nodes/ComfyUI-3D-Pack
- /workspace/ComfyUI/custom_nodes/comfyui-sound-lab
# Minimal: Only orchestrator and ComfyUI base
minimal:
- /workspace/ai
- /workspace/ComfyUI
- /workspace/ComfyUI/custom_nodes/ComfyUI-Manager
# Deployment scripts for RunPod instances
scripts:
# Initial setup
setup/full: |
cd /workspace/ai
cp .env.example .env
echo "Edit .env and set HF_TOKEN, then run: ansible-playbook playbook.yml"
setup/essential: |
cd /workspace/ai
cp .env.example .env
echo "Edit .env and set HF_TOKEN, then run: ansible-playbook playbook.yml --tags comfyui-essential"
# Model linking (run after models are downloaded)
models/link-comfyui: |
cd /workspace/ComfyUI/models/diffusers
ln -sf /workspace/huggingface_cache/models--black-forest-labs--FLUX.1-schnell FLUX.1-schnell
ln -sf /workspace/huggingface_cache/models--black-forest-labs--FLUX.1-dev FLUX.1-dev
ln -sf /workspace/huggingface_cache/models--stabilityai--stable-diffusion-xl-base-1.0 stable-diffusion-xl-base-1.0
ln -sf /workspace/huggingface_cache/models--stabilityai--stable-diffusion-xl-refiner-1.0 stable-diffusion-xl-refiner-1.0
ln -sf /workspace/huggingface_cache/models--stabilityai--stable-diffusion-3.5-large stable-diffusion-3.5-large
cd /workspace/ComfyUI/models/clip_vision
ln -sf /workspace/huggingface_cache/models--openai--clip-vit-large-patch14 clip-vit-large-patch14
ln -sf /workspace/huggingface_cache/models--laion--CLIP-ViT-bigG-14-laion2B-39B-b160k CLIP-ViT-bigG-14
ln -sf /workspace/huggingface_cache/models--google--siglip-so400m-patch14-384 siglip-so400m-patch14-384
cd /workspace/ComfyUI/models/diffusion_models
ln -sf /workspace/huggingface_cache/models--THUDM--CogVideoX-5b CogVideoX-5b
ln -sf /workspace/huggingface_cache/models--stabilityai--stable-video-diffusion-img2vid stable-video-diffusion-img2vid
ln -sf /workspace/huggingface_cache/models--stabilityai--stable-video-diffusion-img2vid-xt stable-video-diffusion-img2vid-xt
echo "Models linked to ComfyUI"
# Workflow linking (link production workflows from git repo)
workflows/link-comfyui: |
mkdir -p /workspace/ComfyUI/workflows
cd /workspace/ComfyUI/workflows
ln -sf /workspace/ai/models/comfyui/workflows/text-to-image text-to-image
ln -sf /workspace/ai/models/comfyui/workflows/image-to-image image-to-image
ln -sf /workspace/ai/models/comfyui/workflows/image-to-video image-to-video
ln -sf /workspace/ai/models/comfyui/workflows/text-to-music text-to-music
ln -sf /workspace/ai/models/comfyui/workflows/upscaling upscaling
ln -sf /workspace/ai/models/comfyui/workflows/advanced advanced
ln -sf /workspace/ai/models/comfyui/workflows/templates templates
ln -sf /workspace/ai/models/comfyui/workflows/README.md README.md
ln -sf /workspace/ai/models/comfyui/workflows/WORKFLOW_STANDARDS.md WORKFLOW_STANDARDS.md
echo "Production workflows linked to ComfyUI"
echo " - 4 text-to-image workflows (FLUX, SDXL, SD3.5)"
echo " - 3 image-to-image workflows (IP-Adapter)"
echo " - 3 image-to-video workflows (CogVideoX, SVD)"
echo " - 4 text-to-music workflows (MusicGen)"
echo " - 3 upscaling workflows"
echo " - 3 advanced workflows (ControlNet, AnimateDiff, Batch)"
echo ""
echo "Access workflows at: /workspace/ComfyUI/workflows/"
# Service management (Supervisor-based)
services/start: bash /workspace/ai/scripts/start-all.sh
services/stop: bash /workspace/ai/scripts/stop-all.sh
services/restart: bash /workspace/ai/scripts/stop-all.sh && bash /workspace/ai/scripts/start-all.sh
services/status: bash /workspace/ai/scripts/status.sh
services/logs: supervisorctl -c /workspace/supervisord.conf tail -f comfyui
services/logs-orchestrator: supervisorctl -c /workspace/supervisord.conf tail -f orchestrator
services/restart-comfyui: supervisorctl -c /workspace/supervisord.conf restart comfyui
services/restart-orchestrator: supervisorctl -c /workspace/supervisord.conf restart orchestrator
services/start-orchestrator: supervisorctl -c /workspace/supervisord.conf start orchestrator
services/stop-orchestrator: supervisorctl -c /workspace/supervisord.conf stop orchestrator
# Dependency installation
deps/comfyui-nodes: |
pip3 install -r /workspace/ComfyUI/custom_nodes/ComfyUI-Manager/requirements.txt
pip3 install -r /workspace/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite/requirements.txt
pip3 install 'numpy<2.0.0' --force-reinstall
echo "Custom node dependencies installed"
# Ansible provisioning shortcuts
ansible/base: cd /workspace/ai && ansible-playbook playbook.yml --tags base,python,dependencies
ansible/supervisor: cd /workspace/ai && ansible-playbook playbook.yml --tags supervisor
ansible/vllm: cd /workspace/ai && ansible-playbook playbook.yml --tags models
ansible/comfyui: cd /workspace/ai && ansible-playbook playbook.yml --tags comfyui,comfyui-essential
ansible/comfyui-all: cd /workspace/ai && ansible-playbook playbook.yml --tags comfyui,comfyui-models-all,comfyui-nodes
ansible/full: cd /workspace/ai && ansible-playbook playbook.yml
# Health checks
health/orchestrator: curl http://localhost:9000/health
health/comfyui: curl http://localhost:8188
health/vllm: curl http://localhost:8000/health
# System checks
check/gpu: nvidia-smi
check/disk: df -h /workspace
check/models: du -sh /workspace/huggingface_cache
check/cache: find /workspace/huggingface_cache -type d -name 'models--*' -maxdepth 1
# Deployment notes
notes: |
RunPod AI Model Orchestrator - Quick Start
1. Fresh Deployment:
- Clone repositories: arty sync --env prod
- Configure environment: cd /workspace/ai && cp .env.example .env
- Set HF_TOKEN in .env file
- Run Ansible: ansible-playbook playbook.yml --tags comfyui-essential
- Link models: arty run models/link-comfyui
- Link workflows: arty run workflows/link-comfyui
- Install node deps: arty run deps/comfyui-nodes
- Start services: arty run services/start
2. Model Downloads:
- Essential (~80GB): ansible-playbook playbook.yml --tags comfyui-essential
- All models (~137GB): ansible-playbook playbook.yml --tags comfyui-models-all
3. Service Management:
- Start: arty run services/start
- Stop: arty run services/stop
- Restart: arty run services/restart
4. Health Checks:
- Orchestrator: arty run health/orchestrator
- ComfyUI: arty run health/comfyui
- vLLM: arty run health/vllm
5. Environment Profiles:
- Production (essential only): arty sync --env prod
- Development (all nodes): arty sync --env dev
- Minimal (orchestrator + ComfyUI only): arty sync --env minimal
6. ComfyUI Workflows:
- Link workflows: arty run workflows/link-comfyui
- Location: /workspace/ComfyUI/workflows/
- 20 production-ready workflows across 6 categories
- Text-to-Image: FLUX Schnell, FLUX Dev, SDXL+Refiner, SD3.5
- Image-to-Image: IP-Adapter (style, face, composition)
- Image-to-Video: CogVideoX, SVD, SVD-XT
- Text-to-Music: MusicGen (small/medium/large/melody)
- Upscaling: Ultimate SD, Simple, Face-focused
- Advanced: ControlNet, AnimateDiff, Batch processing
- Documentation: README.md, WORKFLOW_STANDARDS.md
7. Important Files:
- Configuration: /workspace/ai/playbook.yml
- Model registry: /workspace/ai/model-orchestrator/models.yaml
- Environment: /workspace/ai/.env
- Services: /workspace/ai/scripts/*.sh
8. Ports:
- Orchestrator: 9000
- ComfyUI: 8188
- vLLM: 8000+
9. Storage:
- Models cache: /workspace/huggingface_cache (~401GB)
- ComfyUI models: /workspace/ComfyUI/models (symlinks to cache)
- ComfyUI workflows: /workspace/ComfyUI/workflows (symlinks to git)
- Project: /workspace/ai
For detailed documentation, see:
- /workspace/ai/README.md
- /workspace/ai/CLAUDE.md
- /workspace/ai/COMFYUI_MODELS.md
- /workspace/ai/MODELS_LINKED.md
- /workspace/ai/models/comfyui/workflows/README.md
Reference in New Issue View Git Blame Copy Permalink