runpod/comfyui/workflows

ComfyUI Production Workflows

Comprehensive collection of production-ready ComfyUI workflows for RunPod AI Model Orchestrator.

Overview

This directory contains 20 sophisticated, battle-tested workflows designed for production use with the RunPod orchestrator. Each workflow is optimized for 24GB VRAM and includes API compatibility, error handling, and quality gates.

Directory Structure

workflows/
├── text-to-image/       # Text-to-image generation workflows
├── image-to-image/      # Image-to-image transformation workflows
├── image-to-video/      # Image-to-video animation workflows
├── text-to-music/       # Text-to-music generation workflows
├── upscaling/           # Image upscaling and enhancement workflows
├── advanced/            # Advanced multi-model workflows
├── templates/           # Reusable workflow templates
├── README.md            # This file
└── WORKFLOW_STANDARDS.md # Workflow development standards

Workflows by Category

Text-to-Image (4 workflows)

Workflow	Model	Speed	Quality	Use Case
flux-schnell-t2i-production-v1.json	FLUX.1-schnell	Fast (4 steps)	Good	Rapid prototyping, iteration
flux-dev-t2i-production-v1.json	FLUX.1-dev	Medium (20-50 steps)	Excellent	High-quality final images
sdxl-refiner-t2i-production-v1.json	SDXL + Refiner	Medium (30+20 steps)	Excellent	Detailed, refined outputs
sd35-large-t2i-production-v1.json	SD3.5-large	Medium (28 steps)	Excellent	Latest Stable Diffusion

Image-to-Image (3 workflows)

Workflow	Technique	Use Case
ipadapter-style-i2i-production-v1.json	IP-Adapter	Style transfer, composition
ipadapter-face-i2i-production-v1.json	IP-Adapter + Face	Portrait generation, face swap
ipadapter-composition-i2i-production-v1.json	IP-Adapter Multi	Complex scene composition

Image-to-Video (3 workflows)

Workflow	Model	Length	Use Case
cogvideox-i2v-production-v1.json	CogVideoX-5b	6s @ 8fps	AI-driven video generation
svd-i2v-production-v1.json	SVD	14 frames	Quick animations
svd-xt-i2v-production-v1.json	SVD-XT	25 frames	Extended animations

Text-to-Music (4 workflows)

Workflow	Model	Duration	Use Case
musicgen-small-t2m-production-v1.json	MusicGen-small	30s	Fast generation, low VRAM
musicgen-medium-t2m-production-v1.json	MusicGen-medium	30s	Balanced quality/speed
musicgen-large-t2m-production-v1.json	MusicGen-large	30s	Highest quality
musicgen-melody-t2m-production-v1.json	MusicGen-melody	30s	Melody conditioning

Upscaling (3 workflows)

Workflow	Technique	Scale	Use Case
ultimate-sd-upscale-production-v1.json	Ultimate SD	2x-4x	Professional upscaling with detailing
simple-upscale-production-v1.json	Model-based	2x-4x	Fast, straightforward upscaling
face-upscale-production-v1.json	Face-focused	2x	Portrait enhancement

Advanced (3 workflows)

Workflow	Technique	Use Case
controlnet-fusion-production-v1.json	Multi-ControlNet	Precise composition control
animatediff-video-production-v1.json	AnimateDiff	Text-to-video animation
batch-pipeline-production-v1.json	Batch processing	Multiple variations

Quick Start

Using with ComfyUI Web Interface

1. Open ComfyUI at http://localhost:8188
2. Click "Load" button
3. Navigate to /workspace/ai/models/comfyui/workflows/
4. Select desired workflow category and file
5. Adjust parameters as needed
6. Click "Queue Prompt"

Using with RunPod Orchestrator API

# Example: FLUX Schnell text-to-image
curl -X POST http://localhost:9000/api/comfyui/generate \
  -H "Content-Type: application/json" \
  -d '{
    "workflow": "text-to-image/flux-schnell-t2i-production-v1.json",
    "inputs": {
      "prompt": "A serene mountain landscape at sunset",
      "seed": 42,
      "steps": 4
    }
  }'

# Example: Image upscaling
curl -X POST http://localhost:9000/api/comfyui/generate \
  -H "Content-Type: application/json" \
  -d '{
    "workflow": "upscaling/ultimate-sd-upscale-production-v1.json",
    "inputs": {
      "image": "path/to/image.png",
      "scale": 2
    }
  }'

Workflow Features

All production workflows include:

• API Compatibility: Input/output nodes for orchestrator integration
• Error Handling: Validation, fallback nodes, graceful degradation
• Quality Gates: Preview nodes, checkpoints, validation steps
• VRAM Optimization: Model unloading, efficient memory management
• Documentation: Embedded descriptions, parameter guides
• Versioning: Semantic versioning in filenames

Model Requirements

Required Models (Essential)

These models are required by most workflows and are auto-downloaded by Ansible:

• FLUX.1-schnell: Fast text-to-image (17GB)
• FLUX.1-dev: High-quality text-to-image (23GB)
• SDXL Base + Refiner: Stable Diffusion XL (13GB)
• SD3.5-large: Latest Stable Diffusion (16GB)
• CLIP ViT-L/14: Image-text understanding (1.7GB)

Optional Models

• CogVideoX-5b: Text-to-video, image-to-video (9.7GB)
• SVD/SVD-XT: Image-to-video (10GB)
• MusicGen variants: Text-to-music (1.5-3.4GB)
• IP-Adapter: Image conditioning (varies)
• ControlNet models: Precise control (varies)

Check /workspace/ai/COMFYUI_MODELS.md for complete model list.

VRAM Considerations

All workflows are designed for 24GB VRAM with these optimizations:

• Sequential Loading: Only one heavy model loaded at a time
• Model Unloading: Explicit cleanup between stages
• Attention Slicing: Enabled for large models
• VAE Tiling: For high-resolution processing
• Batch Size Limits: Capped at VRAM-safe values

Performance Tips

For Speed

• Use FLUX Schnell (4 steps) or SDXL base (20 steps)
• Lower resolution: 512x512 or 768x768
• Disable refiners and upscalers
• Use --lowvram flag if needed

For Quality

• Use FLUX Dev (50 steps) or SDXL + Refiner
• Higher resolution: 1024x1024 or higher
• Enable face enhancement (Impact-Pack)
• Use Ultimate SD Upscale for final output

For VRAM Efficiency

• Enable model unloading between stages
• Use VAE tiling for >1024px images
• Process batches sequentially, not in parallel
• Monitor with nvidia-smi during generation

Troubleshooting

Out of Memory (OOM) Errors

# Check VRAM usage
nvidia-smi

# Solutions:
# 1. Lower resolution
# 2. Reduce batch size
# 3. Enable model unloading
# 4. Use tiled VAE
# 5. Restart ComfyUI to clear VRAM
supervisorctl restart comfyui

Missing Models

# Check which models are linked
ls -lah /workspace/ComfyUI/models/diffusers/
ls -lah /workspace/ComfyUI/models/clip_vision/

# Re-run Ansible to download missing models
cd /workspace/ai
ansible-playbook playbook.yml --tags comfyui-models-all

# Re-link models
arty run models/link-comfyui

Workflow Load Errors

# Check ComfyUI logs
supervisorctl tail -f comfyui

# Common issues:
# - Missing custom nodes: Check custom_nodes/ directory
# - Node version mismatch: Update ComfyUI and custom nodes
# - Corrupted workflow: Validate JSON syntax

Development

Creating New Workflows

See WORKFLOW_STANDARDS.md for detailed guidelines on creating production-ready workflows.

Quick checklist:

• Use semantic versioning in filename
• Add API input/output nodes
• Include preview and save nodes
• Add error handling and validation
• Optimize for 24GB VRAM
• Document all parameters
• Test with orchestrator API

Testing Workflows

# Manual test via ComfyUI UI
# 1. Load workflow in ComfyUI
# 2. Set test parameters
# 3. Queue prompt
# 4. Verify output quality

# API test via orchestrator
curl -X POST http://localhost:9000/api/comfyui/generate \
  -H "Content-Type: application/json" \
  -d @test-payload.json

# Batch test multiple workflows
cd /workspace/ai/models/comfyui/workflows
for workflow in text-to-image/*.json; do
  echo "Testing $workflow..."
  # Add test logic here
done

Contributing

When adding new workflows:

1. Follow naming convention: {category}-{model}-{type}-production-v{version}.json
2. Place in appropriate category directory
3. Update this README with workflow details
4. Add to comfyui_models.yaml if new models are required
5. Test with both UI and API
6. Document any special requirements or setup

Resources

• ComfyUI Documentation: https://github.com/comfyanonymous/ComfyUI
• Custom Nodes Manager: Install via ComfyUI-Manager in UI
• Model Registry: /workspace/ai/model-orchestrator/models.yaml
• Ansible Playbook: /workspace/ai/playbook.yml
• Orchestrator API: http://localhost:9000/docs

License

MIT License - Part of RunPod AI Model Orchestrator

Support

For issues or questions:

1. Check ComfyUI logs: supervisorctl tail -f comfyui
2. Check orchestrator logs: supervisorctl tail -f orchestrator
3. Review /workspace/ai/CLAUDE.md for troubleshooting
4. Check GPU status: nvidia-smi