refactor: consolidate model management into Ansible playbook
Remove flux/musicgen standalone implementations in favor of ComfyUI: - Delete models/flux/ and models/musicgen/ directories - Remove redundant scripts (install.sh, download-models.sh, prepare-template.sh) - Update README.md to reference Ansible playbook commands - Update playbook.yml to remove flux/musicgen service definitions - Add COMFYUI_MODELS.md with comprehensive model installation guide - Update stop-all.sh to only manage orchestrator and vLLM services All model downloads and dependency management now handled via Ansible playbook tags (base, python, vllm, comfyui, comfyui-essential). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
320
COMFYUI_MODELS.md
Normal file
320
COMFYUI_MODELS.md
Normal file
@@ -0,0 +1,320 @@
|
||||
# ComfyUI Models & Nodes - Usage Guide
|
||||
|
||||
This document explains how to use the extended playbook.yml for installing ComfyUI models and custom nodes.
|
||||
|
||||
## What's Been Added
|
||||
|
||||
### Model Categories
|
||||
|
||||
**Image Generation Models:**
|
||||
- FLUX.1 Schnell (23GB, essential) - Fast 4-step inference
|
||||
- FLUX.1 Dev (23GB, optional) - Balanced quality/speed
|
||||
- SDXL Base 1.0 (7GB, essential) - Industry standard
|
||||
- SDXL Refiner 1.0 (6GB, optional) - Enhances base output
|
||||
- SD 3.5 Large (18GB, optional) - Latest Stability AI
|
||||
|
||||
**Video Generation Models:**
|
||||
- CogVideoX-5B (20GB, essential) - Professional text-to-video
|
||||
- SVD (8GB, essential) - 14 frame image-to-video
|
||||
- SVD-XT (8GB, optional) - 25 frame image-to-video
|
||||
|
||||
**Audio Generation Models:**
|
||||
- MusicGen Small (3GB, optional) - Fast generation
|
||||
- MusicGen Medium (11GB, essential) - Balanced quality
|
||||
- MusicGen Large (22GB, optional) - Highest quality
|
||||
|
||||
**Support Models:**
|
||||
- CLIP H (2GB, essential) - For SD 1.5 IP-Adapter
|
||||
- CLIP G (7GB, essential) - For SDXL IP-Adapter
|
||||
- SigLIP (2GB, essential) - For FLUX models
|
||||
|
||||
### Custom Nodes
|
||||
|
||||
**Essential Nodes (installed by default):**
|
||||
- ComfyUI-Manager - Install/manage custom nodes and models
|
||||
- ComfyUI-VideoHelperSuite - Video operations
|
||||
- ComfyUI-AnimateDiff-Evolved - Video generation
|
||||
- ComfyUI_IPAdapter_plus - Style transfer
|
||||
- ComfyUI-Impact-Pack - Auto face enhancement
|
||||
- comfyui-sound-lab - Audio generation
|
||||
|
||||
**Optional Nodes:**
|
||||
- ComfyUI-CogVideoXWrapper - CogVideoX integration
|
||||
- Comfyui-Inspire-Pack - Additional tools
|
||||
- ComfyUI-Advanced-ControlNet - Advanced control
|
||||
- ComfyUI-3D-Pack - 3D asset generation
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Quick Setup (Essential Only)
|
||||
|
||||
Install ComfyUI with only essential models and nodes:
|
||||
|
||||
```bash
|
||||
ansible-playbook playbook.yml --tags comfyui-essential
|
||||
```
|
||||
|
||||
This downloads:
|
||||
- Image: FLUX Schnell (23GB) + SDXL Base (7GB)
|
||||
- Video: CogVideoX-5B (20GB) + SVD (8GB)
|
||||
- Audio: MusicGen Medium (11GB)
|
||||
- Support: All 3 CLIP models (11GB)
|
||||
- **Total: ~80GB**
|
||||
|
||||
### Selective Installation
|
||||
|
||||
#### Install ComfyUI Base Only
|
||||
|
||||
```bash
|
||||
ansible-playbook playbook.yml --tags comfyui
|
||||
```
|
||||
|
||||
#### Install All Image Models
|
||||
|
||||
```bash
|
||||
ansible-playbook playbook.yml --tags comfyui-models-image
|
||||
```
|
||||
|
||||
#### Install All Video Models
|
||||
|
||||
```bash
|
||||
ansible-playbook playbook.yml --tags comfyui-models-video
|
||||
```
|
||||
|
||||
#### Install All Audio Models
|
||||
|
||||
```bash
|
||||
ansible-playbook playbook.yml --tags comfyui-models-audio
|
||||
```
|
||||
|
||||
#### Install Support Models (CLIP, IP-Adapter)
|
||||
|
||||
```bash
|
||||
ansible-playbook playbook.yml --tags comfyui-models-support
|
||||
```
|
||||
|
||||
#### Install Custom Nodes
|
||||
|
||||
```bash
|
||||
ansible-playbook playbook.yml --tags comfyui-nodes
|
||||
```
|
||||
|
||||
### Combined Installation
|
||||
|
||||
#### Full ComfyUI Setup
|
||||
|
||||
```bash
|
||||
ansible-playbook playbook.yml --tags comfyui,comfyui-models-all,comfyui-nodes
|
||||
```
|
||||
|
||||
#### ComfyUI + Image + Video Only
|
||||
|
||||
```bash
|
||||
ansible-playbook playbook.yml --tags comfyui,comfyui-models-image,comfyui-models-video,comfyui-nodes
|
||||
```
|
||||
|
||||
## Adding New Models
|
||||
|
||||
### Add Image Model
|
||||
|
||||
Edit `playbook.yml` and append to `comfyui_image_models`:
|
||||
|
||||
```yaml
|
||||
- name: "author/model-name"
|
||||
type: "checkpoint"
|
||||
category: "image"
|
||||
size_gb: 15
|
||||
vram_gb: 18
|
||||
format: "fp16"
|
||||
description: "Model description"
|
||||
essential: false # Set to true for default installs
|
||||
```
|
||||
|
||||
### Add Custom Node
|
||||
|
||||
Append to `comfyui_custom_nodes`:
|
||||
|
||||
```yaml
|
||||
- name: "NodeName"
|
||||
repo: "https://github.com/author/repo.git"
|
||||
category: "image" # or video, audio, control, etc.
|
||||
description: "What it does"
|
||||
essential: false
|
||||
```
|
||||
|
||||
## VRAM Requirements Summary
|
||||
|
||||
For your **24GB GPU**, you can run:
|
||||
|
||||
### Simultaneously Loadable Combinations
|
||||
|
||||
**Option 1: Image Generation Focus**
|
||||
- FLUX Schnell FP16 (23GB) → Full model, leaves 1GB for operations
|
||||
- FLUX Schnell FP8 (12GB) + SDXL (12GB) → Both loaded
|
||||
- SDXL (12GB) + Refiner (12GB) → Both loaded with room
|
||||
|
||||
**Option 2: Video Generation**
|
||||
- CogVideoX-5B with optimizations (12GB) + SDXL (12GB)
|
||||
- SVD-XT (20GB) → Good frame count
|
||||
|
||||
**Option 3: Multi-Modal**
|
||||
- SDXL (12GB) + MusicGen Medium (8GB)
|
||||
- CogVideoX-5B (12GB) + MusicGen Small (4GB)
|
||||
|
||||
## Storage Requirements
|
||||
|
||||
### Essential Models Only
|
||||
- Image: 30GB
|
||||
- Video: 28GB
|
||||
- Audio: 11GB
|
||||
- Support: 11GB
|
||||
- **Total: ~80GB**
|
||||
|
||||
### All Models (including optional)
|
||||
- Image: 54GB
|
||||
- Video: 36GB
|
||||
- Audio: 36GB
|
||||
- Support: 11GB
|
||||
- **Total: ~137GB**
|
||||
|
||||
## Directory Structure Created
|
||||
|
||||
```
|
||||
/workspace/ComfyUI/
|
||||
├── models/
|
||||
│ ├── checkpoints/ # FLUX, SDXL, SD3 models
|
||||
│ ├── unet/ # Diffusion U-Nets
|
||||
│ ├── vae/ # VAE models
|
||||
│ ├── loras/ # LoRA adapters
|
||||
│ ├── clip/ # CLIP text encoders
|
||||
│ ├── clip_vision/ # CLIP vision models
|
||||
│ ├── controlnet/ # ControlNet models
|
||||
│ ├── ipadapter/ # IP-Adapter models
|
||||
│ ├── embeddings/ # Text embeddings
|
||||
│ ├── upscale_models/ # Upscalers
|
||||
│ ├── video_models/ # Video generation models
|
||||
│ ├── animatediff_models/ # AnimateDiff models
|
||||
│ ├── audio_models/ # Audio generation models
|
||||
│ └── configs/ # Model configs
|
||||
└── custom_nodes/ # Extension nodes
|
||||
```
|
||||
|
||||
## Extending the Playbook
|
||||
|
||||
### Add a New Model Category
|
||||
|
||||
1. Add variable list in `vars` section:
|
||||
|
||||
```yaml
|
||||
comfyui_new_category_models:
|
||||
- name: "model-repo-id"
|
||||
type: "new_type"
|
||||
category: "new_category"
|
||||
size_gb: 10
|
||||
description: "Description"
|
||||
essential: true
|
||||
```
|
||||
|
||||
2. Create download task section:
|
||||
|
||||
```yaml
|
||||
- name: Download New Category Models
|
||||
tags: [comfyui-models-newcat, comfyui-models-all]
|
||||
block:
|
||||
# Copy pattern from existing model download tasks
|
||||
```
|
||||
|
||||
3. Update tags documentation at top of file
|
||||
|
||||
## Model Download Notes
|
||||
|
||||
- **Async Downloads**: Large models download asynchronously with 1-hour timeouts
|
||||
- **Smart Caching**: Models are only downloaded if not already cached
|
||||
- **HuggingFace Token**: Set `HF_TOKEN` environment variable for gated models
|
||||
- **Error Handling**: Download failures are ignored to allow partial installations
|
||||
|
||||
## Performance Tips
|
||||
|
||||
### For 24GB GPU:
|
||||
|
||||
1. **Use FP8 Quantized FLUX** when you need multiple models loaded
|
||||
2. **Use Full FP16 FLUX** when you want maximum quality
|
||||
3. **Enable CogVideoX Optimizations** to fit in 12GB (edit model config)
|
||||
4. **Use GGUF variants** for FLUX to save VRAM (manually download)
|
||||
|
||||
### Recommended First Install:
|
||||
|
||||
```bash
|
||||
# Essential setup (~80GB, ~1 hour)
|
||||
ansible-playbook playbook.yml --tags comfyui-essential
|
||||
```
|
||||
|
||||
Then add specific models as needed:
|
||||
|
||||
```bash
|
||||
# Add video models later
|
||||
ansible-playbook playbook.yml --tags comfyui-models-video
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Downloads Failing
|
||||
|
||||
Check HuggingFace token:
|
||||
```bash
|
||||
export HF_TOKEN="your_token_here"
|
||||
ansible-playbook playbook.yml --tags comfyui-models-image
|
||||
```
|
||||
|
||||
### Out of Space
|
||||
|
||||
Check available storage:
|
||||
```bash
|
||||
df -h /workspace
|
||||
du -sh /workspace/huggingface_cache
|
||||
```
|
||||
|
||||
Remove optional models:
|
||||
```bash
|
||||
# Keep only essential models in playbook vars
|
||||
```
|
||||
|
||||
### Custom Node Dependencies
|
||||
|
||||
Some nodes require manual intervention:
|
||||
```bash
|
||||
cd /workspace/ComfyUI/custom_nodes/NodeName
|
||||
pip3 install -r requirements.txt
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
After installation:
|
||||
|
||||
1. **Start ComfyUI:**
|
||||
```bash
|
||||
cd /workspace/ComfyUI
|
||||
python3 main.py --listen 0.0.0.0 --port 8188
|
||||
```
|
||||
|
||||
2. **Access via Browser:**
|
||||
```
|
||||
http://your-runpod-ip:8188
|
||||
```
|
||||
|
||||
3. **Install Additional Nodes via Manager:**
|
||||
- ComfyUI Manager provides GUI for installing more nodes
|
||||
- Browse and install from the manager interface
|
||||
|
||||
4. **Download Model Variants:**
|
||||
- Use ComfyUI Manager to download LoRAs
|
||||
- Download ControlNet models as needed
|
||||
- Get additional checkpoints from Civitai
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
- Check Ansible logs for download status
|
||||
- Verify HuggingFace token is valid
|
||||
- Ensure sufficient storage space
|
||||
- Check VRAM requirements for your GPU
|
||||
Reference in New Issue
Block a user