Added new 'verify' command to artifact_huggingface_download.sh that performs comprehensive health checks on all downloaded models and their symlinks.
Features:
- Download status verification (existence, size, location, timestamps)
- Link status verification (valid, broken, or missing symlinks)
- Size mismatch detection (warns if actual differs >10% from expected)
- Per-model detailed logging with beautiful formatting
- Category-level and global statistics summaries
- Actionable fix suggestions for detected issues
- Disk space usage analysis
New Functions:
- get_model_disk_usage() - Calculate actual model file sizes
- format_bytes() - Human-readable size formatting
- verify_model_download() - Check model download status
- verify_model_links() - Verify symlink integrity
- verify_category() - Process category with verification
- display_verification_summary() - Show global results
Usage:
artifact_huggingface_download.sh verify -c models.yaml
Output includes:
✓ Downloaded/Missing model counts
✓ Properly linked/Broken link statistics
✓ File sizes and locations
✓ Last modified timestamps
⚠️ Size mismatch warnings
📊 Disk space usage per category
💡 Fix suggestions with exact commands
🤖 Generated with Claude Code (https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
The build_reference_tree() function was prepending config_dir to all
'into' paths without expanding environment variables or checking if the
result was absolute. This caused paths like /workspace/ai//workspace/ComfyUI
instead of /workspace/ComfyUI.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Fixed issue where absolute paths from environment variables (like $COMFYUI_ROOT=/workspace/ComfyUI) were being treated as relative paths and prepended with the config directory.
Now checks if the expanded path is absolute (starts with /) and uses it as-is, otherwise treats it as relative to the config file directory.
Before: /home/valknar/Projects/runpod//workspace/ComfyUI
After: /workspace/ComfyUI
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Adds support for environment variables like $COMFYUI_ROOT in the 'into' field of references when running 'arty deps'. Features:
- New expand_env_vars() function with nested variable expansion support
- Validates that no undefined variables remain after expansion
- Fails with clear error messages showing undefined variables
- Lists available environment variables from arty.yml in error output
- Expands variables from envs section before cloning repositories
Example usage in arty.yml:
```yaml
envs:
default:
COMFYUI_ROOT: /workspace/ComfyUI
references:
- url: https://github.com/ltdrdata/ComfyUI-Manager.git
into: $COMFYUI_ROOT/custom_nodes/ComfyUI-Manager
```
Expands to: /workspace/ComfyUI/custom_nodes/ComfyUI-Manager
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add artifact_civitai_download.sh with beautiful purple/magenta CLI
- Rename artifact_comfyui_download.sh to artifact_hugginface_download.sh
- Remove comfyui_models.example.yaml (moved to runpod repo)
Features:
- Dedicated downloader for CivitAI models
- Beautiful CLI with progress bars and retry logic
- Same architecture as HuggingFace downloader
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Automatically detects RunPod environment (checks for /workspace directory)
- Uses RunPod paths by default: /workspace/huggingface_cache and /workspace/ComfyUI/models
- Falls back to local paths (~/.cache/huggingface and ~/ComfyUI/models) on non-RunPod systems
- Updated help text to show both RunPod and local defaults
- Updated example YAML with RunPod paths
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Removed automatic checkpoint linking logic that was incorrectly
duplicating diffusers models to checkpoints directory. Models should
only be linked to their explicitly configured directory type.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Added parse_file_mappings() function to extract file mappings from YAML
- Modified link_model() to accept and use explicit source→dest mappings
- Falls back to automatic prefixing if no mappings specified
- Updated process_category() to parse and pass file mappings
- This allows workflows to reference models by their expected names
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Extract model name from repo_id (e.g., 'musicgen-medium' from 'facebook/musicgen-medium')
and prefix all linked files with it for better organization and clarity.
Examples:
- pytorch_model.bin -> musicgen-medium-pytorch_model.bin
- model.safetensors -> musicgen-small-model.safetensors
HuggingFace cache stores files as symlinks to blobs, so stat without -L
was returning the symlink size (76 bytes) instead of actual file size (6.5GB).
This caused the >500MB check to fail and skip linking checkpoint files.
- Apply same sys.argv pattern to find_model_files function
- Ensures all Python heredocs avoid bash interpolation issues
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Initialize HF_TOKEN="${HF_TOKEN:-}" before conditional checks
- Fixes "unbound variable" error with set -euo pipefail
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Try PROJECT_ROOT/ai/.env first
- Fallback to PROJECT_ROOT/.env
- Fallback to /workspace/ai/.env
- Fixes HF_TOKEN loading on RunPod
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Check for models in cache_dir/models--xxx--yyy/ first
- Fallback to cache_dir/hub/models--xxx--yyy/ for compatibility
- Fixes issue where script couldn't find downloaded models
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Adds 'arty notes' command to display project notes from arty.yml with beautiful markdown rendering. Features include:
- Reads 'notes' field from arty.yml
- Supports full markdown syntax (headers, bold, italic, code blocks, links, lists)
- Smart rendering fallbacks: glow → mdcat → python3 → plain text
- Beautiful terminal output with colors and formatting
- ANSI escape codes for syntax highlighting
- Horizontal separators for code blocks
Also renamed arty.sh to artifact_git_download.sh for better clarity.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Adds subcommand structure (download, link, both) to the ComfyUI downloader script, allowing users to download models from HuggingFace and/or create symlinks to ComfyUI model directories. Features include:
- New subcommands: download (download only), link (symlink only), both (default)
- ComfyUI directory configuration (--comfyui-dir, default: ~/ComfyUI/models)
- Smart symlink creation to appropriate ComfyUI subdirectories (checkpoints, vae, loras, controlnet, upscale_models, etc.)
- YAML configuration extended with 'type' and 'filename' fields for precise model organization
- Automatic ComfyUI subdirectory creation
- Graceful handling of existing symlinks and files
- HF_TOKEN validation only when needed (download/both commands)
- Example configuration file (comfyui_models.example.yaml) demonstrating proper setup
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Adds artifact_comfyui_download.sh for downloading AI models from HuggingFace with configuration-driven approach, beautiful CLI output, and flexible options. Features include optional YAML configuration, customizable cache directory (defaults to ~/.cache/huggingface), HF_TOKEN support, and robust error handling.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fix race condition in load_env_vars() caused by process substitution
- Replace unreliable `< <(yq eval ...)` with stable variable capture
- Ensure environment variables are consistently loaded on every execution
- Add yq binary (mikefarah/yq v4.48.1) for reliable YAML parsing
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit introduces a comprehensive collection of utility scripts for shell
automation, color manipulation, and documentation generation:
Core Scripts:
- artifact_github_download.sh: Download GitHub Action artifacts via CLI
- css_color_filter.sh: Generate CSS filter values using SPSA algorithm
- css_color_palette.sh: Generate comprehensive color palettes (monochromatic, triadic, etc.)
- css_json_convert.sh: Convert CSS variables to JSON/YAML formats
- doc_bash_generate.sh: Auto-generate README.md with animated GIF demos
- doc_rust_generate.sh: Generate Rust project documentation
- jinja_template_render.sh: Render Jinja2 templates from CLI
- mime_mp4_gif.sh: Convert MP4 videos to GIF format
Documentation Features:
- Comprehensive README.md with table of contents
- 8 animated GIF demos showing real command examples
- Sandboxed demo execution in temporary directories
- 15-second timeout protection for intensive computations
- Automatic example extraction from --help output
Technical Implementation:
- Pure bash color utilities using only bc for arithmetic
- tput-based color codes for portability
- IFS-safe string parsing using parameter expansion
- Stdout/stderr isolation to prevent contamination
- Base64 encoding for multi-line text preservation
All scripts include detailed --help documentation with usage examples.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
