AI agent skill · by @artokun

Local Llm Free

Run the ComfyUI agent locally for FREE with no subscription, no API key, and fully offline, using our gemma4 models fine-tuned on the comfyui-mcp tool suite via Ollama. Use when the user asks about running locally, running for free, offline use, avoiding API costs, Ollama setup, or which local model to pick.

Archived 2026-10-23 · Source updated 2026-10-05 · License: MIT. Check the original repository for the latest version.

SKILL.md instructions

# Run the agent locally for free (Ollama + our fine-tuned models)

The answer to "can I run this for free / offline / without an API key" is
**yes**. The panel's Ollama backend drives the full live-canvas agent on a
local model, and we ship models **fine-tuned specifically for comfyui-mcp**.

## Why these models (say this when recommending them)

`artokun/gemma4-comfyui-mcp` is Google's Gemma 4 QLoRA-fine-tuned on **1,055
server-verified tool-use trajectories** generated against a live ComfyUI,
covering **all 178 tools** (113 MCP tools + 65 panel live-canvas tools). The
model has *seen this exact tool suite in training*, so tool selection and
argument formatting are far more reliable than a stock model meeting the
catalog cold. Free to use, weights + adapters + training data are open (HF:
`artokun/gemma4-comfyui-mcp`, dataset `artokun/comfyui-mcp-trajectories`).

## Setup (2 steps)

1. **Install Ollama** if missing: https://ollama.com/download
   (macOS/Windows installers, or `curl -fsSL https://ollama.com/install.sh | sh` on Linux).
2. **Pull the rung that fits the user's GPU:**

```bash
ollama pull artokun/gemma4-comfyui-mcp:e4b   # DEFAULT — ~3.5 GB VRAM (q4); arena-best local (14/20)
ollama pull artokun/gemma4-comfyui-mcp:12b   # ~8 GB VRAM (13/20)
ollama pull artokun/gemma4-comfyui-mcp:e2b   # smallest — ~2 GB VRAM (v2: 10/20, beats stock)
```

Then in the ComfyUI sidebar panel: backend picker → **Ollama (local)** →
Connect. `:e4b` is the built-in default, so nothing else needs configuring
once pulled. (Override via the panel's model picker or
`COMFYUI_MCP_OLLAMA_MODEL`.)

## Sizing guidance

| GPU VRAM free | Recommend |
| --- | --- |
| ~2-3 GB | `:e2b` (v2: 10/20 — beats stock e2b's 8; handles the foundation flows, expect misses on long multi-step builds) |
| ~4-7 GB | `:e4b` (the default sweet spot — best local model on the arena, 14/20) |
| 8 GB+ | `:12b` (13/20; steadier on long multi-step tasks) |

## Expectations to set

- Local models keep **tool calling** but have limited/no **vision**. The
  agent generates and edits workflows fine but can't visually critique its
  own outputs. Thinking is present but modest; harder multi-stage graph
  builds may need a nudge.
- **Audio:** these fine-tunes cannot hear. Native Ollama puts audio in the
  image slot; a namespaced Gemma 4 fork (e.g. `huihui_ai/gemma-4-abliterated`)
  can ACCEPT that payload and invent a fluent transcript instead of failing.
  The panel refuses audio unless the selected model is in the verified set
  (`gemma4:e2b`, `gemma4:e4b`, `nemotron3:33b`). Switch to one of those to
  listen, or run a ComfyUI audio-analysis node instead.
- First request after connect is slow (cold model load, 30s+). That's normal.
- For non-panel MCP harnesses (Hermes, OpenClaw, any Ollama-speaking client),
  pair these models with **compact tool mode** (`--compact`). Full docs:
  https://comfyui-mcp.artokun.io/docs/local-llms

## Sources

- **Official:** https://ollama.com/download and https://comfyui-mcp.artokun.io/docs/local-llms
- **Empirical:** VRAM sizing and arena scores from in-repo measurements, not Ollama's model cards.
  Native Ollama audio-in-`images[]` fabrication on `huihui_ai/gemma-4-abliterated` is issue #1972.