Any OCI registry
Docker Hub, GHCR, quay, or something self-hosted. The local store is a plain OCI Image Layout, so docker and podman can read it too.
Models are just OCI artifacts.
llmman packages LLM models as CNCF ModelPack artifacts, stores them in any OCI registry you already run, and serves them over Ollama-, OpenAI- and Anthropic-compatible APIs.
Built on the CNCF ModelPack specification — an open format, not llmman's own.
What you get
Three APIs, one server
llmman serve answers Ollama, OpenAI (including /v1/responses) and Anthropic routes on the same port, so existing clients need no patching.
Bring your own engine
GGUF models are served by llama-server, safetensors by vllm, or by MLX on Apple Silicon. llmman fetches a matching build if you have none.
Launch integrations
llmman launch claude --model gemma4 starts the daemon, preloads the model, wires up the environment and execs the tool.
Registry-to-registry transfer
llmman transfer streams an image straight from HuggingFace to an OCI registry without ever landing the weights on your disk.
Hosted models too
--provider openrouter points the same integrations at a model llmman does not serve itself, with the provider list resolved from models.dev at runtime.
From the blog
- An aggregation of llmman
llmman serve daemons on several machines can now pool their hardware. Name the peers and a request to any node is served by whichever one has the model loaded, or the most room to load it.
- Verify the model, not the hub
llmman now checks cosign signatures before it will run a model, against a trust policy you write with keys you hold.
- Mirror Hugging Face models into any OCI registry
If you want somewhere other than Hugging Face to keep model weights, the boring answer is the registry infrastructure you already run.
- Introducing llmman
A command-line tool that treats LLM models as ordinary OCI artifacts, and serves them over the Ollama, OpenAI and Anthropic APIs your tools already speak.