#serve

The web UI has a Models page now: popular models from Docker Hub and Hugging Face, a search of both, and a card per model with every tag or quantization you can pull, its size, and whether it fits your machine.

POST /v1/systemone turns a state and a few typed questions into a probability per answer, read from a local model's own token probabilities on ggml. Nothing is generated, so there is no text to parse.

llmman run qwen-image-2.1 turns a prompt into a 1024×1024 picture, straight from the model's Diffusers safetensors, on ggml. Done in collaboration with Unsloth.

llama.cpp's only prebuilt GPU binary for Windows on ARM is an OpenCL one, not Vulkan. llmman now probes for it, so machines having it stop silently falling back to CPU.

Antigravity's CLI talks to models over the Gemini API. llmman now translates that protocol, so AGY can run against a local model or a hosted one.

DeepSeek Harness treats the model as a plugin. llmman runs any model on your own hardware, in one command.

llmman serve now keeps a history of the prompts it is sent, and llmman log shows it the way git log shows commits.

A diffusion model is a model like any other: llmman run ltx-2.3 turns a prompt into a picture, an mp4 with a soundtrack, or a wav, on whatever GPU you have.

One model name can now carry a local model and a hosted one. llmman serve picks a side per request, and everything that fits stays on your machine.

llmman serve daemons on several machines can now pool their hardware. Name the peers and a request to any node is served by whichever one has the model loaded, or the most room to load it.