Use case · decision ranking
Production model serving
Expose model inference to applications through a production serving boundary.
6 reviewed matches, ranked by fit and deterministic project health.
#1
Editorialhuggingface/transformers: 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Editorial90% fitServing runtimes are candidates for exposing model inference to applications; validate model, hardware, batching, and API requirements.Compare this repository → ★ 164.4KPythonApache-2.0apiclillm-serving
#2
Editorialollama/ollama: Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
Editorial90% fitServing runtimes are candidates for exposing model inference to applications; validate model, hardware, batching, and API requirements.Compare this repository → ★ 179.4KGoMITapiclillm-serving
#3
Editorialsgl-project/sglang: SGLang is a high-performance serving framework for large language models and multimodal models.
Editorial90% fitServing runtimes are candidates for exposing model inference to applications; validate model, hardware, batching, and API requirements.Compare this repository → ★ 32.4KPythonApache-2.0apiclillm-serving
#4
Editorialkvcache-ai/ktransformers: A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations.
Editorial90% fitServing runtimes are candidates for exposing model inference to applications; validate model, hardware, batching, and API requirements.Compare this repository → ★ 19.3KPythonApache-2.0apiclillm-serving
#5
Editorialmicrosoft/BitNet: Official inference framework for 1-bit LLMs.
Editorial90% fitServing runtimes are candidates for exposing model inference to applications; validate model, hardware, batching, and API requirements.Compare this repository → ★ 40.1KC++MITapiclillm-serving
#6
Editorialopen-webui/pipelines: Pipelines: Versatile, UI-Agnostic OpenAI-Compatible Plugin Framework.
Editorial90% fitServing runtimes are candidates for exposing model inference to applications; validate model, hardware, batching, and API requirements.Compare this repository → ★ 2.4KPythonMITapillm-serving