vllm-mlx MCP Server
waybarrios/vllm-mlx
[](https://agentrank-ai.com/tool/waybarrios--vllm-mlx/?utm_source=badge&utm_medium=readme&utm_campaign=agentrank_badge) High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
claude mcp add agentrank -- npx -y agentrank-mcp-server Overview
waybarrios/vllm-mlx is a Python tool licensed under Apache-2.0. High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support. Topics: apple-silicon, llm, macos, mlx, multimodal-ai, speech-to-text, text-to-speech, vision-language-model, vllm, anthropic, claude-code, anthropic-api, continuous-batching, inference-server, local-llm, mcp, openai, openai-api, openai-compatible, tool-calling.
Ranked #551 out of 24846 indexed tools.
In the top 3% of all indexed tools.
Has 1,619 GitHub stars.
Has 61 contributors.
Actively maintained with commits in the last week.
Ecosystem
Score Breakdown
1,619 stars → well-known project
Last commit 2d ago → actively maintained
159/262 issues closed → decent issue management
61 contributors → active community
No dependents → no downstream usage
2,744 weekly installs → gaining traction
Weights: Freshness 20% · Issue Health 20% · Dependents 22% · Stars 10% · Contributors 8% · How we score →
Badge all embed codes →
[](https://agentrank-ai.com/tool/waybarrios--vllm-mlx/?utm_source=badge&utm_medium=readme&utm_campaign=agentrank_badge) <a href="https://agentrank-ai.com/tool/waybarrios--vllm-mlx/?utm_source=badge&utm_medium=readme&utm_campaign=agentrank_badge"><img src="https://agentrank-ai.com/api/v1/badge/waybarrios--vllm-mlx" alt="AgentRank"></a> Matched Queries
From the README
# vllm-mlx
**Continuous batching + OpenAI + Anthropic APIs in one server. Native Apple Silicon inference.**
**Read this in other languages:** [English](README.md) · [Español](README.es.md) · [Français](README.fr.md) · [中文](README.zh.md)
---
## What is vllm-mlx?
A vLLM-style inference server for Apple Silicon Macs. Unlike `Ollama` or `mlx-lm` used directly, it ships **continuous batching, paged KV cache, prefix caching, and SSD-tiered cache**, and exposes **both OpenAI `/v1/*` and Anthropic `/v1/messages`** from a single process. Run LLMs, vision models, audio, and embeddings on Metal with unified memory, no conversion step.
## Quick start (30 seconds)
```bash
pip install vllm-mlx
vllm-mlx serve mlx-community/Llama-3.2-3B-Instruct-4bit --port 8000 --continuous-batching
```
**OpenAI SDK:**
```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
r = client.chat.completions.create(model="default", messages=[{"role": "user", "c Read full README on GitHub → Get the weekly AgentRank digest
Top movers, new tools, ecosystem insights — straight to your inbox.