Herramienta de evaluación comparativa MCP vLLM

Esta es una prueba de concepto sobre cómo utilizar MCP para evaluar de forma interactiva vLLM.

No somos nuevos en benchmarking, lea nuestro blog:

Esta es solo una exploración de posibilidades con MCP.

Uso

Clonar el repositorio
Agreguelo a sus servidores MCP:

{
    "mcpServers": {
        "mcp-vllm": {
            "command": "uv",
            "args": [
                "run",
                "/Path/TO/mcp-vllm-benchmarking-tool/server.py"
            ]
        }
    }
}

Entonces puedes indicar un ejemplo como este:

Do a vllm benchmark for this endpoint: http://10.0.101.39:8888 
benchmark the following model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B 
run the benchmark 3 times with each 32 num prompts, then compare the results, but ignore the first iteration as that is just a warmup.

Hacer:

Debido a algunas salidas aleatorias de vllm, es posible que se indique que encontró un JSON no válido. Aún no lo he investigado.

This server cannot be installed

security - not tested

license - not found

quality - not tested

How are these scores calculated?

remote-capable server

The server can be hosted and run remotely because it primarily relies on remote services or has no dependency on the local environment.

Una herramienta interactiva que permite a los usuarios evaluar los puntos finales de vLLM a través de MCP, lo que permite realizar pruebas de rendimiento de modelos LLM con parámetros personalizables.

Related MCP Servers

mcp-llm
sammcj
A
security
A
license
A
quality
An MCP server that provides LLMs access to other LLMs
Last updated 4 months ago
4
559
57
JavaScript
MIT License
Claude-LMStudio Bridge
infinitimeless
-
security
F
license
-
quality
An MCP server that allows Claude to interact with local LLMs running in LM Studio, providing access to list models, generate text, and use chat completions through local models.
Last updated 4 months ago
8
Python
MCP Prompt Tester
rt96-hub
-
security
A
license
-
quality
An MCP server that allows agents to test and compare LLM prompts across OpenAI and Anthropic models, supporting single tests, side-by-side comparisons, and multi-turn conversations.
Last updated 5 months ago
Python
MIT License
Just Prompt
disler
A
security
F
license
A
quality
A lightweight MCP server that provides a unified interface to various LLM providers including OpenAI, Anthropic, Google Gemini, Groq, DeepSeek, and Ollama.
Last updated a month ago
6
491
Python

View all related MCP servers

Appeared in Searches

A service for checking the weather forecast

MCP vLLM Benchmarking Tool