mcp-vitals — reliability grades (A–F) for MCP servers

mcp-vitals grades any MCP server A–F for reliability and agent-usability — the two things security scanners and hosted leaderboards don't measure: does the tool actually work, and can an agent figure out how to use it?

I pointed it at the official MCP reference servers and published the results:

It grades three layers, run locally on your own server, in CI:

L1 Static — schema quality: descriptions, typed & documented params, naming.

L2 Behavioral — auto-generated tests run against the live server (success rate, graceful errors, latency), read-only by default.

L3 Agent-usability — an LLM is handed the tool catalog and a real task, and has to pick the right tool and build a valid call. The layer nobody else measures.

The finding that matters: on the filesystem server, a model asked to "read this file" couldn't tell read_file from read_text_file and picked the tool that doesn't exist. A security scanner or a static grader never sees that — but it silently breaks agents in production.

Honest scope: the F for sequential-thinking is a conservative-grader signal, not a broken server — its single tool needs a payload the schema-only test generator can't synthesize, so the behavioral layer couldn't exercise it. I document that plainly in the report.

Open source (MIT): a CLI + a drop-in GitHub Action + a shields.io badge + a CI gate that fails the build when a server's grade regresses.

→ Live report: https://enached134-ctrl.github.io/mcp-vitals → Repo: https://github.com/enached134-ctrl/mcp-vitals