Grading MCP Servers: A Deep Dive into Reliability and Usability

Grading MCP Servers: A Deep Dive into Reliability and UsabilityGrading MCP Servers: A Deep Dive into Reliability and Usability

The network for creativity

Join 1.25M professional creatives like you

Connect with clients, get discovered, and run your business 100% commission-free

Creatives on Contra have earned over $150M and we are just getting started

Back to feedPost

Daniel Enache

• Jul 4

I graded the official MCP servers. Two got an A, one got an F — and the most useful finding came from a layer nobody else measures.

mcp-vitals (open source) grades an MCP server's reliability A–F across three layers: static schema quality, behavioral tests run against the live server, and — the novel part — agent-usability: hand an LLM the tool list and a real task, and see if it picks the right tool and builds a valid call.

Results on the official reference servers: • memory — A (94) • everything — A (92) • filesystem — C (71) • sequential-thinking — F (57)*

The finding that matters came from the agent-usability layer. On the filesystem server, a model given a "read this file" task couldn't tell read_file from read_text_file and picked the tool that doesn't exist. A security scanner or a static grader never sees that — but it silently breaks agents in production.

*The F is honest: sequential-thinking's single tool needs a payload my schema-only generator can't synthesize, so the behavioral layer couldn't exercise it. A conservative grader that won't vouch for what it can't verify — I document that plainly in the report.

Security scanners tell you a server is safe. mcp-vitals tells you if it's any good.

Live report → https://enached134-ctrl.github.io/mcp-vitals Repo (MIT, drop-in GitHub Action) → https://github.com/enached134-ctrl/mcp-vitals

enached134-ctrl.github.io

The State of MCP Reliability — graded A–F

I graded the official MCP reference servers for reliability and agent-usability. Two got an A, one got an F. Open-source CLI + GitHub Action.

Stephen Kisong'e

• 1d

A hands-on overview of CAI, an AI-assisted cybersecurity agent framework configured and explored in a Linux terminal environment. The demonstration focuses on how AI agents can be organized and used to support different areas of cybersecurity testing, analysis, and research.

The walkthrough covers command-line navigation, available help options, agent selection, model configuration, and the use of specialized security agents. It includes a closer look at DFIR-focused agents for digital forensics and incident response, along with other agent categories designed for bug bounty research, red team activities, network security, reverse engineering, Wi-Fi security, and reporting.

The setup also highlights parallel agent configuration, showing how multiple AI-driven security agents can be prepared for structured analysis and task separation. This makes the environment useful for handling different cybersecurity activities in a more organized and scalable way.

Overall, the work reflects practical experience with AI-powered security tooling, terminal-based security environments, agent configuration, cybersecurity automation, and ethical AI-assisted security research.

Cybersecurity Security Engineering Bash Python

Sly

pro

• 2d

FixFlo — The Smart Client Feedback Layer for Websites

Completed work

Full Stack Development Base44

Omolaja Deborah

• 2d

Clients notice when you're disorganized.

A missed follow-up. A calendar clash. An email that sits too long.

None of it feels huge in the moment, but it adds up. And clients remember.

That's the quiet value of a good EA: not just getting things done, but making sure nothing falls through the cracks in the first place. The inbox that's actually managed. The calendar that never double-books. The follow-up that happens before anyone has to ask.

It's not flashy. But it's the difference between a client who trusts you completely and one who's quietly looking elsewhere.