The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Back to feedPost
A green test suite doesn't always mean your code is correct.
One thing I've been thinking about while building AI-powered developer tools is how easily a test can pass without actually verifying the behavior that matters.
For example, a test might call a method and finish with assertTrue(true). Everything looks green, but the test hasn't proved that the expected behavior works.
AI coding assistants make writing code and tests faster. That makes it even more important to ask a few questions before merging:
Does the test verify the behavior the PR changes?
Are mocks hiding the behavior we're supposed to test?
Would the test catch a broken implementation?
Are we reviewing the wider impact, not just the changed lines?
These questions have influenced how I'm building CodeGuard AI, a self-hosted GitHub PR reviewer that analyzes code changes alongside repository context and supports custom team rules.
I'm curious about other developers' experiences:
What's the most misleading passing test you've encountered in a real project?
I'd love to hear how you caught it.
Exactly , that's the dangerous part. If AI introduces a bug and then writes a test based on the same mistaken assumption, both the implementation and the test can agree while the actual requirement is still broken.
That's why I think reviewing what a test truly verifies matters...
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Related posts
The fix for an agent that keeps getting your design system wrong isn't a better prompt.
It's putting the rules where the agent reads them.
In the repo. In skills. In checks it can run itself.
Writing a course about exactly this.
What sells it for me is the refusal telling the agent exactly where the token should go. A bare "no hex" error usually just sends it looping. Does guard.mjs run only as an agent hook, or in pre-commit too so it catches humans as well?
Mary Holt© is currently ranked #1 among Agent Templates on the Framer Marketplace! 🏆
For those who haven't checked out the new Agent Templates yet, they come with dedicated AI skills, a Design System page, and video tutorials to help users customize and extend the template using the Framer Agent.
Mary Holt© currently includes 4 custom skills, making it easier to build new sections, customize existing components, and keep everything consistent with the original design system.
I'm not sure how long it'll stay at #1 as more creators start converting their templates, but it's exciting to see it there!
Testing two color directions for Pilotly, an AI agent platform for product teams.
Which direction would you pick for an AI automation product?
3 voted
100%
0 voted
0%
3 votes
Closed
I'd pick Violet Desk. Blue is what every SaaS uses, violet stands out and still says AI, and the orange sticky notes pop more against it. Does the hero chip row under the headline work on mobile?