Missing outputs belong in an AI writing-QA report.
If a scorer returns ratings for 79 of 80 cases, agreement on the returned cases cannot explain what happened to the missing response.
Track coverage and validity beside agreement. Keep an explicit distinction between a missing output, an invalid output and a response that is intentionally unrateable.
When comparing runs, match case IDs before comparing scores. Otherwise, a dropped difficult case can make a headline result look better.
AI Score Benchmark Pro provides spreadsheets and an offline evaluator for this review workflow. The free preview explains the approach.
AI Score Benchmark Pro uses AI-authored provisional synthetic references. Independent human rating has not been completed.
Free preview: https://contra.com/products/UDTrn11c-ai-score-benchmark-pro-free-writing-qa-preview
100+ people already use my free skill file to stop their AI from making the same 25 design mistakes.
Drop it into your project and your agent gets better at typography, spacing, color, and dark mode. Link in the comments.
My Contra Discovery Score has reached 114 points, placing me in the top 10% of creatives on the platform. My profile quality and profile completion are both at 100%, with a 5/5 average rating and availability turned on.
As a software engineer specializing in React, .NET, API development, and SAP Business One, I'm continuing to strengthen my Contra presence, showcase my portfolio, and connect with potential clients for new freelance opportunities.