Investigate score disagreements across a larger, defined sample of English writing. For assessment and EdTech teams that need response-level evidence to prioritise scorer changes.
What you receive
Blind human scoring of up to 60 existing written responses using one agreed rubric.
An item-level comparison with your model output, with descriptive agreement and score-difference analysis.
An error taxonomy, annotated disagreement examples and a prioritised improvement plan.
A 30-minute team readout and an editable comparison file.
Scope and delivery
One assessment format, one target learner population and one model/version. Up to five scoring criteria, 600 words per response and 30,000 words overall. One human reviewer is included. $1,500 fixed fee, delivered in two weeks after complete inputs.
What I need from you
De-identified responses and prompts, your scoring rubric, separately held AI scores, task metadata and the intended use of results. Agree permissions, confidentiality and transfer arrangements before sharing data.
Revisions
One consolidated clarification and factual-correction round within seven days. Additional model versions, a second independent rater and post-change retesting require a separate quote.
What this review covers
Disagreements are sample-specific. A single comparison does not establish longitudinal drift or certification. Psychometric validation, representative fairness testing, software changes and claims of guaranteed AI accuracy are excluded.
Start here
Send your rubric, sample size, task format and the decision your team needs to make. I'll confirm the comparison scope.
Investigate score disagreements across a larger, defined sample of English writing. For assessment and EdTech teams that need response-level evidence to prioritise scorer changes.
What you receive
Blind human scoring of up to 60 existing written responses using one agreed rubric.
An item-level comparison with your model output, with descriptive agreement and score-difference analysis.
An error taxonomy, annotated disagreement examples and a prioritised improvement plan.
A 30-minute team readout and an editable comparison file.
Scope and delivery
One assessment format, one target learner population and one model/version. Up to five scoring criteria, 600 words per response and 30,000 words overall. One human reviewer is included. $1,500 fixed fee, delivered in two weeks after complete inputs.
What I need from you
De-identified responses and prompts, your scoring rubric, separately held AI scores, task metadata and the intended use of results. Agree permissions, confidentiality and transfer arrangements before sharing data.
Revisions
One consolidated clarification and factual-correction round within seven days. Additional model versions, a second independent rater and post-change retesting require a separate quote.
What this review covers
Disagreements are sample-specific. A single comparison does not establish longitudinal drift or certification. Psychometric validation, representative fairness testing, software changes and claims of guaranteed AI accuracy are excluded.
Start here
Send your rubric, sample size, task format and the decision your team needs to make. I'll confirm the comparison scope.