I built an AI-assisted testing system that completed every assessment in two of EduAI Pro's teacher learning pathways, handed developers a prioritized defect report, and found integrity flaws a manual click-through would miss.

Client: EduAI Pro (live CPD and certification platform for Nigerian teachers)

My role: QA lead and product owner Timeline: 2026, in three phases Tools: Playwright, Node.js, Claude Code

Every EduAI Pro pathway has 12 units, and each unit ends in a scored assessment that counts towards a certificate. Testing that by hand is slow and unreliable:

A full pathway is 160+ questions, and each answer needs curriculum knowledge.

The quiz restarts at question 1 if the browser loses focus, which breaks ordinary automation tools.

Assessments write real progress, so tests have to run on controlled accounts without touching a live school's records.

A pass proves little on its own. The real question is whether the platform scores, gates and reports correctly.

I built a Playwright system that maps the real user journey and passed all 12 timed assessments. The first three units took months. Once I added an AI answer layer, the remaining nine passed in a single day.

My biggest find was that a teacher who completed Higher Education was issued a certificate for a different programme.

I tested the admin surfaces against production, with read-only rules for real data and a separate test school for anything that writes. I handed the developers a prioritized report with evidence and acceptance criteria for every finding, then fixed several defects myself through reviewed pull requests. Across Phases 1 and 2: i found 28 defects, 8 of them Critical.

A small team of AI agents now runs the suite, each with one job:

Explorer maps a live screen read-only before any test is written

Analyst runs the test and classifies each result as a real defect or an automation artefact

Verifier re-checks every claim from the evidence alone

Fix-check confirms a fix is live before a finding is closed Nothing is reported until the verifier confirms it.

12 of 12 units passed: 161 questions in total, with 94% average

About 3 minutes per unit, unattended; the whole pathway in about 37 minutes

147 questions answered by the AI resolver, 0 fallbacks after the fix

The passes weren't the valuable part. These were:

Assessment integrity (High): the answer key followed a predictable pattern, strong enough for a learner to pass units without studying. Details are shared only with the EduAI Pro team until it's fixed.

Module lock not enforced: the dashboard says "Complete all modules to unlock", yet every assessment opened and scored without them.

Slow load with no feedback: a median of 31 seconds from Start to the first question, with nothing on screen to show progress.

Content gaps: 9 of 12 units have fewer questions than the specification requires.

Every safeguard exists because something went wrong first:

An expired sign-in once made the AI fail silently and fall back to guessing. Now a preflight check stops the run before the browser opens, and every result reports AI vs fallback answers.

The script once assumed 15 questions and counted the last one twice. It now reads the real total and only reports a pass if Submit was actually clicked.

The analyst once reported a 32 s median load time; the verifier recomputed 31 s and the figure was corrected before it was reported. Credentials never pass through the agents, and writes are limited to test accounts.

OUTCOME

What used to take months of manual testing now runs unattended in minutes, with every finding backed by evidence developers can reproduce.

If you want this for your product, my Automated QA Audit tests your key user journeys and gives you a prioritized list of what breaks. See my services to get started.

https://contra.com/s/rkgrmrNe-automated-qa-audit-find-what-breaks-before-your-users-do