CAMS Research Case Study | Speaking Assessment & Feedback

This case study documents my published research, not a completed consulting engagement for a prospective client.

Online English tutors often have to teach, assess and give feedback in the same session. A rubric can make expectations visible, but it also needs shared interpretation, usable feedback and a routine learners can follow.

I conceptualised and investigated the CEFR-Aligned Monitoring System (CAMS): an analytic speaking rubric, rater calibration, structured written feedback and an in-session rubric routine. The paper records my contribution to methodology, investigation, analysis, writing and project administration. The rubric was developed with two assessment specialists.

The mixed-methods quasi-experimental study followed 214 adult learners aged 19–47 across three anonymised commercial online ESL platforms in Southeast Asia and the Middle East. There were 112 learners in the CAMS arm and 102 in the comparison arm, with 26 instructors: 14 and 12 respectively. Learners were grouped by instructor condition rather than individually randomised.

The 16-week implementation used five speaking criteria on a 0–5 scale: Range and Accuracy, Fluency and Coherence, Phonological Control, Interaction Management and Task Fulfilment. CAMS instructors attended a six-hour workshop with recorded samples, independent ratings, benchmark comparison and discussion of persistent discrepancies. Category-based feedback addressed performance, the next target and a concrete practice action. At least one rubric category was discussed with learners in each four-session cycle.

Pre- and post-test tasks included a short monologue, an opinion task with follow-up and a transactional interaction. Recordings were rated by two trained external raters blind to group. Qualitative evidence comprised 38 learner interviews and four focus groups involving 12 instructors; learner and instructor datasets were analysed separately before comparison.

Instructor session ratings had reported ICC(2,k) values of .87 (95% CI .82–.91) in the CAMS arm and .61 (.49–.71) in the comparison arm. The paper defines these as two-way random-effects, average-measures coefficients across 14 and 12 instructors respectively. They are not pairwise exact agreement and do not establish the reliability of one tutor's rating.

The reported composite effect size was d = .56; Fluency and Coherence was d = .74. Phonological Control did not show a statistically significant differential effect (p = .17). Learners described clearer expectations, and some reported using the rubric to monitor their own speaking. Instructors valued the structure but raised workload, task-fit and pronunciation-feedback concerns; the feedback protocol added approximately five to eight minutes of post-session work.

The findings concern CAMS as implemented as a system. Self-selection into conditions, attrition, the 16-week window and the use of outcome categories aligned to the intervention limit causal and general claims. The design cannot isolate the rubric's contribution, demonstrate broad communicative transfer, or establish validity for other populations and decisions.

An organisation can use these findings to ask practical questions: Do raters understand the same descriptors? Do tasks elicit the evidence being scored? Can teachers produce specific feedback within the available time? Do learners understand the next target? Who checks disagreements and whether the routine is sustainable?

My CAMS implementation pilot addresses those questions in one adult programme through a bounded review, provisional local adaptation, a two-hour introductory workshop and a monitoring handover. It is a new service, not a reproduction of the study. No client results, testimonials, certification or guaranteed reliability improvements are claimed. Local validation and follow-up evidence remain necessary.

Abdelaty, S. (2026). From rubric to monitoring system: a CEFR-aligned approach to reliability and learning in online ESL speaking. Frontiers in Education, 11, 1828864. Published 28 May 2026. https://doi.org/10.3389/feduc.2026.1828864