A voice-first AI teaching assistant designed for Haryana government schools. Teachers can speak in Hinglish and the AI explains concepts, generates quizzes, and creates lesson plans — all projected on a smart board.

A standalone hero screen opens before the dashboard — an animated voice orb (tap or click to enter), the app's pitch in Hinglish, and three feature chips. Click the orb or the "Shiksha AI Shuru Karo" button to enter the main app. The logo in the dashboard header is also clickable to return here.

Teacher speaks or types a topic — works in Hinglish, English, or a mix

AI returns both a Hinglish and a clean English version of the explanation, grade-appropriate (Class 3–12); a pill toggle switches between them

AI also generates a real Mermaid.js diagram, rendered live on the smart board (not just a text description) — with a "Diagram Banao" retry button if generation fails or returns nothing

Includes a Visual Tip for blackboard drawing as a teacher's drawing cue

TTS speaks the Hinglish version aloud automatically (the assignment's primary spoken-classroom requirement) — a "Sunao" button replays it anytime, with a "Stop" control while it's playing, matching the quiz and lesson plan screens

"Ready for a quick quiz?" button instantly jumps into Feature 2 — a connected lesson flow, not two disconnected tools

Say "create a quiz on photosynthesis" / "quiz shuru karo" — fully voice-triggered, not just a button

AI generates 3 MCQ questions, each with a Hinglish and English version — same language toggle as the explanation screen

Full spoken readout, not just an announcement: the app speaks "Quiz shuru! Topic hai...", then reads every question aloud one at a time, including all four lettered options ("Option A: ...", "Option B: ..."), before going silent so students can answer

The question currently being read is highlighted on screen and auto-scrolled into view, so a class watching the smart board can follow along with what's being spoken

"Sunao" button to re-trigger the full readout anytime; "Stop" button appears while reading, both in the quiz header and in the sidebar mic control

Countdown timer, visual A/B/C/D options

Shows bilingual explanations after submission

Say "make a lesson plan for fractions" / "lesson plan banao" — also fully voice-triggered

AI generates a structured 10-min lesson in both Hinglish and English: concept → example → activity → quiz → homework

Each section renders as its own labeled card with an icon (Concept, Example, Activity, Quiz, Homework) instead of one merged paragraph — the backend returns each section as a separate JSON field specifically so the model can't accidentally collapse them into running prose

Spoken Hinglish confirmation when ready (not the full plan read aloud — see Known Limitations)

Runs on its own dedicated system prompt (previously shared one prompt with the explainer feature, which caused inconsistent formatting)

While the app reads text aloud (explanation, quiz questions/options, or the lesson plan via "Sunao"), the word currently being spoken is highlighted in saffron in sync with the audio, across all three features.

This is implemented using the Web Speech API's boundary event, which fires as the speech engine crosses each word and reports a charIndex. That index is mapped to a word position, and the matching <span> is highlighted. Two notes on reliability:

Best-effort, not guaranteed: boundary event support varies by browser/OS/voice combination, especially for non-Latin-script voices like hi-IN — on some platforms it doesn't fire at all. When it doesn't, speech still plays normally; the text just isn't highlighted. There's no broken state, only a graceful absence of the effect.

Segmented text needs offset math: the quiz reads a full question (with "Sawaal number N" and all 4 lettered options) as one spoken string, and the lesson plan reads all 5 sections as one combined string with labels — but each is displayed as separate, smaller pieces (one option button, one section card). The highlighting logic tracks each displayed piece's starting word-offset within the full spoken text, so the right word lights up in the right card even though TTS is reading one long continuous string behind the scenes.

All three features are voice-triggered from a single mic button — say a command with or without a topic, and the app routes it correctly:

Say this Routes to "Explain photosynthesis" / "photosynthesis samjhao" Concept Simplification "Create a quiz on water cycle" / "quiz shuru karo" Voice-Triggered Quiz "Make a lesson plan for fractions" / "lesson plan banao" Lesson Plan Generator Just a topic name, no command word Defaults to Concept Simplification

Why en-IN and not hi-IN for speech recognition: an earlier version used hi-IN, which transcribes English loanwords like "quiz" into Devanagari script or phonetic spellings — silently breaking keyword-based command detection (the app would fall through to "explain" instead of recognizing "quiz"). en-IN keeps English command words in Latin script while still recognizing common Hindi words phonetically, which matters far more for a command-driven app than perfect Hindi transcription accuracy. Command detection itself still matches both Latin-script and Devanagari keywords as a safety net (क्विज, सवाल, पाठ योजना, etc.) in case the recognizer guesses a Hindi-heavy utterance anyway.

Topic extraction and headline display: people naturally phrase commands conversationally — "explain me photosynthesis," "can you explain water cycle," "I want to know about gravity" — and any word not explicitly recognized as a command/filler term would otherwise leak into the displayed topic (e.g. an earlier bug showed "Me Photosynthesis" as the headline instead of "Photosynthesis"). The filler-word list covers common wrappers in both English and Hindi, with multi-word phrases like "tell me" checked before single-word patterns like "me" so they don't get fragmented out of order. Whatever topic string remains is then title-cased for display (utils/format.js) — minor words like "of"/"the"/"a" stay lowercase unless they're the first word, so "laws of motion" reads naturally instead of "Laws Of Motion."

Layer Technology Frontend React 18 + Vite + Tailwind CSS Backend Node.js + Express.js AI Groq API (groq-sdk) — Llama 3.3 70B, JSON mode (response_format: json_object) for structured bilingual output Diagrams Mermaid.js — rendered live from AI-generated syntax, sanitized server-side Speech-to-Text Browser Web Speech API, en-IN locale (free, no API key) Text-to-Speech Browser SpeechSynthesis API, hi-IN voice (free, no API key) Logging morgan (HTTP request logging) Database None (stateless; session stored in memory)

Why this works: All three content-generating endpoints use Groq's JSON mode (response_format: { type: "json_object" }) alongside an explicit schema in the prompt — this combination is far more reliable than prompt instructions alone, which is what the earlier prose-based explanation prompt relied on. Grade-level instruction ensures age-appropriate language in both languages equally. The bilingual requirement means a teacher (or evaluator) can read the same content in whichever language suits the moment, while Hinglish stays the default and the only one spoken aloud via TTS. The diagram prompt is intentionally separated from the explanation prompt and run in parallel — if diagram generation fails or the model returns malformed Mermaid syntax, the explanation still succeeds (diagram is best-effort, not blocking).

Why the lesson plan schema uses one JSON field per section, not one big string: an earlier version asked the model to write "5 numbered sections" inside a single free-text field and trusted it to insert real paragraph breaks. It often didn't — sections ran together into one block of prose with no visual separation, since nothing in JSON forces a string to contain a literal newline between "1. Concept" and "2. Example." Making each section its own field (concept, example, activity, quiz, homework) removes that failure mode structurally: the model physically cannot merge sections, because they're separate JSON keys, not parts of one string it has to remember to break up correctly.

In-memory session store: resets on every backend restart/redeploy. Acceptable for a 4-day prototype; a real deployment would use Redis or a database.

No persistent settings: grade preference resets on page refresh. Documented here rather than hidden.

Mermaid diagrams are best-effort, with safeguards: Llama 3.3 occasionally returns near-valid Mermaid syntax (stray parentheses, wrong diagram type, markdown fences). The backend runs every diagram response through a sanitizeMermaid() cleaner and rejects anything that still doesn't start with a valid graph TD header rather than passing it through. The standalone /api/diagram regenerate endpoint also retries once at a lower temperature before giving up. If no valid diagram comes back, the UI shows a clear "Diagram nahi bana" state with a one-tap "Diagram Banao" retry button instead of a silent gap or a broken render.

Lesson plan isn't read aloud in full: TTS speaks a short Hinglish confirmation ("Lesson plan ready hai...") rather than all 5 sections, since reading a full lesson plan aloud would take 30-60+ seconds and feel tedious in a live classroom demo. The full bilingual text is shown on screen immediately, and the "Sunao" button reads it aloud on demand if the teacher wants it.

STT and TTS deliberately use different locales: speech recognition uses en-IN (keeps English command words in Latin script for reliable detection) while speech synthesis uses hi-IN (natural-sounding spoken Hinglish). This is intentional, not an oversight — see Voice Command Detection above for why hi-IN recognition silently broke command matching.

Quiz readout uses a token-guarded speech queue, not a naive loop: the browser's speechSynthesis.cancel() does not reliably fire onend across browsers (Safari fires onerror instead; Chrome is inconsistent on long utterances), so a naive "speak next question when this one ends" chain can keep talking even after the teacher hits Stop. useSpeech's speakSequence() assigns each readout an incrementing token; every step checks the token is still current before speaking the next item, so stopping mid-quiz reliably halts the whole sequence instead of just the current sentence.

Output language: every AI-generated response (explanation, quiz, lesson plan) comes back in both Hinglish and clean English, with a pill toggle to switch — the assignment's Hinglish requirement stays the default experience, English is there for anyone who prefers reading it that way

Examples: Roti, khet, bazaar, nadi, suraj — relatable Indian daily life, present in the Hinglish version of every response

Speech recognition: en-IN locale — keeps English command words ("quiz", "explain", "lesson plan") in Latin script so voice commands are detected reliably; still recognizes common Hindi words phonetically

Speech synthesis: hi-IN voice — always speaks the Hinglish version aloud, since that's the natural spoken register for a Haryana classroom

Tone: Warm, encouraging teacher voice in both languages — "Shabash!", "Samjhe?", "Bahut achha!" in Hinglish; natural equivalents in English

Node.js 18+

OpenAI API key

Open http://localhost:5173 in Chrome (required for Web Speech API).

Frontend: Vercel — connect GitHub repo, set root to /frontend

Backend: Render — set root to /backend, add GROQ_API_KEY env var

CORS: Already configured for cross-origin requests

After deploying backend, set VITE_API_URL=https://your-render-url.onrender.com/api in Vercel env vars.

Open app — show the landing page, tap the voice orb to enter

Select "Class 7", say "Photosynthesis explain karo"

AI responds in Hinglish, reads aloud with each word highlighting in sync, shows a live Mermaid diagram + Visual Tip — tap the English pill to show the bilingual toggle

Say "create a quiz on photosynthesis" directly into the mic — the app auto-switches to the Quiz tab and reads the announcement plus all 3 questions and their options aloud, highlighting each question and option as it's spoken

Answer questions, submit, see score with bilingual explanations

Say "make a lesson plan for fractions" — auto-switches tabs, speaks a confirmation, shows each of the 5 sections (Concept, Example, Activity, Quiz, Homework) as its own labeled card in both languages

Criterion Implementation Technical (40%) React + Express + OpenAI + Web Speech API; clean component architecture Empathy/UX (30%) Dark smart-board UI; large text; Hinglish throughout; encouraging tone AI/Prompt Design (30%) Strict system prompts; grade-level adaptation; Hinglish guardrails; JSON schema for quiz