Development and Launch of VIV Platform
Jongeum Kim
VIV — AI-Powered Vocabulary Intelligence Platform
An AI-powered vocabulary intelligence platform that combines linguistic data, real-world web evidence, and AI analysis to understand not only what a word means, but where it is actually used.
Overview
VIV is a vocabulary intelligence platform built around a simple question:
Is this word worth learning, and where will I encounter it in real life?
Instead of relying only on dictionary definitions or difficulty levels, VIV combines linguistic frequency data, dictionary information, real-world search results, and AI-assisted analysis to build persistent vocabulary profiles.
I designed and built the backend intelligence layer, VIV Engine, along with an administrative dashboard for operating and monitoring its data pipelines.
My Role
Backend & AI Engineer — Personal Project
I designed the system architecture and implemented the backend APIs, asynchronous processing pipelines, database models, external service integrations, and administrative monitoring workflow.
Tech Stack
Python · FastAPI · PostgreSQL · SQLAlchemy · Celery · Redis · Docker · OpenAI API · Brave Search API · React · TypeScript
The Problem
Most vocabulary applications answer questions such as:
What does this word mean?
How difficult is it?
What is its CEFR level?
But difficulty alone doesn't tell a learner whether a word is practically useful.
A rare technical term and a commonly encountered advanced word may have similar difficulty levels while having very different learning value.
I wanted VIV to combine linguistic difficulty with evidence of real-world usage.
For each word, the system can build a structured profile containing:
Frequency and difficulty
Estimated CEFR level
Definitions and pronunciation
Real-world search evidence
Common usage domains
Aggregated topic signals
AI-generated explanations of where the word is commonly encountered
What I Built
Rather than building VIV as a request-time AI application, I designed the backend as a persistent asynchronous data pipeline.
The system is divided into three layers:
VIV Engine generates and enriches vocabulary intelligence.
PostgreSQL acts as the persistent source of truth for both intermediate pipeline state and completed vocabulary profiles.
VIV App consumes the already-enriched data without repeating expensive web searches or AI inference during user requests.
This separates expensive intelligence generation from the user-facing application.
Asynchronous Processing Architecture
Analyzing a word may involve dictionary lookups, web searches, multiple AI inference calls, and aggregation.
Running all of this inside a single HTTP request would create long-running requests and make application reliability dependent on several external services.
Instead, I built the processing workflow around FastAPI, Celery, and Redis.
FastAPI controls pipeline execution while Celery workers independently process enrichment stages in the background.
Each stage persists its output to PostgreSQL before the next stage consumes it.
A typical research pipeline looks like:
Word → Web Search → Evidence Storage → Topic Classification → Signal Aggregation → AI Summary
This architecture allows expensive processing to scale independently from the user-facing application.
Designing for Failure and Recovery
One of the key architectural decisions was to use PostgreSQL as persistent pipeline state, rather than passing all intermediate results through Celery task chains.
Each enrichment stage reads the state it needs from the database and writes its result back.
That means completed work survives worker failures or temporary external API outages.
If processing fails halfway through a vocabulary item, previously completed enrichment does not need to be discarded and recomputed.
The same database later becomes the source of truth consumed by the user-facing application.
AI Based on Real-World Evidence
I also wanted to avoid treating the language model as an unquestioned source of truth.
Instead of simply asking an AI:
“What topic is this word used in?”
VIV first collects real-world search results.
Each result is classified independently into domains such as technology, business, education, medicine, entertainment, or daily life.
Those classifications are then aggregated into a usage profile.
The language model therefore operates over observed evidence, rather than generating the entire answer from a single unconstrained prompt.
This produces a more structured and explainable foundation for vocabulary analysis.
Cost-Aware Processing
Not every word needs expensive AI and web-search processing.
I therefore separated enrichment into two levels:
Basic enrichment handles frequency, difficulty, CEFR estimation, definitions, and pronunciation.
Research enrichment performs web search, AI classification, aggregation, and usage summarization only when deeper analysis is useful.
This creates a natural control point for API cost, processing time, retries, and future enrichment strategies.
Pipeline Monitoring
I also built a React and TypeScript administrative dashboard for operating the engine.
Each pipeline execution creates a persistent
PipelineRun, allowing the dashboard to display:Pipeline status
Processing progress
Recent execution logs
Because operational state is persisted outside individual workers, the dashboard can monitor the system without needing direct knowledge of Celery processes.
Result
VIV Engine evolved from a vocabulary-analysis experiment into a small data processing and AI knowledge-generation platform.
The architecture now provides:
Asynchronous and independently scalable processing
Persistent recovery-friendly pipeline state
Separation between AI generation and user-facing requests
Evidence-based AI classification
Cost-aware enrichment stages
Administrative monitoring
A reusable vocabulary intelligence layer for downstream applications
The biggest lesson from the project was that expensive AI operations become much easier to control when they are treated as one stage of a persistent data pipeline, rather than the center of every user request.
Project Links
Like this project
Posted Sep 20, 2026
Developed VIV, a platform for AI-driven vocabulary analysis with real-world usage data.
Likes
0
Views
0
