AI-Powered Video Processing Pipeline (Vision-LLM + Kafka + FastAPI) I built an end-to-end pipeline that turns an uploaded video into searchable, summarized knowledge. Videos are pulled from S3 and sampled at 1 frame per second with OpenCV. Blank videos are rejected early with a pixel-variance check. Frames are sent in batches of 500 to a Vision LLM, which describes each batch. In parallel, FFmpeg extracts the audio and Whisper turbo on Groq transcribes it. gpt-oss-120b then combines the batch notes and the transcript into one end-to-end summary, and everything is stored in Qdrant for semantic search. Kafka and FastAPI run the jobs asynchronously, so large uploads never hit HTTP timeouts. Ingestion: The frontend uploads the video to Amazon S3. A FastAPI endpoint publishes a job to Kafka and returns immediately, and a consumer downloads the video with boto3.

Visual path: OpenCV samples 1 frame per second. A blank-frame check (standard deviation below 3) terminates invalid input early. Valid frames are Base64-encoded in batches of 500, and a Vision LLM describes each batch.

Audio path: FFmpeg extracts the audio track, and Whisper turbo on Groq converts speech to text.

Synthesis: gpt-oss-120b on Groq merges all batch notes and the transcript into a single summary of the whole video.

Storage: The batch notes and the final summary are stored in Qdrant, so the video’s content can be searched and used in RAG workflows. Tools to tag Python, FastAPI, Kafka, OpenCV, FFmpeg, Amazon S3, Whisper, Groq, OpenAI, Qdrant, Vision LLM, RAG