Back to portfolio
AI MediaProduction-ready MVPFull Case Study

Caption CC

Code-Switching Subtitle Generator

Caption CC: Speech-to-Text
Speech-to-Text
Caption CC: Subtitle Editor
Subtitle Editor

Caption CC generates accurate English subtitles from Tamil-English code-switched video, with optional AI dubbing. A 5-stage caption pipeline and 7-stage dubbing pipeline handle everything from download to export, supporting YouTube URLs and direct file uploads up to 500 MB.

5

Caption Stages

7

Dubbing Stages

10 min

Max Duration

SRT, VTT, MP4

Export Formats

01

Caption Pipeline

Five-stage pipeline: download or upload → FFmpeg extracts WAV (16kHz mono) → Groq Whisper transcribes → Gemini corrects code-switching and translates to English → subtitle generation as SRT, VTT, or burned-in MP4. Subtitle segments are editable in-app before export. Real-time progress via Server-Sent Events.

02

AI Dubbing Pipeline

Seven-stage dubbing flow adds TTS and audio assembly on top of captioning: after translation, ElevenLabs synthesizes each segment, FFmpeg assembles the full audio track, and the dubbed audio is merged back into the video. The final dubbed MP4 is available for one-click download. Timing is best-effort matched to original segment boundaries.

03

Architecture & Limits

Full-stack TypeScript monorepo with npm workspaces (client + server). MongoDB tracks job state, subtitle segments, and usage per user. LLM temperature set to 0.3 with 3 retries and backoff. Supports MP4, MP3, WAV, M4A, MOV, WEBM. Max file size 500 MB, max duration 10 minutes.

Operational Value

Cuts manual captioning and translation effort. Expands regional content into English-speaking audiences. Creates reusable subtitle and dubbed media assets for publishing.

Tech Stack

FrontendReact 18, TypeScript, Vite, Tailwind CSS
BackendNode.js 18+, Express, TypeScript
DatabaseMongoDB: jobs, segments, usage, user_access
STTGroq Whisper (whisper-large-v3-turbo), ElevenLabs Scribe
LLMGoogle Gemini 2.5 Flash: correction + translation
TTS / DubElevenLabs eleven_multilingual_v2
MediaFFmpeg (audio extraction, video merge), yt-dlp (YouTube)
AuthGoogle OAuth 2.0 via Passport.js

Full Stack

React 18TypeScriptViteNode.jsExpressMongoDBGroq WhisperGoogle GeminiElevenLabsFFmpegyt-dlp

Related Service