Transcribing a podcast is like turning a live lecture into a textbook with chapters and a searchable index - and doing it while the ink is still wet. A 90-minute episode contains roughly 13,500 words of spoken audio. Sequential transcription of that file through a GPU-accelerated ASR model takes around 3 minutes end-to-end. That sounds fast, but at 50,000 hours of audio uploaded per day, sequential processing would require hundreds of GPU instances running 24 hours a day with no slack for burst uploads. The system must parallelize.

The first tension is between chunking for speed and accuracy at boundaries. If you split a 90-minute audio file into 30-second chunks for parallel processing, words that span a chunk boundary get cut in half. “Unbelievable” becomes “Unbelie-” at the end of chunk 12 and “-vable” at the start of chunk 13. The fix is overlapping chunks - extend each chunk by 3 seconds on each side - but now the stitch worker must resolve duplicate transcriptions of the overlap region without doubling words.

The second tension is between chapter detection quality and latency. The best chapter boundary detection uses a large language model that reads the full transcript and identifies semantic shifts. But a 90-minute transcript is too large for a single LLM context window, and waiting for full transcription before running chapter detection defeats the goal of having chapters available within 5 minutes of upload. The system must detect boundaries incrementally, operating on sliding windows of transcript as chunks complete.

The third tension is cost versus latency. A100 GPUs that process 30-second audio chunks in under 2 seconds cost significantly more per hour than CPU instances. For popular podcasts where listeners expect near-instant transcripts, the GPU cost is justified. For a long-tail episode with 200 expected listeners, it is not. We need tiered processing: GPU-accelerated for popular or premium content, batch CPU processing for the long tail. We solve for chunked audio processing, ASR model integration, speaker diarization, topic boundary detection, transcript indexing, pipeline parallelism, and cost-latency tradeoff simultaneously.

Requirements and Constraints

Functional Requirements

• Transcribe audio uploaded to the platform with word-level timestamps
• Detect natural topic boundaries and generate chapter markers with titles
• Identify speaker transitions and label segments (Speaker 1, Speaker 2, …)
• Make the transcript full-text searchable within 10 minutes of upload completion
• Support episodes up to 4 hours in length

Non-Functional Requirements

• Transcription available within 5 minutes for episodes under 60 minutes
• Chapter detection accuracy: at least 70% of human-labeled boundaries identified
• Search index freshness: under 10 minutes from upload completion
• Daily throughput: 50,000 hours of audio per day
• ASR word error rate (WER): under 8% for clean studio audio
• System availability: 99.9% (degraded mode acceptable, total outage not)

Constraints and Assumptions

• Audio is in MP3, AAC, or WAV format; video is supported by extracting the audio track first
• Speaker identification is probabilistic - we label “Speaker 1, Speaker 2” not actual names without voice enrollment
• Chapter titles are generated heuristically from keyword extraction, not via LLM (to control latency)
• Transcription cost is a first-class constraint: GPU time must be tracked per episode

High-Level Architecture

The pipeline has five processing stages that run with as much parallelism as possible. Audio arrives at an Upload API, is written to Object Storage (S3), and triggers a Job Queue entry. The Job Queue fans work out to five parallel processor types: a Chunker that splits audio into overlapping 30-second segments, an ASR Worker Pool that transcribes each chunk on GPU, a Diarization Worker that identifies speakers, a Chapter Detector that finds topic boundaries, and an Index Writer that writes the completed transcript to Elasticsearch.

The key architectural insight is that chunking and ASR run in parallel with diarization. Diarization requires the full audio file (it needs global speaker context to label consistently), so it runs as a separate parallel job against the original file rather than waiting for chunk transcripts. When both the ASR stitch and diarization output are ready, a Merge Worker aligns them by timestamp to produce speaker-labeled transcript segments.

Chapter detection can begin as soon as the first 5 minutes of transcript is available. It processes the transcript in a sliding window, emitting preliminary chapter boundaries that are refined as more transcript arrives. The final chapter list is produced when the full transcript is assembled.

Chunked Audio Processing