Paper Discussion: CrisperWhisper

Join us for a deep-dive paper discussion on CrisperWhisper, a novel approach that significantly improves word-level timestamp accuracy in speech transcriptions using OpenAI's Whisper model. We'll explore how the authors modified Whisper's tokenizer, applied Dynamic Time Warping (DTW) to cross-attention scores, and fine-tuned the model to handle noise, disfluencies, and multiple speakers.

Paper link: https://arxiv.org/abs/2408.16589