Repository navigation
feat(pipelines): clean up a recording's sound before it's transcribed - #569
Merged
Merged
Conversation
A new pipeline step, enhance (Clean up audio), runs a recording through WebRTC's audio processing with pywebrtc-audio (Apache-2.0, ~0.5 MB, fetched the first time the step runs): noise suppression, automatic gain, a high-pass filter, echo cancellation for two-channel call recordings, and voice detection that leaves pauses out of what's transcribed. The transcript's times are mapped back onto the recording's own timeline. The step keeps its options and what it measured on the recording (audio_cleanup); transcribing then hears the cleaned audio. The file is never changed and no copy is stored. No pipeline has the step until one adds it, so defaults are unchanged. Settings > Transcription > Audio cleanup holds the options (transcribe.cleanup). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014WcBxU5ARBoLWMYZU46St6
…enhance step in #479 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014WcBxU5ARBoLWMYZU46St6
18 of 23 tasks
…-zrzew2 # Conflicts: # CHANGELOG.md # fastapi_backend/app/domain/pipelines.py # nextjs-frontend/components/pipelines/pipeline-canvas.tsx
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014WcBxU5ARBoLWMYZU46St6
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014WcBxU5ARBoLWMYZU46St6
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014WcBxU5ARBoLWMYZU46St6
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014WcBxU5ARBoLWMYZU46St6
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014WcBxU5ARBoLWMYZU46St6
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014WcBxU5ARBoLWMYZU46St6
This was referenced Oct 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Adeel · project thread
Before: speech-to-text heard a recording exactly as it was: fans, hiss, a quiet speaker, the far side of a call echoing back, and long silences that cloud engines bill for and Whisper imagines words in.
After: a new pipeline step, Clean up audio (
clean_audio), runs a recording through WebRTC's audio processing with pywebrtc-audio (Apache-2.0, ~0.5 MB, numpy only, wheels for Linux x86_64/aarch64 and macOS): noise suppression (off/6/12/18/21 dB), automatic gain, a high-pass filter, echo cancellation for two-channel call recordings, and pauses of 2 s or longer left out of what's transcribed. The transcript's times (and word times) are put back on the recording's own timeline, so the player and citations stay right. No pipeline has the step until someone adds it, so defaults are unchanged.How: the step measures speech with WebRTC's voice detection and keeps its options and what it found on the recording (
audio_cleanup, also inGET /api/v1/recordings/{id}).transcribe_onethen hears the cleaned audio, cut to its speech spans, and maps segment times back through the joins. The file is never changed and no copy is stored (cleaning is ~10 µs per 10 ms frame, so it's redone each time). Speaker separation and voice IDs still use the original. Options live in Settings → Transcription → Audio cleanup (transcribe.cleanup, validated server side). The library is an extra (webrtc) fetched on first use by a component, or inline by the step; a worker that can't get it transcribes the original and says so. The step is in the pipeline editor, canvas, Reprocess and batch Run on. It's namedclean_audioto stay apart from #479's imageenhancestep.Checked by hand on a real speech sample (Open Speech Repository): voice detection tracked the sentences; on synthetic tests echo cancellation took the far side down by >20x and the trimmed audio was 10 s of 22 s.
Done questions
{"steps": ["clean_audio", "transcribe"]}), results inaudio_cleanupon the recording, settings viaPUT /settings/transcribe(test_an_agent_reads_what_the_step_did).workers.stepslists withtranscriberun the step too.tests/domain/test_audio_cleanup.py(12 tests), the Reprocess jest test,docs/processing.md#cleaning-up-audio, components table, CHANGELOG,docs/requirements.md; client regenerated withmake openapi. Full backend suite: 1654 passed locally.Generated by Claude Code