You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(enhance): clean up low quality sound before it's transcribed - #596
Before: the enhance step only measured and cleaned up images; recordings were transcribed straight from the uploaded audio, so a noisy, quiet or clipped recording transcribed badly.
After: the enhance step now listens to audio and video too. A low quality recording gets a cleaned-up copy (declipped, rumble and steady noise taken out, its level evened), and transcription hears that copy, kept in step with the original sample for sample so word timings still land where they were said. The original, its waveform and voice matching stay as they were. Off by default (Settings → Enhance). ffmpeg's filters are the default engine; WebRTC's noise suppression and gain (webrtc-noise-gain, Apache-2.0, fetched on first use) is an optional engine. Enhance and transcribe again from a recording's Details tab, the API (POST /api/v1/resources/{rid}/enhance), the MCP enhance tool, or the enhance automation action.
This folds in the previously parked "WebRTC audio cleanup node" idea as the webrtc engine.
Motivation and Context
Low quality sound is the biggest cause of bad transcripts, and the agent reads transcripts. Cleaning the audio before transcription raises quality at no cost to installs that leave it off. Follow-up to #479 (image enhancement), which built the enhance step and its settings surface.
Types of changes
New feature (non-breaking change which adds functionality)
Checklist:
The title is a Conventional Commit.
My code follows the code style of this project (make hooks checks it on commit).
I added or updated tests.
My change requires documentation updates, and I have updated docs/ accordingly.
Users need more than the title to understand it, so I wrote a changelog.d/ fragment (the new scheme replacing ## Unreleased).
Closing as superseded by #569, which landed on main as the clean_audio pipeline step (WebRTC noise suppression, gain, high-pass, echo cancellation and voice detection before transcription, original untouched, off by default). That covers the same "clean up low quality sound before it's transcribed" need, so this PR would be a second parallel cleanup step, which AGENTS.md (reuse first) rules out.
The only behaviour here that clean_audio doesn't have: gating the cleanup on a measured quality check, and keeping the cleaned audio as a stored rendition. #569 deliberately keeps no copy and cleans whenever the step is added. If those additions are wanted they belong on top of clean_audio, not as a rival step. The image half of this work shipped separately in #479.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Before: the
enhancestep only measured and cleaned up images; recordings were transcribed straight from the uploaded audio, so a noisy, quiet or clipped recording transcribed badly.After: the
enhancestep now listens to audio and video too. A low quality recording gets a cleaned-up copy (declipped, rumble and steady noise taken out, its level evened), and transcription hears that copy, kept in step with the original sample for sample so word timings still land where they were said. The original, its waveform and voice matching stay as they were. Off by default (Settings → Enhance). ffmpeg's filters are the default engine; WebRTC's noise suppression and gain (webrtc-noise-gain, Apache-2.0, fetched on first use) is an optional engine. Enhance and transcribe again from a recording's Details tab, the API (POST /api/v1/resources/{rid}/enhance), the MCPenhancetool, or theenhanceautomation action.This folds in the previously parked "WebRTC audio cleanup node" idea as the
webrtcengine.Motivation and Context
Low quality sound is the biggest cause of bad transcripts, and the agent reads transcripts. Cleaning the audio before transcription raises quality at no cost to installs that leave it off. Follow-up to #479 (image enhancement), which built the
enhancestep and its settings surface.Types of changes
Checklist:
make hookschecks it on commit).docs/accordingly.changelog.d/fragment (the new scheme replacing## Unreleased).Done questions (AGENTS.md)
enhancetool and theenhanceautomation action all enhance audio.enhance.audio_engineand thresholds in Settings → Enhance, with defaults, out of the wizard.test_enhance_audio.py,docs/enhance.md,make openapi).webrtc-noise-gainis Apache-2.0, an opt-in extra, so Lens stays MIT.enhancestep, ffmpeg and the existing rendition/deletion machinery.Generated by Claude Code