Skip to content

feat(speech): send long recordings to a provider in pieces that fit its limits - #362

Merged
adeelahmad merged 1 commit into
mainfrom
claude/project-thread-j2ydqi
Oct 14, 2026
Merged

adeelahmad merged 1 commit into
mainfrom
claude/project-thread-j2ydqi

Conversation

@adeelahmad

Copy link
Copy Markdown
Owner

Requested by Adeel · project thread

Description

Before: only OpenAI-compatible servers got long recordings in ten-minute parts. ElevenLabs, AssemblyAI and Deepgram were sent the whole recording in one request, so one over the provider's limit (10 hours for ElevenLabs and AssemblyAI; for Deepgram, anything that takes it over 10 minutes of work, which it answers with a 504) failed.

After: every speech provider gets a recording that is too long or too large for one request in pieces that fit it. Each piece ends at a pause, is transcribed on its own, and the transcripts are joined back into one on the recording's timeline. Each provider starts with its published limits, and Settings → Speech providers has Longest piece sent at once and Largest file sent at once for each one.

Provider Longest piece Largest file Source
OpenAI-compatible 10 min 25 MB OpenAI (25 MB; GPT-4o transcription takes up to 25 min)
ElevenLabs 600 min 3000 MB ElevenLabs (10 h, 3 GB)
AssemblyAI 600 min 2200 MB AssemblyAI (10 h; 2.2 GB upload)
Deepgram 120 min 2000 MB Deepgram (2 GB; 504 after 10 min of work)

How: speech.Engine.transcribe now owns the splitting for every provider. pieces() cuts at the quietest tenth of a second in the half minute before each duration limit (numpy energy over the samples already decoded). clips() encodes each piece and halves one whose file is still over the size limit, again at a pause. The provider functions now take one encoded clip. Segments, word timings and speaker labels are shifted onto the recording's timeline, and speaker labels get a cN: prefix per piece, as the OpenAI parts had before. The activity ledger still writes one row per recording, now with pieces when there was more than one.

Motivation and Context

Adeel asked for large audio files to be segmented automatically to each provider's limits.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change; ! in the title)

Checklist:

  • The title is a Conventional Commit.
  • My code follows the code style of this project (ruff, mypy, eslint, prettier, tsc clean locally).
  • I added or updated tests.
  • My change requires documentation updates, and I have updated docs/ accordingly (docs/speech-providers.md#long-recordings; mkdocs build --strict clean).
  • Users need more than the title to understand it, so I wrote it under ## Unreleased in CHANGELOG.md.

Done questions (AGENTS.md)

  • Goal: long recordings are split to each provider's limits and joined; I merge it once CI is green.
  • Agents: the limits are ordinary settings (speech.{provider}_max_minutes and _max_mb), so the settings API and the assistant's change_settings tool can change them. The splitting needs nothing from anyone.
  • Zero setup: defaults come from each provider's published limits, with no new dependency (ffmpeg and numpy were already used).
  • Fewest clicks: nothing to do. The two fields per provider are only there for proxies or compatible servers with other limits.
  • Configurable: Settings → Speech providers has both fields for each provider that transcribes, with defaults. Nothing was added to the wizard.
  • Light: a numpy pass over samples already in memory, with no new package or model.
  • Private and secure: the same audio goes to the same provider the admin picked, in pieces. No new surface, so the threat model is unchanged (n/a).
  • Cost: same audio, same provider, one ledger row per recording with pieces. Budgets still gate the call.
  • User decides: nothing is sent until an admin picks a provider, as before. Defaults are unchanged for local engines, and OpenAI keeps its ten-minute parts.
  • Upgrades: n/a. No stored data changes shape, and the new settings keys fall back to defaults.
  • Honest: each default limit cites its provider's docs. The docs say a new voice heard in two pieces shows as two speakers until merged.
  • Tests, docs and generated files: backend tests for limits, settings validation, cutting at pauses, a 3-piece Deepgram run joined back, and halving by file size; a frontend test for the fields. OpenAPI is unchanged (openapi-check diff empty).
  • Open standards; no new dependencies.
  • Result reported in the project thread. Refine-later items are in DEVLOG.md: link a voice across pieces by voiceprint, decode and send a piece at a time to save memory, send pieces in parallel, and overlap cuts that have no pause near them.

🤖 Generated with Claude Code

https://claude.ai/code/session_01L9FY5GYBynyZQi49uouhUj


Generated by Claude Code

…ts limits

A recording longer or larger than a speech provider takes in one request is
cut at pauses into pieces within its limits, each piece is transcribed, and
the transcripts are joined on the recording's timeline. Each provider starts
with its published limits (speech.{provider}_max_minutes and _max_mb),
editable in Settings → Speech providers. Replaces the ten-minute parts only
OpenAI-compatible servers had.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L9FY5GYBynyZQi49uouhUj
@adeelahmad adeelahmad self-assigned this Oct 14, 2026
@adeelahmad
adeelahmad merged commit 0921a4b into main Oct 14, 2026
10 checks passed
@adeelahmad
adeelahmad deleted the claude/project-thread-j2ydqi branch October 14, 2026 15:57
adeelahmad pushed a commit that referenced this pull request Oct 14, 2026
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014pKRz6HkdQ7hTqEbpnyhYo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants