Skip to content

bug(gateway): Telegram voice notes emit bogus incapability message instead of using local Whisper fallback #1033

@atuida

Description

@atuida

Summary

On Telegram, incoming voice notes can trigger a canned fallback message like:

"The user sent a voice message but I can't listen to it right now~ VOICE_TOOLS_OPENAI_KEY isn't set up yet"

That user-facing message is wrong when local transcription is actually available on the machine. In my environment, local whisper and ffmpeg are installed and the same inbound .ogg can be transcribed successfully, but the gateway/session path still emits the failure message before the agent can use the local fallback.

Why this is bad

This presents a fake capability failure to the user even though the machine can transcribe the voice note. It makes Hermes look less capable than it is and breaks trust.

Repro

Environment characteristics:

  • Telegram DM / gateway session
  • VOICE_TOOLS_OPENAI_KEY unset
  • local Whisper available at /opt/homebrew/bin/whisper
  • local ffmpeg available at /opt/homebrew/bin/ffmpeg
  • inbound Telegram voice arrives as .ogg

Observed behavior:

  1. User sends Telegram voice note
  2. Hermes emits the canned "can't listen" / missing-key message
  3. But the same .ogg is transcribable locally with:
    PATH=/opt/homebrew/bin:$PATH whisper <file>.ogg --model tiny --output_format txt

Root cause (likely)

The active inbound voice transcription path appears to:

  1. gate too early on VOICE_TOOLS_OPENAI_KEY, and/or
  2. fail to see local binaries because gateway PATH does not include /opt/homebrew/bin, and/or
  3. emit the user-facing failure before attempting local STT fallback

Expected behavior

If OpenAI voice credentials are unavailable, Hermes should:

  1. try configured local STT fallback first
  2. normalize PATH or support absolute binary path config
  3. only emit a user-facing failure if all STT backends fail

Requested fix

  • Prefer local STT fallback before sending any failure text to the user
  • Ensure gateway/runtime PATH includes common local bin dirs (/opt/homebrew/bin, /usr/local/bin) or support explicit binary path config
  • Make the user-facing message accurate: no canned incapability notice when local fallback exists

Related work

This seems related to the local STT fallback work in #942, but the Telegram/gateway behavior still reproduces in real use if local binaries are not discovered correctly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions