#127
[Proposal]: Add ai audio sfx and ai audio music commands
Summary
Add ai audio sfx and ai audio music so people and agents can create audio:
ai audio sfx "a heavy wooden door slams shut" --duration 2
ai audio music "lo-fi beat with warm piano" --length 30 --instrumentalMotivation
The Gateway already serves sound effect and music models, but its catalog lists them with type speech, so ai models mixes them in with text-to-speech models, and ai audio speak is the only command that reaches them. That command is named and documented for reading text aloud, and it can't set a track's length or any other provider setting.
Dedicated commands make these models discoverable and let callers set their length, both in scripts and for agents using ai mcp.
Implementation notes
- Commands:
audio sfxtakes--duration,--loop, and--prompt-influence.audio musictakes--lengthand--instrumental. ElevenLabs is the only provider of these models in the catalog today, soelevenlabs/sound-effects-v2andelevenlabs/music-v2.5are the natural defaults. - Provider options: keep flag names provider-neutral and map them to each provider's options, so another provider can be added later without changing the commands. Check values against the provider's ranges before any request, and add
--provider-options <path>for settings that have no flag. - Models: split the Gateway's speech models by their
sound-effectsandmusictags, and letai models --typeacceptsfxandmusic. - MCP: add
audio_sfxandaudio_musictools to the media profile.
Testing notes
No response
Risks or tradeoffs
ai models --type speech would list only text-to-speech models, and ai models --json would report sfx or music instead of speech for these models.
Co-contributor credit
Yes, credit my GitHub username
Checklist
- I have searched existing issues for duplicates.
- I understand pull requests are limited to collaborators and maintainers will handle implementation.
I made a working implementation for reference:
kevintraver/ai-cli@feat/audio-generate.