Playing Music from an Ability

One function on CapabilityWorker. You call it, it plays a track and blocks until playback genuinely ends, and it tells you why it ended.

result = await self.capability_worker.stream_music_from_url(url, duration_seconds=210.0)
result["outcome"]     # "finished" | "paused" | "stopped" | "unplayable" | "error"

You do not manage the audio pipeline, the device buffer, music mode, or the recovery afterwards. All of that is inside the call. What you do own is deciding what happens next based on outcome — and that’s the whole job. Speaking is one-shot: you hand speak() a sentence and it comes back when the sentence is over. Music isn’t like that. A track is minutes long, the user can interrupt it halfway with their voice, and after that interruption the device has to be walked back to a state where your next sentence is actually audible. This function is one await that covers a whole listening session and returns a decision point.

`stream_music_from_url()`

result = await self.capability_worker.stream_music_from_url(
    url,                                     # required
    headers={"Authorization": "Bearer ..."}, # optional
    duration_seconds=210.0,                  # strongly recommended
    start_seconds=0.0,                       # resume only
    byte_offset=0,                           # resume only
    announce="Playing Blinding Lights.")     # optional

Parameter	Required	What it’s for
`url`	yes	A progressive mp3 link. Not an HLS or DASH manifest.
`headers`	no	Extra request headers, e.g. an `Authorization` line.
`duration_seconds`	recommended	FULL track length in seconds. `0` guesses three minutes.
`start_seconds`	resume only	Seconds already heard. Pass `result["position"]` from the previous leg.
`byte_offset`	resume only	Where in the file to pick up. Pass `result["byte_offset"]`.
`announce`	no	Spoken before playback starts. Leave empty on a resume.

Returns {"outcome", "position", "sent", "byte_offset"}.

duration_seconds is the full track length, not what’s left — the call subtracts start_seconds for you. Bad values don’t crash anything, they just make the reported position inaccurate, so a later resume starts in the wrong place. When you don’t know the real length, pass 0 and take the estimate.

During playback

While a stream is live the Ability is in music mode. Normal conversation is suspended: your Ability is not receiving transcriptions, and you must not call speak() or run_io_loop(). Two things get through, and they’re the two the user actually needs.

The user says	What happens	What you get
”pause”	audio stops, buffer cleared, position remembered	call returns `outcome: "paused"`
”stop”	audio stops, buffer cleared	call returns `outcome: "stopped"`
(track runs out)	playback completes naturally	call returns `outcome: "finished"`

You never handle those events yourself. The platform raises them, the function is watching for them, and it converts them into a return value — typically within a few tenths of a second of the user speaking. From your side a pause is simply “the await came back, and outcome says paused.” However a call ends — finished, paused, stopped, even error — the client is audible again and the platform is listening when it returns. You can speak() or open an IO loop on the very next line. That’s true of the error path too, so you never need cleanup of your own.

Reading `outcome`

`outcome`	Means	Usual response
`finished`	the track played to the end	queue the next one, or ask what’s next
`paused`	user wants it held	ask, then resume or leave
`stopped`	user wants playback over	acknowledge, stop asking
`unplayable`	the link never delivered audio	try the next candidate track
`error`	something failed mid-stream	apologise once, don’t retry the same track

position is seconds heard (including start_seconds), and it’s the halt instant — not “when cleanup finished” — so feeding it into a resume lands where the user left off. sent is bytes delivered, useful for logs.

Resume is just calling again

There is no separate resume function. A resume is the same call, with two values carried over from the previous result:

result = await self.capability_worker.stream_music_from_url(
    url,                                # re-resolve it: signed links expire while paused
    duration_seconds=full_duration,     # unchanged — still the FULL length
    start_seconds=result["position"],   # where the user stopped hearing
    byte_offset=result["byte_offset"])  # where in the file that was

Which makes the natural shape of playback a loop: play, and if it came back "paused", ask the user what they want and either loop or leave.

position, offset = 0.0, 0
while True:
    url = self.stream_url(track)                 # fresh link every pass
    result = await self.capability_worker.stream_music_from_url(
        url,
        duration_seconds=track["duration"],
        start_seconds=position,
        byte_offset=offset,
        announce=f"Playing {track['title']}." if position == 0.0 else "")

    position, offset = result["position"], result["byte_offset"]
    if result["outcome"] != "paused":            # finished/stopped/error/unplayable
        break

    reply = await self.capability_worker.run_io_loop("Paused. Say resume or stop.")
    if "resume" not in (reply or "").lower():
        break

The five rules that live in your code, not in the function

Re-resolve the url on every pass. Signed CDN links expire while the user sits paused; the loop above pays one cheap call to never hit that.
Only "paused" loops. finished, stopped, error and unplayable all leave — a loop that retries on error spins.
An unclear pause reply should stop, not resume. A user who asked for silence must not get the track back because a reply didn’t parse.
announce on the first pass only, or every resume re-announces the title.
Never speak() while a stream is live. Speak before the call or after it returns.

Starting point

The music-template is a complete Ability with the loop above already written and the outcome routing already correct — see Example 4: Music Playback for the full file. Two methods are yours to fill in:

def search_track(self, request)   # {"id", "title", "duration"}, or None if there isn't one
def stream_url(self, track)       # a fresh progressive mp3 link for that track

duration is the FULL track length in seconds, and title is what gets announced before playback. Everything else in that file — the pause menu, the outcome branch, the resume_normal_flow() on exit — can be left alone.

Reach for play_from_audio_file() instead for short audio bundled with your Ability — a chime, a sound effect, a recorded line. stream_music_from_url() is for the minutes-long, interruptible case.

Building Abilities

Integrations

Playing Music from an Ability

`stream_music_from_url()`

During playback

Reading `outcome`

Resume is just calling again

The five rules that live in your code, not in the function

Starting point

​stream_music_from_url()

​During playback

​Reading outcome

​Resume is just calling again

​The five rules that live in your code, not in the function

​Starting point

`stream_music_from_url()`

During playback

Reading `outcome`

Resume is just calling again

The five rules that live in your code, not in the function

Starting point