Skip to main content
The architecture starts with Companion Dashboards — one sentence: your Ability POSTs to a Flask server on Replit, which keeps the latest state in memory and serves it to a polling frontend. This page is the full build guide.
The Ability is the truth. The dashboard is a mirror. Everything between is fire-and-forget JSON.

The four rules

Every production-grade OpenHome + Replit system obeys these. Break any of them and you’ll eventually find yourself debugging at 2 AM.

1. Fire and forget, always

Your Ability’s main loop cadence is sacred. A slow dashboard POST must never block your next Gemini call, Deepgram transcription, or audio capture. Wrap every outbound request in a session task.
def post(self, endpoint, payload):
    async def _send():
        try:
            await asyncio.to_thread(
                requests.post,
                f"{DASHBOARD_URL}/{endpoint}",
                json=payload,
                timeout=5,
            )
        except Exception:
            pass  # Dashboard failures must never break the ability
    self.worker.session_tasks.create(_send())
Three things worth noting:
  • session_tasks.create is the OpenHome-sanctioned way to kick off a background coroutine — raw asyncio.create_task is blocked in the sandbox
  • asyncio.to_thread runs synchronous requests.post off the event loop, so your main await cadence stays clean
  • The empty except is on purpose. Dashboard problems are not Ability problems.

2. Send full state snapshots, not diffs

Replit’s free tier restarts. Browser tabs refresh. Networks drop. Every POST should contain enough context that if the frontend missed the last ten updates, the one it just received is still useful on its own. Include the session ID, elapsed time, full speaker registry, latest tone, recent log entries. Bandwidth is cheap. Debugging state drift is not.

3. One endpoint per event type, not per field

Group updates by cadence and meaning. A typical ambient Ability has four or five endpoints — not forty.
EndpointCadencePurpose
/api/<ability>/session_startOnceLifecycle
/api/<ability>/updateEvery 10–30sFast cycle
/api/<ability>/deepEvery 60s or on insightSlow cycle
/api/<ability>/heartbeatEvery 10s when idleKeepalive
/api/<ability>/session_endOnceFinal summary
This maps cleanly to tabs or cards on your dashboard. Resist the urge to make a new endpoint every time you add a new field to the payload — add the field to the existing snapshot and let the frontend decide what to render.

4. Heartbeat when idle

If your Ability goes quiet — no speakers in the room, no new readings — the frontend has no way to tell whether it’s alive, crashed, or waiting. Send a heartbeat every 10 seconds with session_uptime_seconds and whatever lightweight counter makes sense. The UI can then confidently show “online, listening” instead of a blank card that looks broken.

The payload shape that works

Every POST opens with the same four keys. After that, nest whatever is specific to this event. Do not flatten. The frontend is much happier reading payload.tone.voices[0].emotion than payload_tone_voice_0_emotion.
{
    "session_id": "session-1772046000",    // Unique per ability invocation
    "timestamp": 1772046010.5,              // Unix timestamp, for ordering
    "elapsed": "00:10",                     // MM:SS, for human display
    "cycle_id": 1,                          // Monotonic counter, for dedup

    // Everything below is event-specific
    "tone": { "...": "..." },
    "scene": { "...": "..." },
    "speaker_registry": { "...": "..." },
    "running_log": ["..."]                  // Truncate on the way out
}
Truncate before you POST. Your Ability might hold 50 log entries in memory, but the dashboard only needs the last 15. Slicing at the source saves bandwidth, keeps payloads under a couple hundred KB, and makes the frontend’s job trivial.

Copy-paste snippets

Drop these directly into your Ability file. They compose cleanly with the OpenHome SDK and follow every sandbox rule.

1. Config block — top of main.py

# =============================================================================
# DASHBOARD CONFIG
# =============================================================================

DASHBOARD_URL = "https://your-repl-name.replit.app/api/<ability>"
SESSION_ID_PREFIX = "session"
HEARTBEAT_INTERVAL = 10     # seconds, when idle
POST_TIMEOUT_FAST = 5       # seconds, for tight-loop posts
POST_TIMEOUT_SLOW = 15      # seconds, for session summaries

2. Universal fire-and-forget POST helper

def post(self, endpoint: str, payload: dict, timeout: int = POST_TIMEOUT_FAST):
    """Fire-and-forget POST to the dashboard. Never blocks the main loop."""
    async def _send():
        try:
            resp = await asyncio.to_thread(
                requests.post,
                f"{DASHBOARD_URL}/{endpoint}",
                json=payload,
                timeout=timeout,
            )
            if resp.status_code != 200:
                self.log_error(f"[DASH] {endpoint}: {resp.status_code}")
        except Exception as e:
            self.log_error(f"[DASH] {endpoint} failed: {e}")
    self.worker.session_tasks.create(_send())

3. Session lifecycle bookends

# At the start of run():
self.session_id = f"{SESSION_ID_PREFIX}-{int(time.time())}"
self.session_start = time.time()
self.post("session_start", {
    "session_id": self.session_id,
    "timestamp": time.time(),
    "ability_version": "1.0.0",
})

# At the end of run() (in the finally block):
self.post("session_end", {
    "session_id": self.session_id,
    "timestamp": time.time(),
    "duration_seconds": time.time() - self.session_start,
    "final_state": self.serialize_state(),
}, timeout=POST_TIMEOUT_SLOW)

4. Heartbeat loop

async def heartbeat_loop(self):
    """Ping the dashboard every 10s so the frontend knows we're alive."""
    while self.is_running:
        self.post("heartbeat", {
            "session_id": self.session_id,
            "timestamp": time.time(),
            "uptime_seconds": time.time() - self.session_start,
            "cycle_id": self.cycle_id,
        })
        await self.worker.session_tasks.sleep(HEARTBEAT_INTERVAL)

# In run(), launch it alongside your main loop:
self.worker.session_tasks.create(self.heartbeat_loop())

The Replit backend — minimal

Eighty lines of Flask handle any ambient Ability you can throw at it. Save this as main.py on your Repl and hit Run.
from flask import Flask, request, jsonify, send_from_directory
from flask_cors import CORS
from collections import deque
import time

app = Flask(__name__)
CORS(app)  # Accept from anywhere — the Ability sandbox has no fixed IP

STATE = {
    "session_id": None,
    "last_update": None,
    "last_event": None,
    "tone": {},
    "scene": {},
    "speaker_registry": {},
    "running_log": [],
    "deep_insight": {},
}
HISTORY = deque(maxlen=100)  # Last 100 cycle snapshots

@app.route("/api/<ability>/<event>", methods=["POST"])
def ingest(ability, event):
    payload = request.get_json(silent=True) or {}
    print(f"[{ability}/{event}] {len(str(payload))} bytes from {payload.get('session_id','?')}")

    STATE["last_update"] = time.time()
    STATE["last_event"] = event
    STATE["session_id"] = payload.get("session_id", STATE["session_id"])

    # Merge nested objects; append to lists
    for key, val in payload.items():
        if isinstance(val, dict) and key in STATE and isinstance(STATE[key], dict):
            STATE[key].update(val)
        elif isinstance(val, list) and key in STATE and isinstance(STATE[key], list):
            STATE[key] = val[-50:]  # Keep last 50
        else:
            STATE[key] = val

    if event == "update":
        HISTORY.append(payload)

    return jsonify({"ok": True})

@app.route("/api/state")
def get_state():
    return jsonify(STATE)

@app.route("/api/history")
def get_history():
    return jsonify(list(HISTORY))

@app.route("/")
def index():
    return send_from_directory(".", "index.html")

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)
A few things about this scaffold are deliberately minimal. No auth, no database, no persistence across restarts. If the Repl wakes up cold, STATE is an empty dict and HISTORY is an empty deque. That’s almost always what you want for an ambient dashboard — the Ability is the source of truth, and on reconnect it floods the backend with fresh state within one cycle. Don’t add Postgres until you have a reason to.

The two gotchas that cause 90% of silent failures

If POSTs aren’t arriving, check exactly two things first:
  1. CORS must be permissive. CORS(app) with no arguments is correct for development.
  2. Flask must bind to 0.0.0.0, not 127.0.0.1, or Replit’s proxy can’t reach it.

Coding best practices

Polling interval: 1–2s for live, 5s for heavy

Your Ability’s cadence is the ceiling. Polling faster than you POST just wastes bandwidth. For a 10s fast-loop Ability, a 2s frontend poll gives five chances per cycle to catch the update.

Session IDs on every POST

Generate a session ID at startup — f"session-{int(time.time())}" is fine — and stamp every POST with it. The frontend can then detect restarts and reset its view without confusing old and new data.

Log POST status on the Ability side

You want [DASH] update: 200 in your logs so you know the pipe is open. But only log errors for heartbeats — otherwise, at one heartbeat every ten seconds, logs drown in noise within half an hour.

Never send raw audio

Keep audio local to the Ability. Send transcripts, summaries, tone descriptors, metadata — never raw PCM or WAV bytes.
  • 1 minute of 16kHz 16-bit audio = 2 megabytes
  • 1 minute of transcript = 2 kilobytes
The ratio speaks for itself.

Truncate on the way out

If your Ability holds 50 log entries but the dashboard only shows 15, slice at the source: self.running_log[-15:]. Do this for every list you POST. Your bandwidth, Replit free tier, and frontend rendering will all thank you.

Timeouts

  • 5s for frequent (heartbeats, cycle updates)
  • 10s for chunky (insights with analysis)
  • 15–30s for rare (session summaries with full transcripts)
Anything longer and you should be questioning why it’s on the dashboard at all — move it to a persistent store and link out.

One DASHBOARD_URL constant, not N hardcoded URLs

Define DASHBOARD_URL once at the top. Build every endpoint path from it. When you migrate from staging to production, or from replit.app to a custom domain, you change exactly one line.

The fire-and-forget test

If your Replit is turned off completely, your Ability should run indefinitely without slowing down, crashing, or logging more than a trickle of connection errors. That’s the test. If the Ability degrades when the dashboard is down, your POSTs aren’t truly fire-and-forget.

Design philosophy

Companion dashboards look different from traditional web apps. They’re less application, more instrument panel — something you glance at to confirm that the ambient system you can’t see is working, and what it’s thinking.

Voice-first, screen-supporting — not the other way around

The screen should never become the primary interface. If users start reaching for the mouse, the Ability has failed. The dashboard exists to confirm, visualize, and occasionally archive — not to drive.

Always render the current state, not a form to modify it

Companion dashboards are read-only by default. If you need to change behavior, change it by talking to the Ability. The dashboard is a window into the system’s mind; not a control panel.

Visible liveness, always

Every screen should make it obvious whether the Ability is online. A pulsing dot. An elapsed timer. A last-update timestamp. If users can’t tell at a glance, they’ll doubt everything the dashboard says.

Low-chrome, high-signal

Strip the app-shell cruft. No navbars with ten links, no footers, no about pages. The dashboard should look like a cockpit, not a website. Dark themes work well here — they signal seriousness and make transient data easier to scan.

Design for the glance

  • The most important number on the screen should be readable from across the room
  • The second-most-important should be readable from the desk
  • Everything else is detail on demand
If a user has to lean in to see whether the Ability is listening, you’ve failed the glance test.

Build for your own use first

The fastest path to a good dashboard is to build the one you personally want to leave open on a second monitor. If you don’t want to look at it, nobody will. Polish comes from daily use, not design reviews.

Go build something ambient

The best Abilities aren’t the ones that answer questions. They’re the ones that notice things — a partner’s frustration rising in a conversation, a meeting that drifted off-topic seven minutes ago, an air-quality reading that crossed a threshold three hours ago and never recovered. These are the signals ambient intelligence was built to surface. The dashboard is how you make them visible without stealing attention from the real world.

See also