Adopt & Extend
This accelerator is meant to be taken apart. Use it as a running reference, a library of Azure integrations, or a full starting point for your own voice agent — pick the level of commitment that fits your project.
Framing: everything below assumes you are not the maintainer of this repo. You're evaluating how much of it to bring into a project of your own, and how to keep your fork or your extracted pieces upgradeable as the accelerator evolves.
Three Ways to Adopt
There is no single "right" way to consume this accelerator — pick based on how much of the real-time voice stack you actually need to own.
| Mode | What you keep | What you own | Best for |
|---|---|---|---|
| Fork & run | Everything — apps/, src/, infra/ |
Agent YAML, scenarios, tools, branding, deployment config | Teams whose product is a voice agent and want the full pipeline (telephony, orchestrators, UI) day one. |
| Library reuse | src/ only (ACS, Speech, AOAI, Redis, Cosmos, pools, VAD) |
Your own FastAPI app, your own orchestrator loop | Teams with an existing backend who just want production-grade Azure integrations without rebuilding client pooling, retry, and auth from scratch. |
| Pattern reuse | Nothing — no code dependency | A re-implementation of the same shape: YAML-declared agents, a centralized tool registry, scenario-based handoff routing | Teams on a different stack (Node, .NET, a different cloud) who want the architecture, not the Python. |
Which One Do I Want?
real-time voice agent?"} q2{"Do you already have a
backend / orchestration loop?"} q3{"Are you on Python + FastAPI?"} fork["Fork & run
Full accelerator"] lib["Library reuse
Import src/ modules"] pat["Pattern reuse
Re-implement the shape"] q1 -- "Yes, it's the whole product" --> fork q1 -- "No, it's one feature
of a bigger app" --> q2 q2 -- "No" --> fork q2 -- "Yes" --> q3 q3 -- "Yes" --> lib q3 -- "No" --> pat classDef qNode fill:#F0F7FF,stroke:#0078D4,color:#0A2540 classDef aNode fill:#E6FAEA,stroke:#107C10,color:#0B3D0B class q1,q2,q3 qNode class fork,lib,pat aNode
Most teams start with "Fork & run" and prune. It's easier to delete a module (e.g. Genesys support, or the React demo frontend) you don't need than to bolt on connection pooling, barge-in handling, and MemoManager session recovery later under deadline pressure. Trim apps/artagent/backend/voice/genesys/ or the entire apps/artagent/frontend/ if they're not relevant to your product — both are cleanly separable.
Extension Points Matrix
Choosing telephony? Read Telephony Options before swapping a channel. Teams Phone via TPE retains ACS; Graph and SIP-gateway alternatives require new control/media integrations. The guide separates the public code baseline, proposed integration designs, and publicly documented product previews.
Whichever mode you pick, these are the seams the codebase was designed to be cut along. Each row is a real interface or factory you can swap without touching the orchestrators.
| Concern | Where it lives | How to swap it |
|---|---|---|
| Telephony / channel | apps/artagent/backend/voice/{genesys,handler.py} |
ACS and Genesys are two independent WebSocket bridges that both call into the same LiveOrchestrator / SpeechCascade core. See Add a Channel for the contract a third bridge (e.g. Twilio, Vonage) needs to satisfy. |
| Speech-to-text / text-to-speech | src/speech/speech_recognizer.py, src/speech/text_to_speech.py |
Only used by the SpeechCascade orchestrator (VoiceLive bundles STT/TTS itself). Both are thin wrappers around the Azure Speech SDK behind pooled client managers — replace the SDK calls, keep the pooling/lifecycle contract. See Swap a Provider. |
| LLM provider | src/aoai/client_manager.py (AoaiClientManager) |
Accepts an injectable factory / async_factory callable at construction — pass a factory that returns any OpenAI-compatible client (Azure OpenAI, OpenAI, or a self-hosted gateway) instead of replacing call sites. |
| Orchestration mode | apps/artagent/backend/voice/{speech_cascade,voicelive}/ |
Selected per-call by ACS_STREAMING_MODE. Add a third mode by implementing the same handler contract (on_audio, on_control, session lifecycle) and registering it alongside the existing two. |
| Session state | src/redis/manager.py (MemoManager) |
Backed by Redis for sub-millisecond reads during a live call. Swap the backing store by re-implementing MemoManager's get/set/mirror surface — every consumer (orchestrators, agent-builder, handoffs) goes through this one class. |
| Persistent storage | src/cosmosdb/manager.py |
Used for transcripts, post-call analytics, and audit records once a call ends. Swap for any document store by matching the manager's write/query surface — nothing upstream depends on Cosmos-specific query syntax. |
| Tools / business logic | apps/artagent/backend/registries/toolstore/registry.py |
Anything decorated with @register_tool becomes callable by any agent. This is the seam for connecting your own CRM, ticketing system, or internal APIs — see Add a Tool. |
| Agent behavior & routing | registries/agentstore/*/agent.yaml, registries/scenariostore/*/orchestration.yaml |
No code changes required — new agents and handoff graphs are pure YAML. See Create an Agent and Create a Scenario. |
| Observability | utils/telemetry_config.py, utils/ml_logging.py |
OpenTelemetry-based; point the exporter at any OTLP-compatible backend instead of Application Insights without touching instrumented call sites. |
Principles Worth Stealing
Even in "pattern reuse" mode with zero shared code, these design choices (detailed with their full rationale in Architecture → Key Design Decisions) are the ones worth carrying into your own stack:
- Declare agents in config, not code. A YAML file with a system-prompt template and a tool list is reviewable by non-engineers and hot-swappable per session — you don't need a Python subclass per persona.
- Separate "what an agent can do" from "how agents connect." Tools and agents are reusable primitives; a scenario is just a routing graph over them. The same agent behaves differently in different scenarios without duplicating its definition.
- Centralize side-effecting capabilities behind one registry. Every tool goes through one decorator and one lookup path — no tool is ever defined inline inside an agent, which keeps discovery and auditing tractable as the tool count grows.
- Pool expensive clients; never construct them per request. STT, TTS, and LLM clients cost 100-500ms to stand up. In a system with a 1-second turn budget, that's not optional — it's the difference between a usable and unusable product.
- Keep session state external and cheap to read. Conversation state lives in Redis, not in-process globals, so any container can pick up any call — this is what makes horizontal scaling and worker restarts safe mid-conversation.
- Own the orchestration loop; don't hide it behind a generic agent-framework abstraction. When every millisecond of a turn matters, you need to see and control each stage (VAD → STT → LLM → TTS → playback) directly. See Architecture → Why this complexity? for the full argument.
Minimal-Footprint Checklist
If you're bringing only a slice of this accelerator into an existing project, this is the shortest path that stays coherent:
| Step | Do this |
|---|---|
| 1 | Decide your adoption mode (above) before copying any code — it determines whether you take apps/, just src/, or nothing. |
| 2 | If reusing src/: bring the whole module you need (e.g. src/aoai/) rather than individual files — the pooling, retry, and auth-refresh logic is split across a few cooperating classes. |
| 3 | Keep the tool-registry pattern even if you drop everything else — it's the cheapest single change that keeps your agent's capabilities auditable as they grow. |
| 4 | Budget your own turn latency the same way: measure STT-start → LLM-first-token → TTS-first-chunk separately, and pool anything that crosses a network boundary. |
| 5 | Read Key Design Decisions and the Production Deployment checklist before you assume a dev-mode config is safe to ship. |
Next: once you know what you're taking, Swap a Provider and Add a Channel in the Guides walk through the two most common customizations step by step.