Home Assistant Voice Integration¶

Give Home Assistant's Assist voice pipeline speech-to-text and text-to-speech backed by Amazon Transcribe and Amazon Polly, through stdapi.ai's OpenAI-compatible audio routes.

At a glance¶

%%{init: {'flowchart': {'htmlLabels': true}} }%%
flowchart LR
  ha["<img src='../styles/logo_home_assistant.svg' style='height:64px;width:auto;vertical-align:middle;' /> Home Assistant<br/>Assist"] -->|Wyoming| proxy["<img src='../styles/logo_wyoming.png' style='height:64px;width:64px;vertical-align:middle;' /> wyoming-openai"]
  proxy -->|OpenAI API| stdapi["<img src='../styles/logo.svg' style='height:64px;width:auto;vertical-align:middle;' /> stdapi.ai"]
  stdapi --> transcribe["<img src='../styles/logo_amazon_transcribe.svg' style='height:64px;width:auto;vertical-align:middle;' /> Amazon Transcribe"]
  stdapi --> polly["<img src='../styles/logo_amazon_polly.svg' style='height:64px;width:auto;vertical-align:middle;' /> Amazon Polly"]

About Home Assistant Assist and Wyoming¶

🔗 Links: Home Assistant Assist | wyoming-openai | Wyoming protocol

Home Assistant's Assist voice pipeline speaks Wyoming, a lightweight protocol for local voice satellites and speech services—not the OpenAI or Anthropic APIs directly. wyoming-openai is an open-source proxy that bridges Wyoming to any OpenAI-compatible speech-to-text and text-to-speech backend, which is what lets Assist reach stdapi.ai.

What the proxy adds on top of the audio routes:

• Wyoming discovery - Advertises configured speech-to-text models and text-to-speech voices to Assist
• Streaming synthesis - Speaks a response as it is generated, in overlapping chunks, rather than waiting for the whole sentence
• Format translation - Reassembles the response as raw PCM frames for Assist's audio pipeline

Connect Your Own Instance¶

Point any Home Assistant instance's Wyoming bridge at stdapi.ai — the deployment underneath doesn't matter to Assist.

Prerequisites¶

Configuration¶

wyoming-openai is configured through environment variables, split into a speech-to-text half and a text-to-speech half. Point both at your stdapi.ai deployment.

# Speech to text
STT_OPENAI_URL=https://YOUR_STDAPI_URL/v1
STT_OPENAI_KEY=YOUR_STDAPI_KEY
STT_MODELS=amazon.transcribe

# Text to speech
TTS_OPENAI_URL=https://YOUR_STDAPI_URL/v1
TTS_OPENAI_KEY=YOUR_STDAPI_KEY
TTS_MODELS=amazon.polly-neural
TTS_VOICES=alloy

# Backend selection
STT_BACKEND=OPENAI
TTS_BACKEND=OPENAI

The proxy calls POST /v1/audio/transcriptions (see Audio Transcriptions API) for speech to text and POST /v1/audio/speech (see Audio Speech API) for text to speech, so STT_MODELS must be a speech-to-text-capable model and TTS_MODELS a text-to-speech-capable model from the correct family.

Streaming Speech to Text¶

Enables: recognizing a spoken command phrase by phrase, instead of after the whole utterance has been recorded.

STT_STREAMING_MODELS=amazon.transcribe

Only the models listed there are called in streaming mode, which is what makes the proxy ask stdapi.ai for a streamed transcription. The gateway returns each phrase as it is recognized whenever the request names the language to expect; if the proxy sends none, set AWS_TRANSCRIBE_STREAM_LANGUAGES on stdapi.ai to the languages your satellites actually speak and those requests take the same fast path. Streamed transcription stages nothing, so it works on a deployment with no S3 bucket configured.

Streaming Text to Speech¶

Enables: speaking a response as it is generated, instead of waiting for the whole sentence to synthesize.

TTS_STREAMING_MODELS=amazon.polly-neural

Naming the same model in both TTS_MODELS and TTS_STREAMING_MODELS puts its voice in the proxy's streaming program, so Assist can use it for both a plain synthesis request and a streamed one. The proxy splits a streamed reply into sentences and synthesizes several /v1/audio/speech calls concurrently, then replays the audio in the original order.

Voice Mapping¶

TTS_VOICES lists OpenAI-style voice names (alloy, echo, fable, and so on); stdapi.ai maps each one to an Amazon Polly voice of matching gender and language. List one entry per voice you want Assist to offer.

Conversation Agent¶

Enables: the middle stage of the Assist pipeline — turning the recognized text into an answer and into device actions.

The speech halves above use the Wyoming bridge; the conversation agent does not need one. Home Assistant's built-in Ollama integration talks to stdapi.ai directly through the Ollama-compatible API.

Home Assistant 2026.4 and later splits this across two dialogs: the first adds the server, the second adds a conversation agent on it as a config subentry. Each numbered list below is one dialog, and the second one only opens once the first has been submitted.

Each conversation agent registers its own conversation.* entity, named after the agent. Select that entity as the conversation agent of your Assist pipeline.

Limits and behaviour to know¶

The proxy speaks the Wyoming protocol over its own TCP port, not HTTP—there is no /health endpoint to check readiness with a plain web request. Wait for a successful Wyoming describe exchange (or check the container logs) rather than polling an HTTP path.

Deploy the Full Stack on AWS¶

The sample below is one worked example of a credible AWS deployment, not the only architecture that works. The gateway is a normal HTTP service, and Home Assistant, wyoming-openai and stdapi.ai itself can run anywhere you like — your own ECS or EKS cluster, EC2, another cloud, or a laptop.

Architecture¶

The diagram below is the topology the Terraform sample builds: a public-facing Home Assistant behind an ALB, wyoming-openai as a sidecar in the same ECS task, and the stdapi.ai gateway as a separate, internally-reachable service in the same VPC.

%%{init: {'flowchart': {'htmlLabels': true, 'nodeSpacing': 20, 'rankSpacing': 40, 'subGraphTitleMargin': {'top': 8, 'bottom': 10}}} }%%
flowchart TB
  user["👤 Household members<br/>(browser · Assist microphone)"]

  subgraph public["Your VPC · public subnets"]
    alb["<img src='../styles/logo_amazon_load_balancing.svg' style='height:40px;width:auto;vertical-align:middle;' /> Application Load Balancer<br/>HTTPS · ACM cert, when a custom domain is set"]
  end

  subgraph private["Your VPC · private app subnets — no inbound route from the internet"]
    ha["<img src='../styles/logo_home_assistant.svg' style='height:40px;width:auto;vertical-align:middle;' /> Home Assistant<br/>ECS Fargate task"]
    wyoming["<img src='../styles/logo_wyoming.png' style='height:40px;width:40px;vertical-align:middle;' /> wyoming-openai<br/>sidecar in the same task"]
    efs["<img src='../styles/logo_amazon_efs.svg' style='height:40px;width:auto;vertical-align:middle;' /> Amazon EFS<br/>.storage · configuration.yaml<br/>encrypted · one task only"]
    rds["Amazon RDS for PostgreSQL<br/>recorder database · Multi-AZ<br/>encrypted"]
    stdapi["<img src='../styles/logo.svg' style='height:40px;width:auto;vertical-align:middle;' /> stdapi.ai<br/>ECS Fargate"]
    egress["<img src='../styles/logo_amazon_vpc.svg' style='height:40px;width:auto;vertical-align:middle;' /> NAT gateways<br/>one per Availability Zone"]
  end

  subgraph regional["AWS service endpoints · your account, the regions you configure"]
    transcribe["<img src='../styles/logo_amazon_transcribe.svg' style='height:40px;width:auto;vertical-align:middle;' /> Amazon Transcribe"]
    polly["<img src='../styles/logo_amazon_polly.svg' style='height:40px;width:auto;vertical-align:middle;' /> Amazon Polly"]
    bedrock["<img src='../styles/logo_amazon_bedrock.svg' style='height:40px;width:auto;vertical-align:middle;' /> Amazon Bedrock"]
    s3["<img src='../styles/logo_amazon_s3.svg' style='height:40px;width:auto;vertical-align:middle;' /> Amazon S3<br/>SSE-KMS"]
    cw["<img src='../styles/logo_amazon_cloudwatch.svg' style='height:40px;width:auto;vertical-align:middle;' /> Amazon CloudWatch<br/>container logs"]
    transcribe ~~~ s3
    polly ~~~ cw
    bedrock ~~~ transcribe
  end

  user -->|"HTTPS · TLS 1.2+"| alb
  alb -->|"HTTP · private subnet"| ha
  ha -->|"Wyoming · TCP<br/>localhost, same task"| wyoming
  wyoming -->|"OpenAI API · API key<br/>private DNS, no public endpoint"| stdapi
  ha --> efs
  ha -->|"PostgreSQL · TLS, verify-full"| rds
  ha -->|"HTTPS · SigV4<br/>seeds configuration.yaml, first boot only"| s3
  stdapi --> egress
  egress -->|"HTTPS · SigV4"| transcribe
  egress -->|"HTTPS · SigV4"| polly
  egress -->|"HTTPS · SigV4"| bedrock
  egress -->|"HTTPS · SigV4<br/>stages audio for non-streaming transcription"| s3
  egress --> cw

The ALB is the only public address in the picture, and it forwards only to Home Assistant — stdapi.ai has no listener of its own and is reached exclusively through AWS Cloud Map private DNS from the wyoming-openai sidecar. A household's .storage and configuration.yaml come to rest on the single EFS volume mounted into the Home Assistant task, and the recorder history (state and long-term statistics) in a Multi-AZ Amazon RDS for PostgreSQL instance — neither on the gateway; the gateway itself is stateless and only its egress path crosses the VPC boundary, over HTTPS with SigV4, to Amazon Transcribe, Amazon Polly and Amazon Bedrock.

What Each AWS Service Does Here¶

Security Measures in This Flow¶

• Authentication — wyoming-openai calls the gateway with a stdapi.ai API key that Terraform generates (api_key_create = true) and injects as STT_OPENAI_KEY/TTS_OPENAI_KEY container secrets; the sample's ALB security group additionally restricts inbound traffic to the deploying operator's own IP address.
• Encryption in transit — HTTPS from the browser to the ALB when a custom domain and certificate are configured; Wyoming stays inside the ECS task over localhost; HTTPS with SigV4 from the gateway to Amazon Transcribe, Amazon Polly and Amazon Bedrock; PostgreSQL over TLS (sslmode=verify-full) from Home Assistant to the Amazon RDS instance.
• Encryption at rest — the EFS volume backing Home Assistant's state, the Amazon RDS instance, and both S3 buckets (config seed, gateway staging) use customer-managed KMS keys.
• Least privilege — the gateway's task role is scoped to the AWS services the module fronts, of which this flow exercises the Transcribe, Polly and Bedrock actions; Home Assistant's task role carries none of them.
• Content policy — a Bedrock guardrail, if configured on the gateway, checks the text to synthesize as INPUT on /v1/audio/speech and the produced transcript as OUTPUT on /v1/audio/transcriptions, through the ApplyGuardrail API rather than a native chat-style integration.
• Data handling — the gateway holds request audio in memory, or briefly in its own S3 bucket when staging a non-streaming transcription job, and does not persist it; Home Assistant's own .storage and configuration.yaml stay on the EFS volume in your account, with its recorder history and long-term statistics in the Amazon RDS instance instead.

What's Included¶

Deploy Home Assistant, wyoming-openai, and stdapi.ai together on ECS Fargate:

📦 stdapi-ai/samples/getting_started_home_assistant

What's included:

• Home Assistant and wyoming-openai in the same ECS Fargate task, talking over localhost
• stdapi.ai gateway connected to Amazon Bedrock, Amazon Transcribe, and Amazon Polly
• configuration.yaml seeded on first boot with the reverse-proxy trust settings Home Assistant needs behind an ALB
• Both container images pulled directly and anonymously from ghcr.io — no local build, no registry credential
• HTTPS-capable ALB on your own domain (needed for microphone access in the browser)

Deploy:

git clone https://github.com/stdapi-ai/samples.git
cd samples/getting_started_home_assistant/terraform
tofu init
tofu apply

Nothing is left manual after tofu apply: a bootstrap container in the same ECS task drives Home Assistant's own APIs to create the owner account, finish onboarding, add the Wyoming integration, add the Ollama conversation agent, and make an Assist pipeline bound to all three the preferred one. Sign in with the credentials Terraform prints and the assistant answers. The conversation model is a Terraform variable, so the two dialogs above are what you follow only when you configure Home Assistant yourself. See the sample's README for the details.

What It Costs to Run¶

Read a model's price before you send anything to it with GET /model_pricing. Setting COST_TRACKING=true additionally puts a per-request cost on each usage entry — estimated from published AWS prices, not read back from your invoice.

What to Watch¶

The gateway logs Polly usage — input_characters, always on the request event — and Transcribe usage — input_seconds, on request normally or on request_stream if you turn on STT_STREAMING_MODELS — with execution_time_ms on every entry. Turning on CLOUDWATCH_METRICS republishes those counts as EMF metrics in the stdapi namespace, dimensioned by Model: Count for characters, Seconds for audio duration.

fields path, execution_time_ms
| filter type = "request" and (path = "/v1/audio/transcriptions" or path = "/v1/audio/speech")
| stats count(*) as calls, avg(execution_time_ms) as avg_ms, pct(execution_time_ms, 95) as p95_ms by path
| sort path

A rising p95 on either path is what a household notices as a slow turn, before it shows up in any cost report.

Next Steps¶