Voice API

Overview

The Voice API provides text-to-speech (TTS) and speech-to-text (STT) capabilities with multiple backend providers. You can synthesize speech from text, transcribe audio files, manage voice models, and integrate voice directly into agent conversations.

Supported Backends

---

Text-to-Speech (TTS)

Synthesize Speech (JSON)

Returns synthesized audio as a base64-encoded string in a JSON response.

POST /api/v1/voice/synthesize

Request Body:

bashCopy
curl -X POST /api/v1/voice/synthesize \
  -H "Authorization: Bearer <your_login_jwt>" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Hello, welcome to Arcanflows.",
    "voice": "en-US-AriaNeural",
    "backend": "edge-tts",
    "format": "mp3"
  }'

Response:

jsonCopy
{
  "audio": "base64_encoded_audio_data...",
  "format": "mp3",
  "duration_ms": 2340,
  "backend": "edge-tts",
  "voice": "en-US-AriaNeural",
  "cached": false
}

Synthesize Speech (Audio File)

Returns the audio file directly as a binary response.

POST /api/v1/voice/synthesize/audio

Same request body as /synthesize. Response is the raw audio file with appropriate Content-Type header (e.g., audio/mpeg).

Routed Synthesis

Auto-selects the best available backend based on voice, cost, and quality settings.

POST /api/v1/voice/synthesize/routed
POST /api/v1/voice/synthesize/audio/routed

Same request body but omit the backend field. The router selects the optimal backend automatically.

---

Speech-to-Text (STT)

Transcribe Audio

POST /api/v1/voice/transcribe

Request: multipart/form-data

bashCopy
curl -X POST /api/v1/voice/transcribe \
  -H "Authorization: Bearer <your_login_jwt>" \
  -F "[email protected]" \
  -F "language=en"

Response:

jsonCopy
{
  "text": "Hello, I need help with my account.",
  "language": "en",
  "duration_ms": 4200,
  "backend": "whisper",
  "confidence": 0.95
}

Routed Transcription

POST /api/v1/voice/transcribe/routed

Auto-selects the best STT backend.

---

Agent Voice Integration

Send Audio to Agent

Send an audio message to an agent conversation. The audio is transcribed and the agent responds as text.

POST /api/v1/agents/:id/chat/audio

Request: multipart/form-data

bashCopy
curl -X POST /api/v1/agents/agent_abc123/chat/audio \
  -H "Authorization: Bearer <your_login_jwt>" \
  -F "[email protected]" \
  -F "conversation_id=conv_xyz789"

Response:

jsonCopy
{
  "conversation_id": "conv_xyz789",
  "transcription": "What is my account balance?",
  "response": "Your current account balance is $1,234.56.",
  "message_id": "msg_abc123"
}

Get TTS for Agent Response

Convert an agent's text response to speech.

POST /api/v1/agents/:id/chat/tts

Request Body:

Response: Audio file binary with Content-Type: audio/mpeg.

---

Voices and Backends

List Available Voices

GET /api/v1/voice/voices

Query Parameters:

bashCopy
curl /api/v1/voice/voices?backend=edge-tts&language=en \
  -H "Authorization: Bearer <your_login_jwt>"

Response:

jsonCopy
{
  "voices": [
    {
      "id": "en-US-AriaNeural",
      "name": "Aria",
      "language": "en-US",
      "gender": "female",
      "backend": "edge-tts",
      "preview_url": "/api/v1/voice/voices/en-US-AriaNeural/preview"
    },
    {
      "id": "en-US-GuyNeural",
      "name": "Guy",
      "language": "en-US",
      "gender": "male",
      "backend": "edge-tts",
      "preview_url": "/api/v1/voice/voices/en-US-GuyNeural/preview"
    }
  ],
  "total": 2
}

List Voice Backends

GET /api/v1/voice/backends

Returns all configured voice backends and their status.

Voice Service Health

GET /api/v1/voice/health

Returns the health and availability of each voice backend.

---

TTS Model Management

For self-hosted TTS backends, you can manage installed models.

List All TTS Models

GET /api/v1/voice/tts/models

Returns all available TTS models (installed and remote).

List Installed Models

GET /api/v1/voice/tts/models/installed

Returns only locally installed TTS models.

List Recommended Models

GET /api/v1/voice/tts/models/recommended

Returns a curated list of recommended models by quality and language.

Download Model

POST /api/v1/voice/tts/models/download

Request Body:

jsonCopy
{
  "model_name": "tts-piper-en-us-amy-medium"
}

Load Model

POST /api/v1/voice/tts/models/load

Request Body:

jsonCopy
{
  "model_name": "tts-piper-en-us-amy-medium"
}

Delete Model

DELETE /api/v1/voice/tts/models/:model_name

Removes an installed model from disk.

---

Cache Management

Invalidate Cache

Clear cached TTS audio files.

POST /api/v1/voice/cache/invalidate

Request Body:

bashCopy
curl -X POST /api/v1/voice/cache/invalidate \
  -H "Authorization: Bearer <your_login_jwt>" \
  -H "Content-Type: application/json" \
  -d '{"backend": "edge-tts", "all": false}'

---

Error Codes