BYOK (Bring Your Own Key) System

Enables customers to use their own API keys for LLM, STT, and TTS providers with per-key concurrency limits and shared key set enforcement across campaigns.

---

Architecture

---

Data Model

ByokKey Collection

Field	Type	Description
keyId	String (UUID)	Unique identifier
userUID	String	Owner customer
orgUID	String	Organization
keyName	String	Display name (max 100 chars)
providerCategory	"LLM" | "STT" | "TTS"	Key category
providerName	String	Provider (openai, deepgram, etc.)
encryptedValue	String	AES-256-GCM ciphertext (base64)
iv	String	12-byte IV (base64)
authTag	String	GCM auth tag (base64)
scope	"global" | string[]	Which agents can use this key
expiresAt	Date | null	Optional expiry
status	"active" | "expired"	Current status
lastUsedAt	Date | null	Last dispatch usage
concurrencyLimit	Number	Max concurrent calls (1–500, required, integer)

Indexes: {userUID, keyName} unique, {userUID, providerCategory}, {status, expiresAt}

Agent byokConfig Sub-Schema

byokConfig: {
  llm: { enabled: Boolean, keyId: String | null, model: String | null },
  stt: { enabled: Boolean, keyId: String | null, model: String | null },
  tts: { enabled: Boolean, keyId: String | null, model: String | null },
  enableCustomConcurrency: Boolean,       // default: false
  customConcurrency: Number | null,       // 1–500, integer
  concurrencyMismatchFlag: Boolean,       // default: false
}

Redis Key Layout

Key	Type	Purpose
byok:{hash}:active_calls	String (int)	Active call counter for key set
byok:{hash}:lock	String	Distributed lock (10s TTL)
byok:{hash}:campaigns	Set	Campaign UIDs using this key set
byok:{hash}:expiry_detected	String	Timestamp of first expiry detection

Key Set Hash = SHA-256(sorted([llmKeyId, sttKeyId, ttsKeyId]).join("|")).substring(0, 16)

---

Concurrency Layers (Most Restrictive Wins)

1. Provider Capacity Platform telephony limits
2. Customer Plan Plan-based call limits
3. Campaign maxConcurrent Per-campaign setting
4. BYOK Key Set customConcurrency ≤ min(llm.concurrencyLimit, stt.concurrencyLimit, tts.concurrencyLimit)

---

API Endpoints

POST /api/byok/keys Create Key

{
  "keyName": "My OpenAI Key",
  "keyValue": "sk-proj-abc123...",
  "providerCategory": "LLM",
  "providerName": "openai",
  "concurrencyLimit": 50,
  "scope": "global",
  "expiresAt": "2026-12-31T23:59:59Z"
}

PUT /api/byok/keys/:keyId Update Key

{
  "keyName": "Updated Name",
  "concurrencyLimit": 100
}

DELETE /api/byok/keys/:keyId Delete Key

Returns { affectedAgents: ["Agent Name"] } on success.

GET /api/byok/keys List Keys (masked values)

GET /api/byok/keys/for-agent/:agentId Keys in scope for agent

PUT /api/v2/agents/:id Agent with Custom Concurrency

{
  "byokConfig": {
    "llm": { "enabled": true, "keyId": "...", "model": "gpt-4o" },
    "stt": { "enabled": true, "keyId": "...", "model": "nova-2" },
    "tts": { "enabled": true, "keyId": "...", "model": "sonic" },
    "enableCustomConcurrency": true,
    "customConcurrency": 25
  }
}

---

Error Codes

Code	HTTP	When
INVALID_CONCURRENCY_LIMIT	400	concurrencyLimit not integer in [1,500]
KEY_IN_ACTIVE_USE	409	Updating concurrencyLimit while campaign running
KEY_DELETION_BLOCKED	409	Deleting key while campaign running
INCOMPLETE_BYOK_COVERAGE	400	Custom concurrency needs all 3 BYOK keys
CONCURRENCY_EXCEEDS_KEY_LIMIT	400	customConcurrency > min key limit
BYOK_KEY_SET_CONFLICT	409	Key set used by another agent with active campaign
BYOK_CONFIG_LOCKED	409	Modifying BYOK config during active campaign
CONCURRENCY_MISMATCH	422	Mismatch flag set or value exceeds limits at campaign start
BYOK_KEYS_INVALID	422	Keys missing or expired at campaign start
BYOK_SLOTS_UNAVAILABLE	409	Lock contention after retries

---

Key Flows

Key Creation & Validation

1. Customer submits key name, value, provider, category, concurrencyLimit
2. Backend validates fields (name length, provider exists, concurrencyLimit 1–500)
3. Validates key against provider API (10s timeout)
4. Encrypts value with AES-256-GCM (random 12-byte IV)
5. Stores in MongoDB, creates audit log entry

Agent Custom Concurrency Setup

1. Customer enables BYOK for all 3 categories and assigns keys
2. Enables "Custom Concurrency" toggle, sets value (e.g., 25)
3. Backend validates:
   • All 3 keys exist and are assigned → else INCOMPLETE_BYOK_COVERAGE
   • customConcurrency ≤ min(key limits) → else CONCURRENCY_EXCEEDS_KEY_LIMIT
   • No conflict with other agents on same key set → else BYOK_KEY_SET_CONFLICT
4. Computes and stores keySetHash on the agent

Campaign Start

1. Validates all 3 BYOK keys exist and are active
2. Checks concurrencyMismatchFlag → blocks if set
3. Validates customConcurrency ≤ current min key limit
4. Acquires distributed lock (SET NX, 10s TTL, 3 retries with backoff)
5. Checks available capacity, adds campaign to Redis set
6. Campaign starts dispatching

Dispatch (fillCallSlots)

1. Computes slotsAvailable from campaign/plan limits
2. Checks BYOK: canDispatch(keySetHash, customConcurrency)
3. If at capacity → slotsAvailable = 0
4. Otherwise → slotsAvailable = min(slotsAvailable, customConcurrency - activeCount)
5. For each call dispatched: INCR counter before, DECR on failure (rollback)
6. On Redis failure: falls back to next concurrency layer (logs warning)
7. resolveAndMergeByok merges ALL BYOK data (provider/model overrides + decrypted API keys) directly into agent_config
8. Agent backend extracts everything from agent_config no top-level BYOK fields in dispatch metadata

Call End (Webhook)

1. On participant_left event, fetches agent
2. If custom concurrency enabled → DECR counter (Lua script, floor at 0)

Campaign Stop

1. Fetches agent, computes keySetHash
2. Calls releaseReservation → removes campaign from Redis set
3. If no campaigns remain → sets 24h TTL on Redis keys

Reconciliation (Every 60s)

1. Scans Redis for byok:*:campaigns keys
2. For each key set: verifies campaigns still running, removes orphans
3. Counts actual active calls, corrects counter drift
4. Also runs once on server startup

---

File Reference

File	Purpose
src/services/byok-concurrency.service.ts	Core: hash, counters, conflict detection, reservation, reconciliation
src/services/byok-reconciliation.job.ts	Periodic reconciliation (60s + startup)
src/services/byok.service.ts	Key CRUD, validation, mismatch flag, deletion guard
src/services/byok-validator.service.ts	Provider API key validation
src/services/encryption.service.ts	AES-256-GCM encrypt/decrypt
src/services/calling.service.ts	Campaign start validation, fillCallSlots, reservation
src/controllers/byok.controller.ts	HTTP handlers for key CRUD
src/controllers/agent.controller.ts	Custom concurrency validation, config lock, auto-disable
src/controllers/webhook.controller.ts	Counter decrement on call end
src/models/ByokKey.ts	Mongoose schema with concurrencyLimit
src/models/Agent.ts	byokConfig with concurrency fields
src/utils/byok-dispatch.ts	Dispatch-time key resolution
src/utils/byok-masking.ts	Key value masking for display
src/repositories/byok.repository.ts	Data access layer
src/routes/byok.routes.ts	Route definitions (JWT + feature gate)
src/scripts/byok-expiry-check.ts	Daily cron for key expiry
src/server.ts	Starts/stops reconciliation job

---

Security

• Encryption at rest AES-256-GCM with unique 12-byte IV per key
• Encryption in transit HTTPS + LiveKit TLS
• No logging Decrypted keys never logged
• Memory-only Plaintext exists only during encrypt/decrypt and in Agent process
• Tamper detection GCM auth tag
• Feature-gated Requires "byok" feature flag
• Ownership checks All operations verify userUID

Required env var: WEBHOOK_SECRET_ENCRYPTION_KEY (64 hex chars = 32 bytes)

---

Mismatch Flag Lifecycle

Event	Flag State
Key's concurrencyLimit updated → agent's customConcurrency > new min	Set true
Agent's customConcurrency updated to ≤ current min	Cleared (false)
Custom concurrency disabled	Cleared (false)
Campaign start with flag set	Blocked (422)

---

Key Expiry During Active Campaign

1. fillCallSlots detects expired key → stops new dispatches
2. Records detection timestamp (SET NX)
3. Allows in-progress calls up to 300s grace period
4. Falls back to platform defaults if available, else pauses
5. Does NOT release concurrency reservation
6. After 300s, reconciliation handles cleanup

---

Troubleshooting

Symptom	Cause	Fix
INVALID_CONCURRENCY_LIMIT on key create	Non-integer or out of range	Use integer 1–500
KEY_IN_ACTIVE_USE on key update	Campaign running with this key	Stop campaign first
INCOMPLETE_BYOK_COVERAGE on agent save	Not all 3 categories have keys	Assign LLM + STT + TTS keys
CONCURRENCY_MISMATCH on campaign start	Mismatch flag set	Lower customConcurrency or update key limits
BYOK_SLOTS_UNAVAILABLE	Lock contention	Retry in a few seconds
Counter drift (more calls than expected)	Redis/app crash	Reconciliation auto-fixes within 60s
Campaign won't start after key update	Mismatch flag was set	Edit agent, lower customConcurrency, save

---

How to Test (Step by Step)

Prerequisites

• MongoDB running (local or Atlas)
• Redis running (local or remote)
• WEBHOOK_SECRET_ENCRYPTION_KEY set in backend .env
• Frontend .env configured with API base URL

Test 1: Key Creation with Concurrency Limit

1. Open BYOK Key Manager page
2. Click "Add Key"
3. Fill: name, API key, provider, category, Concurrency Limit = 50
4. Submit → key appears in table with "50" in Concurrency column
5. Try creating with limit = 0 or 501 → should get validation error

Test 2: Agent Custom Concurrency

1. Edit an agent → Step 2 (BYOK section)
2. Enable BYOK for LLM, STT, TTS assign keys to each
3. "Custom Concurrency" section appears
4. Toggle ON, set value = 25, save → success
5. Set value = 999 → should get CONCURRENCY_EXCEEDS_KEY_LIMIT error

Test 3: Campaign Start Validation

1. Start a campaign with the configured agent
2. Should start normally if keys are valid and concurrency is within limits
3. Update a key's concurrencyLimit to below the agent's customConcurrency
4. Try starting again → should get CONCURRENCY_MISMATCH error

Test 4: Key Deletion Protection

1. Start a campaign
2. Try deleting a BYOK key used by the running campaign
3. Should get KEY_DELETION_BLOCKED error
4. Stop campaign → delete succeeds

Test 5: Config Lock During Campaign

1. Start a campaign
2. Try editing the agent's BYOK config (change a key, disable a category)
3. Should get BYOK_CONFIG_LOCKED error
4. Stop campaign → edit succeeds

Test 6: Counter Tracking (requires Redis CLI)

1. Start a campaign with custom concurrency = 5
2. Watch Redis: redis-cli GET byok:{hash}:active_calls
3. Counter should increment as calls dispatch, decrement as calls end
4. Counter should never exceed customConcurrency value

Test 7: Reconciliation

1. Manually set a wrong counter: redis-cli SET byok:{hash}:active_calls 999
2. Wait 60s (or restart server for startup reconciliation)
3. Counter should be corrected to actual active call count

Test 8: Run Automated Tests

cd backend/client_backend
npx vitest run

All 303 tests should pass.