Multi-Language Provider System Testing & Migration Guide

Table of Contents

1. Pre-Migration Checklist
2. Migration Procedure (Zero-Downtime)
3. SuperAdmin UI Testing
4. Customer Dashboard Testing
5. Dispatch / Call Testing
6. Rollback Plan

---

Pre-Migration Checklist

Before deploying the new code or running the migration:

• [ ] Take a full MongoDB backup (mongodump or Atlas snapshot)
• [ ] Confirm your .env has the correct MONGO_URI and DB_NAME for the target environment
• [ ] Deploy the new backend code first (it's backward-compatible flat-field presets still work)
• [ ] Confirm the app starts without errors and existing agents can still make calls

---

Migration Procedure

The migration is non-destructive and idempotent it only adds new data, never removes existing fields.

Why It Won't Break Production

1. Flat fields remain untouched the migration adds language_combinations entries alongside (not replacing) existing flat provider fields
2. PresetResolver falls back if an agent has no language set (all existing agents), the resolver uses flat fields exactly as before
3. Voice tts_model is additive existing voices get a new field populated; existing code doesn't depend on it
4. Idempotent running twice produces the same result (skips existing entries)

Step-by-Step

# 1. Ensure backend is deployed with new code (backward-compatible)
# 2. Run migration in staging first
cd backend/client_backend
npx tsx src/scripts/migrate-language-combinations.ts

What the script does (3 phases):

Phase	Action	Impact
1	Populates tts_model on Voice documents that don't have it	Additive only new field
2	Creates an initial language_combination entry from each preset's flat fields	Additive flat fields stay
3	Seeds ProviderModel_Registry from distinct provider/model combos	New collection, no impact on existing

After running, check the summary output:

• Voices updated / orphaned
• Presets migrated / skipped
• Registry entries seeded / already existing
• Errors / warnings

Production Migration

# In production, connect to the prod DB
MONGO_URI="mongodb+srv://..." DB_NAME="Oktivo-Prod" npx tsx src/scripts/migrate-language-combinations.ts

If anything fails, the script exits with code 1. Review warnings/errors in output. Re-running is safe.

---

SuperAdmin UI Testing

Step 1: Provider Registry Panel

Where: SuperAdmin Dashboard → Provider Registry (new section)

#	Test	Expected
1	Navigate to Provider Registry page	Page loads, shows seeded entries from migration
2	Filter by provider_type = "tts"	Only TTS entries show
3	Filter by language = "ar"	Only entries with Arabic in supported_languages
4	Create a new entry (e.g., TTS, "google", "wavenet-v2", languages: ["en-US", "ar"])	Entry created, appears in list
5	Try to create a duplicate (same type + provider + model)	Error: "Duplicate entry exists" (409)
6	Edit the entry change display_name	Saves correctly, other fields preserved
7	Toggle is_active → OFF	Entry shows inactive badge
8	Try to delete an entry referenced by a preset	Error: "Cannot delete: referenced by X presets" (409)
9	Delete an unreferenced entry	Succeeds, entry removed from list
10	Pagination: create 25+ entries, verify page navigation	Shows 20 per page, next/prev works

Step 2: Language Combination Editor (within Preset Editor)

Where: SuperAdmin → System Config → Edit a Preset

#	Test	Expected
1	Open an existing preset (post-migration)	See the initial language_combination that was created by migration
2	Click "Add Language Combination"	New empty row appears
3	Select language_code "ar"	Dropdowns filter to show only providers/models supporting Arabic
4	Select LLM provider + model	Only active registry entries appear
5	Select TTS provider + model	Only active entries with "ar" in supported_languages
6	Select voice	Only voices matching selected tts_provider + tts_model + "ar" language
7	Try to add another entry with same language "ar"	Inline error: "Duplicate language code: ar"
8	Add a second entry with "fr-FR"	Saves correctly, two entries shown
9	Remove a language combination	Entry disappears, save succeeds
10	Save preset with 21 entries	Validation error: "cannot exceed 20 entries"
11	Save preset with a voice that doesn't match the TTS model	Error: "Voice mismatches"
12	Verify supported_languages field auto-derives	Array = union of all language_codes + default_tts_language

Step 3: Backward Compatibility

#	Test	Expected
1	Save a preset with NO language_combinations (clear all)	Saves successfully, flat fields remain
2	Existing agents without language set still dispatch	Call works exactly as before
3	Save in old format from API (no language_combinations in payload)	Accepted without error

---

Customer Dashboard Testing

Step 4: Agent Language Selection

Where: Customer Dashboard → Create/Edit Agent

#	Test	Expected
1	Open agent tied to a preset with language_combinations	Language dropdown visible, populated with preset's supported_languages
2	Open agent tied to a preset WITHOUT language_combinations	Language dropdown disabled or hidden
3	Select "ar" from dropdown	Backend resolves Arabic language_combination; displayed resolved providers update
4	Select "en-US" from dropdown	Resolved providers switch to English stack
5	Clear language selection (set to null)	Agent uses flat preset defaults
6	Try to select a language NOT in supported_languages	Should not appear in dropdown (filtered)
7	Save agent with language "ar"	Agent document persists language: "ar"
8	Re-open agent	Language dropdown shows "ar" selected, resolved providers shown correctly

---

Dispatch / Call Testing

Step 5: Language-Aware Calls

#	Test	Expected
1	Make a test call with agent set to language "ar"	Call dispatches with Arabic providers (check Agent Server logs)
2	Check dispatch metadata	llm_provider, tts_provider, stt_provider, tts_voice match the "ar" language_combination
3	tts_language and stt_language in metadata	Both = "ar"
4	Make a test call with agent set to language null	Falls back to flat preset providers
5	Make a test call with agent whose language doesn't match any combo	Falls back to flat fields, WARNING logged
6	Make a web test call with language "ar"	Same dispatch behavior as phone call

Step 6: Existing Agents (No Changes)

#	Test	Expected
1	Test an existing agent that was never edited post-migration	Call works exactly as before (flat fallback)
2	Check that language field is null on existing agents	Confirmed no behavior change
3	Bulk test: run several existing agents	All dispatch correctly without regression

---

Rollback Plan

If issues are found in production:

Quick Rollback (No Code Revert Needed)

The system is fully backward-compatible. If something goes wrong:

1. Agents: Simply ensure language is null on affected agents they'll use flat fallback

   // In MongoDB shell
   db.agents.updateMany({ language: { $ne: null } }, { $set: { language: null } })

2. Presets: Language combinations are additive flat fields still work. Removing them is optional:

   db.SystemConfig.updateMany({}, { $set: { "presets.$[].language_combinations": [] } })

3. Registry: The ProviderModel_Registry collection can be dropped without impacting dispatch:

   db.ProviderModel_Registry.drop()

Full Rollback (Code Revert)

1. Revert to previous backend deployment
2. Old code ignores language_combinations and language fields (they're just unused fields in MongoDB)
3. No data loss migration only added fields, never removed

---

Quick Smoke Test Sequence (5 minutes)

After deployment + migration, run this minimal sequence to confirm nothing is broken:

1. ✅ App starts without errors
2. ✅ Make a test call with an existing agent (pre-migration, no language set) should work identically
3. ✅ Open SuperAdmin → Provider Registry loads with seeded entries
4. ✅ Open a preset editor → language_combinations section visible with migrated entry
5. ✅ Create a new agent, select a language, make a test call → correct providers used
6. ✅ Check dispatch logs for correct tts_language / stt_language

If all 6 pass, you're good to go.

---

Registry-First Workflow (Updated)

The system now enforces a strict workflow order:

1. Provider Registry  →  Register all LLM/TTS/STT provider+model combos
2. Preset             →  Create presets using registry dropdowns (no free-text)
3. Voice              →  Create voices tied to a specific provider + model
4. Agent              →  Assign preset + select language for dispatch

Key Changes from Original Design

Area	Before	After
Preset provider/model fields	Free-text inputs	Dropdowns from Provider Registry
Preset model field	Editable string	Select only from registered models for that provider
Language Combinations	Manual entry	Auto-populated on new preset (intersection of supported langs)
Default TTS/STT Language	Free-text	Dropdown from available languages
Supported Languages	Manually typed	Auto-derived from language_combinations (read-only)
Voice tts_model	Not required	Required dropdown from registered TTS models
Default Voice filter	By tts_provider only	By tts_provider + tts_model

Implications for SuperAdmins

• Before creating a preset, ensure the desired LLM, TTS, and STT providers/models are all registered in Provider Registry
• Before creating a voice, ensure the TTS provider + model are registered
• When adding a new AI model (e.g., new OpenAI model), first add it to Provider Registry with its supported languages, then it becomes available in preset dropdowns

Auto Language Combination Logic

When creating a new preset:

1. Select LLM provider + model
2. Select TTS provider + model
3. Select STT provider + model
4. On save, the system computes the intersection of supported_languages across all three registry entries
5. One language_combination entry is auto-created per intersecting language, using the preset's flat provider settings
6. supported_languages is auto-derived from those entries

Example: If LLM supports [en, ar, fr], TTS supports [en, ar, hi], STT supports [en, ar], the intersection is [en, ar] → two language_combinations are auto-created.