Call Analysis Metrics Developer Guide

Glossary

Term	Meaning
STT	Speech-to-Text converts user audio into text
TTS	Text-to-Speech converts agent text into audio
LLM	Large Language Model generates agent responses
TTFT	Time to First Token how long the LLM takes to start producing output
TTFB	Time to First Byte how long TTS takes to start producing audio
E2E	End-to-End total latency from user finishing speech to agent starting speech
EOU	End of Utterance the moment the system decides the user has stopped talking
VAD	Voice Activity Detection detects when someone is speaking
JWT	JSON Web Token auth token for frontend API calls
PBT	Property-Based Testing testing with randomly generated inputs

Architecture Overview

Three services are involved:

┌──────────────┐     ┌──────────────────┐     ┌─────────────────┐
│ Agent Server │     │ Analysis Service  │     │ Client Backend  │
│  (Python)    │     │ (Python/FastAPI)  │     │ (Node/Express)  │
└──────┬───────┘     └────────┬─────────┘     └────────┬────────┘
       │                      │                        │
       │  POST /transcription-sync                     │
       │  (transcription + session_report)             │
       ├──────────────────────────────────────────────►│
       │                                               │
       │  POST /contact-observability                  │
       │  (per-plugin metrics, interruption data)      │
       ├──────────────────────────────────────────────►│
       │                                               │
       │                      │  POST /analyse/call-metrics
       │                      │◄───────────────────────┤ (triggered by frontend)
       │                      │                        │
       │                      │  Fetch Contact from DB │
       │                      │  Run MetricsEngine     │
       │                      │                        │
       │                      │  POST /call-metrics-sync
       │                      ├───────────────────────►│
       │                      │                        │  $set callMetrics
       │                      │                        │  fire call.metrics_ready

End-to-End Flow: Call End → Metrics Generation

Phase 1: Call Ends Agent Sends Data

When a call ends, post_call_event() in Agent/src/services/post_call.py fires three webhooks:

Step 1 POST /transcription-sync to Client Backend

json

{
  "contactUID": "ct_abc123",
  "transcription": { "items": [...] },
  "provider_transcription": { "conversation": [...] },
  "session_report": {
    "job_id": "AJ_xxx",
    "room_id": "RM_xxx",
    "events": [ ... all session events ... ],
    "options": { "allow_interruptions": true, ... },
    "chat_history": { "items": [...] },
    "usage": [ { "provider": "api.groq.com", ... } ],
    "sdk_version": "1.5.4"
  }
}

The session_report is generated by ctx.make_session_report().to_dict() a LiveKit SDK method that captures the entire session event stream.

Step 2 POST /analyse/conversation to Analysis Service (existing conversation analysis)

Step 3 POST /contact-observability to Client Backend

json

{
  "contactUID": "ct_abc123",
  "observability": {
    "Metrics": {
      "model_usage": [...],
      "metrics_collected": [ ... VAD, LLM, TTS, STT, EOU per-turn data ... ],
      "interruption_metrics": [...],
      "overlapping_speech_events": [...],
      "session_events": [...],
      "options": {...},
      "sdk_version": "..."
    }
  }
}

The metrics_collected data comes from event handlers in agent.py that listen to the LiveKit SDK's metrics_collected callback and classify each event by type (VAD, LLM, TTS, STT, EOU, InterruptionMetrics).

Phase 2: Frontend Triggers Metrics Computation

The user clicks "Analyze" in the dashboard:

Frontend → POST /call-metrics/:contactUID (JWT-protected)
Client Backend (triggerCallMetrics) verifies JWT + user ownership, then forwards to:
Analysis Service → POST /analyse/call-metrics (x-api-secret auth)
Analysis returns 200 immediately and runs computation in a background task

Phase 3: MetricsEngine Computes Everything

MetricsOrchestrator.run() orchestrates:

1. fetch_full_contact(contactUID)     → full Contact doc from MongoDB
2. fetch_agent_prompt(contactUID)     → Contact → Campaign → Agent → Prompt
3. MetricsEngine.compute()            → runs all 12 sub-computations
4. notify_metrics(contactUID, result) → POST /call-metrics-sync to backend

Phase 4: Results Stored and Available

Client Backend (callMetricsSync) stores result as callMetrics on the Contact document
Fires call.metrics_ready webhook event
Frontend polls GET /call-metrics/:contactUID to retrieve the computed metrics

Data Sources Where MetricsEngine Reads From

The compute() method in MetricsEngine uses two data sources with a priority system:

Data	Primary Source	Fallback Source
Session events (agent_state_changed, speech_created, close, session_usage_updated)	`session_report.events`	`observability.Metrics.session_events`
Model usage (LLM/TTS/STT totals)	`session_report.usage`	`observability.Metrics.model_usage`
Session config (options)	`session_report.options`	`observability.Metrics.options`
SDK version	`session_report.sdk_version`	`observability.Metrics.sdk_version`
Per-plugin metrics (VAD, LLM, TTS, STT, EOU per-turn)	`observability.Metrics.metrics_collected`	(only source)
Interruption data	`observability.Metrics.interruption_metrics`	(only source)
Transcription	`contact.transcription`
Provider transcription	`contact.provider_transcription`

The session_report contains the full event stream from the LiveKit SDK but does NOT include per-plugin metrics_collected events (VAD/LLM/TTS/STT/EOU breakdowns) or interruption model data. Those come exclusively from the observability.Metrics payload.

All Files Changed

Client Backend (TypeScript)

File	Change
`src/types/index.ts`	Added `callMetrics` and `session_report` to `IContact` interface. Added `"call.metrics_ready"` to `EVENT_TYPES`.
`src/models/Contact.ts`	Added `callMetrics` and `session_report` fields to Mongoose schema (both `Schema.Types.Mixed, default: null`).
`src/controllers/webhook.controller.ts`	Added `callMetricsSync` handler. Updated `transcriptionSync` to accept and store `session_report`.
`src/routes/webhook.routes.ts`	Registered `POST /call-metrics-sync` route.
`src/controllers/contact.controller.ts`	Added `getCallMetrics` (GET) and `triggerCallMetrics` (POST) handlers.
`src/routes/contact.routes.ts`	Registered `GET /call-metrics/:contactUID` and `POST /call-metrics/:contactUID` routes with JWT + feature access middleware.

Analysis Service (Python)

File	Change
`src/schemas/metrics_schema.py`	New. Pydantic models `CallMetricsRequest` and `CallMetricsResponse`.
`src/routes/metrics_routes.py`	New. `POST /analyse/call-metrics` endpoint with x-api-secret auth and background task.
`app.py`	Registered `metrics_router`.
`src/db/fetching.py`	Added `fetch_full_contact()` and `fetch_agent_prompt()` methods to `DBFetcher`.
`src/pipeline/metrics_engine.py`	New. `MetricsEngine` class with 12 deterministic methods + LLM evaluation + composite scoring.
`src/orchestration/metrics_orchestrator.py`	New. Orchestrates fetch → compute → sync flow.
`src/services/notify_backend.py`	Added `notify_metrics()` method to `ResultSender`.

Agent Server (Python)

File	Change
`agent.py`	Added telemetry collection: `metrics_collected` handler classifies events by type (VAD/LLM/TTS/STT/EOU/InterruptionMetrics). Added handlers for `agent_state_changed`, `speech_created`, `close`, `overlapping_speech`. Captures session options and sdk_version. Stores everything on `ctx.telemetry`.
`src/services/post_call.py`	Sends `session_report` (via `ctx.make_session_report().to_dict()`) in the `/transcription-sync` payload. Merges telemetry data into the `/contact-observability` payload.

MongoDB Schema Changes

Two new fields on the Contact collection:

javascript

// Contact document
{
  // ... existing fields ...
  session_report: { ... },   // NEW full LiveKit session report (events, options, usage, sdk_version)
  callMetrics: { ... },      // NEW computed metrics output from MetricsEngine
}

MetricsEngine What Each Method Computes

`compute_per_turn_latency(transcription)`

Reads each message item in transcription.items:

Assistant turns: extracts llm_node_ttft, tts_node_ttfb, e2e_latency from item.metrics (seconds → ms)
User turns: extracts transcription_delay (STT delay), end_of_turn_delay (seconds → ms)

Produces a turns[] array and aggregates (min/max/avg/median/p95/p99) for each metric type. Null values are excluded from aggregates.

`compute_conversation_flow(transcription)`

Talk ratio: sums started_speaking_at → stopped_speaking_at durations per role, computes user% vs agent%
Interruptions: counts assistant turns where interrupted === true, rate = count / agent turns
Silence gaps: difference between consecutive turns' stop/start timestamps
Turn counts: total, user, agent from message-type items
Avg words per turn: word count / turn count per role

`compute_model_usage(observability, agent_turns)`

Reads observability.Metrics.model_usage array. Each entry has a type field:

llm_usage → provider, model, input_tokens, output_tokens, cached_tokens
tts_usage → provider, model, characters, audio_duration
stt_usage → provider, model, audio_duration

Derived: tokensPerTurn = output tokens / agent turns, ttsEfficiency = characters / audio duration.

`compute_user_behavior(transcription, provider_transcription)`

Avg response time: mean silence gap before user turns (ms)
Repeated phrases: case-insensitive duplicate detection across user utterances
Engagement level: high (≥10 avg words), medium (4-9), low (<4)
Primary language: most frequent language from provider_transcription.conversation[].metadata
Low confidence turns: count where transcript_confidence == 0

`compute_latency_alerts(latency)`

Threshold flags from the latency aggregates:

latencyAlert: avg E2E > 2000ms
highLatencyTurns: turn indices where E2E > 3000ms
llmSlow: avg LLM TTFT > 1000ms
ttsSlow: avg TTS TTFB > 800ms
sttSlow: avg STT delay > 1000ms

`compute_agent_state_metrics(session_events)`

Reads agent_state_changed, speech_created, and agent_handoff events:

Initialization time: agent_handoff → first new_state: "listening"
Thinking time: each listening→thinking / thinking→speaking pair duration
First response time: agent_handoff → first speech_created
Scripted vs LLM turns: counts speech_created by source ("say" vs "generate_reply")
slowFirstResponse: true if first response > 3000ms

`compute_token_growth(session_events)`

Reads session_usage_updated events to track cumulative LLM token counts:

Per-turn input/output deltas via consecutive differencing
Context window growth rate (avg input delta)
Token efficiency (output/input ratio per turn)
Context window utilization % with 80% warning flag
TTS characters total and per-turn

`compute_session_metadata(observability, latency, provider_transcription)`

Session config: from options (allow_interruptions, endpointing delays, etc.)
Session termination: from close event (reason + error)
Post-conversation duration: last agent_state_changed → close event
SDK version: from metrics data
Config-latency correlation: configured vs actual endpointing delay
STT confidence stats: min/max/avg from provider_transcription user entries
TTS characters stats: min/max/avg/total from provider_transcription agent entries

`compute_pipeline_metrics(metrics_collected)`

Reads per-plugin events from observability.Metrics.metrics_collected:

Classifies by type: VAD, LLM, TTS, STT, EOU
Computes aggregates for each component
Correlates EOU + LLM + TTS by speech_id for per-turn pipeline breakdown
totalPipelineLatency = eouDelay + llmTtft + ttsTtfb per speech_id

`compute_interruption_metrics(interruption_data)`

Aggregates from interruption_metrics events (avg duration, detection delay, totals)
Backchannel-to-interruption ratio (null if sum is zero)
Avg interruption/backchannel probabilities from overlapping speech events
interruptionModelSlow: avg detection delay > 500ms
falsePositiveIndicator: count of is_interruption=true AND probability < 0.5
Disabled mode: returns all null when no interruption data exists

`evaluate_agent_performance(transcript, agent_prompt)` (async)

Sends transcript + agent prompt to GPT-4o-mini for structured evaluation:

Prompt adherence score (0-100): how closely agent followed instructions
Guardrails score (0-100): whether agent stayed within scope
Hallucination score (0-100, 100 = none): unsupported factual claims
Each score includes a list of specific issues with turn indices
Retries up to 2 times on LLM failure
Returns null scores with reason when agent prompt unavailable or LLM fails

`compute_call_score(latency, agent_eval, flow, contact_doc)`

Weighted composite score (0-100):

E2E latency: 30% (100 at ≤500ms, 0 at ≥5000ms, linear)
Interruption rate: 15% (100 at 0, 0 at ≥0.5, linear)
Prompt adherence: 20% (direct from LLM eval)
Hallucination: 20% (direct from LLM eval)
Guardrails: 15% (direct from LLM eval)

When a component is null, its weight redistributes proportionally. Quality tiers: excellent (≥80), good (≥60), needs_improvement (≥40), poor (<40). Also computes total duration, effective conversation duration, and identifies the bottleneck component (STT/LLM/TTS).

API Endpoints

Client Backend (JWT-protected, for frontend)

Method	Path	Auth	Description
GET	`/call-metrics/:contactUID`	JWT + callAnalysis feature	Returns `callMetrics` or `{ callMetrics: null, status: "pending" }`
POST	`/call-metrics/:contactUID`	JWT + callAnalysis feature	Triggers metrics computation via Analysis service

Client Backend (internal webhooks, for inter-service)

Method	Path	Auth	Description
POST	`/transcription-sync`	x-api-secret	Stores transcription + session_report on Contact
POST	`/call-metrics-sync`	x-api-secret	Stores computed callMetrics on Contact, fires `call.metrics_ready`

Analysis Service

Method	Path	Auth	Description
POST	`/analyse/call-metrics`	x-api-secret	Triggers background metrics computation, returns 200 immediately

Agent Telemetry Collection (agent.py)

The agent collects two categories of data during a call:

1. Session Report (via LiveKit SDK)

Generated at call end by ctx.make_session_report().to_dict(). Contains the full event stream, session options, chat history, usage summary, and SDK version. Sent via /transcription-sync.

2. Per-Plugin Metrics (via event handlers)

Collected during the call by listening to the metrics_collected callback on the session. Each event is classified:

SDK Class	`type` field	Key fields
`VADMetrics`	`"vad"`	idle_time, inference_duration_total, inference_count
`LLMMetrics`	`"llm"`	duration, ttft, completion_tokens, prompt_tokens, tokens_per_second, speech_id
`TTSMetrics`	`"tts"`	audio_duration, characters_count, duration, ttfb, streamed, speech_id
`STTMetrics`	`"stt"`	audio_duration, duration, streamed, speech_id
`EOUMetrics`	`"eou"`	end_of_utterance_delay, transcription_delay, on_user_turn_completed_delay, speech_id
`InterruptionMetrics`		total_duration, prediction_duration, detection_delay, num_interruptions, num_backchannels

Additional event handlers:

agent_state_changed → records old_state, new_state, timestamp
speech_created → records source ("say" or "generate_reply"), user_initiated, timestamp
close → records reason, error, timestamp
overlapping_speech → records is_interruption, probability, overlap_started_at

All telemetry is wrapped in try/except collection errors never break the call.

Test Coverage

123 Python tests across 8 test files in Analysis/tests/:

File	Tests	Covers
`test_per_turn_latency.py`	9	Metric extraction, null handling, aggregates
`test_conversation_flow.py`	17	Talk ratio, interruptions, silence gaps, word counts
`test_model_usage.py`	16	LLM/TTS/STT extraction, derived ratios
`test_latency_alerts.py`	19	All threshold flags and boundary values
`test_session_metadata.py`	21	Config, termination, correlation, confidence stats
`test_interruption_metrics.py`	15	Aggregates, ratios, false positives, disabled mode
`test_evaluate_agent_performance.py`	7	LLM eval, retries, score clamping, null prompt
`test_call_score.py`	19	Weighted scoring, redistribution, tiers, bottleneck

Client Backend: TypeScript compiles clean (tsc --noEmit passes).

How to Test Step by Step

Prerequisites

Docker Compose running (docker-compose -f docker-compose.dev.yml up)
All three services healthy: cb-client-backend (port 3000), cb-analysis (port 8001), cb-agent
MongoDB accessible at mongodb://localhost:27017
A valid WEBHOOK_SECRET configured in all services (check env/ files)

Step 1: Run Automated Unit Tests

Analysis Service (Python 123 tests)

bash

cd OP_oktivo_backend/Analysis
pip install pytest pytest-asyncio   # one-time setup
python -m pytest tests/ -v

Expected: 123 passed. These test all MetricsEngine computation methods in isolation no DB or network needed.

Client Backend (TypeScript)

bash

cd OP_oktivo_backend/client_backend
npm run test

This runs vitest. The webhook-pipeline integration test requires a live MongoDB connection, so it may fail locally without Docker. The unit tests (apiResponse, webhook.service, rateLimiter, enhancePrompt, promptEnhancer, servicePricing) should all pass.

Also verify TypeScript compiles:

bash

npx tsc --noEmit

Expected: zero errors.

Step 2: Test the Transcription Sync (session_report storage)

This verifies the Agent → Client Backend path for storing session_report.

2a. Create a test contact in MongoDB (via Mongo Express at http://localhost:8081 or mongosh):

javascript

// In the Contact collection
db.Contact.insertOne({
  contactUID: "test_metrics_001",
  customerUID: "cust_test",
  userId: "user_test",
  campaignUID: "camp_test",
  status: "completed",
  callStartedAt: new Date("2025-01-01T10:00:00Z"),
  callEndedAt: new Date("2025-01-01T10:05:00Z"),
})

2b. Send a transcription-sync request with session_report:

bash

curl -X POST http://localhost:3000/transcription-sync \
  -H "Content-Type: application/json" \
  -H "x-api-secret: YOUR_WEBHOOK_SECRET" \
  -d '{
    "contactUID": "test_metrics_001",
    "transcription": {
      "items": [
        {
          "type": "agent_handoff",
          "new_agent_id": "voice_assistant",
          "created_at": 1700000000
        },
        {
          "type": "message",
          "role": "assistant",
          "content": [{"type": "text", "text": "Hello, how can I help?"}],
          "interrupted": false,
          "metrics": {
            "started_speaking_at": 1700000001,
            "stopped_speaking_at": 1700000005,
            "tts_node_ttfb": 0.15,
            "llm_node_ttft": 0.4,
            "e2e_latency": 0.9
          },
          "created_at": 1700000001
        },
        {
          "type": "message",
          "role": "user",
          "content": [{"type": "text", "text": "I want to know about pricing"}],
          "interrupted": false,
          "transcript_confidence": 0.95,
          "metrics": {
            "started_speaking_at": 1700000006,
            "stopped_speaking_at": 1700000009,
            "transcription_delay": 0.3,
            "end_of_turn_delay": 0.25
          },
          "created_at": 1700000009
        },
        {
          "type": "message",
          "role": "assistant",
          "content": [{"type": "text", "text": "Our pricing starts at 5 crore for 3BHK"}],
          "interrupted": false,
          "metrics": {
            "started_speaking_at": 1700000010,
            "stopped_speaking_at": 1700000015,
            "tts_node_ttfb": 0.12,
            "llm_node_ttft": 0.35,
            "e2e_latency": 0.75
          },
          "created_at": 1700000010
        }
      ]
    },
    "session_report": {
      "events": [
        {"type": "agent_state_changed", "old_state": "initializing", "new_state": "listening", "created_at": 1700000000.5},
        {"type": "speech_created", "source": "say", "user_initiated": true, "created_at": 1700000001},
        {"type": "agent_state_changed", "old_state": "listening", "new_state": "speaking", "created_at": 1700000001},
        {"type": "agent_state_changed", "old_state": "speaking", "new_state": "listening", "created_at": 1700000005},
        {"type": "speech_created", "source": "generate_reply", "user_initiated": true, "created_at": 1700000010},
        {"type": "agent_state_changed", "old_state": "listening", "new_state": "thinking", "created_at": 1700000009.5},
        {"type": "agent_state_changed", "old_state": "thinking", "new_state": "speaking", "created_at": 1700000010},
        {"type": "close", "reason": "participant_disconnected", "error": null, "created_at": 1700000020}
      ],
      "options": {
        "allow_interruptions": true,
        "min_endpointing_delay": 0.2,
        "max_endpointing_delay": 2.0
      },
      "usage": [
        {"provider": "groq", "model": "llama-4-scout", "input_tokens": 1200, "output_tokens": 80},
        {"provider": "Cartesia", "model": "sonic-3", "characters_count": 400, "audio_duration": 20.0},
        {"provider": "Sarvam", "model": "saaras:v3", "audio_duration": 5.0}
      ],
      "sdk_version": "1.5.4"
    }
  }'

Expected: 200 OK with "Transcription stored on Contact".

2c. Verify in MongoDB that the Contact now has session_report and transcription fields populated.

Step 3: Test Metrics Trigger (Frontend → Analysis)

This verifies the Client Backend → Analysis Service trigger path.

3a. Get a JWT token by logging in via the auth endpoint (or use an existing token from the frontend).

3b. Trigger metrics computation:

bash

curl -X POST http://localhost:3000/call-metrics/test_metrics_001 \
  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
  -H "Content-Type: application/json"

Expected: 200 OK with "Call metrics computation triggered".

This tells the Client Backend to forward the request to the Analysis service at POST /analyse/call-metrics.

3c. Check Analysis service logs (docker logs cb-analysis -f) for:

Starting metrics computation for contact test_metrics_001
Metrics computation completed for contact test_metrics_001 (status=completed)
Metrics synced to backend for contact test_metrics_001

Step 4: Test Metrics Retrieval

After computation completes (a few seconds):

bash

curl http://localhost:3000/call-metrics/test_metrics_001 \
  -H "Authorization: Bearer YOUR_JWT_TOKEN"

Expected: 200 OK with the full callMetrics object containing:

status: "completed"
perTurnLatency with turns and aggregates
conversationFlow with talk ratio, interruptions, silence gaps
callScore with score, tier, bottleneck
latencyAlerts with threshold flags
agentStateMetrics with initialization time, thinking time
etc.

If metrics haven't been computed yet, you'll get { callMetrics: null, status: "pending" }.

Step 5: Test the Analysis Service Directly (bypass Client Backend)

Useful for debugging the MetricsEngine in isolation:

bash

curl -X POST http://localhost:8001/analyse/call-metrics \
  -H "Content-Type: application/json" \
  -H "x-api-secret: YOUR_WEBHOOK_SECRET" \
  -d '{"contactUID": "test_metrics_001"}'

Expected: 200 OK with {"success": true, "message": "Metrics computation has been triggered."}.

The computation runs in the background. Check logs for progress.

Step 6: Test with Real Call Data

The most realistic test uses the metrics.json file as the session report:

6a. Make a real call through the platform (or use an existing Contact with data).

6b. Verify the Contact document in MongoDB has:

transcription populated by the Agent after call end
session_report populated by the Agent via ctx.make_session_report()
observability.Metrics populated by the Agent with per-plugin telemetry

6c. Trigger metrics from the frontend dashboard or via curl (Step 3).

6d. Check the callMetrics field on the Contact document for the full output.

Step 7: Test Edge Cases

Scenario	How to test	Expected
Contact not found	Trigger metrics for a non-existent contactUID	Analysis logs: "Contact not found skipping"
Empty transcription	Create a Contact with `transcription: null`	`callMetrics.status = "insufficient_data"`
No agent prompt	Create a Contact whose campaign has no Agent	`agentEvaluation.reason = "agent_prompt_unavailable"`, scores null
High latency call	Use transcription with `e2e_latency > 2.0` (seconds)	`latencyAlerts.latencyAlert = true`
No session_report	Contact only has observability data (no session_report)	Engine falls back to `observability.Metrics` for events
Invalid JWT	Call GET/POST `/call-metrics/:id` without token	`401 Unauthorized`
Wrong user	Call with JWT for user who doesn't own the contact	`404 Not Found`
Analysis service down	Trigger metrics when Analysis container is stopped	Client Backend returns `502 Bad Gateway`

Step 8: Test the call.metrics_ready Webhook

If you have webhook endpoints configured for the customer:

Trigger metrics computation (Step 3)
After completion, check the webhook delivery logs
The call.metrics_ready event should fire with the full Contact payload

Quick Smoke Test Checklist

[ ] python -m pytest tests/ -v                    → 123 passed (Analysis)
[ ] npx tsc --noEmit                              → 0 errors (Client Backend)
[ ] npm run test                                   → unit tests pass (Client Backend)
[ ] POST /transcription-sync with session_report   → 200, data stored
[ ] POST /call-metrics/:contactUID (JWT)           → 200, triggers computation
[ ] GET /call-metrics/:contactUID (JWT)            → 200, returns callMetrics
[ ] POST /analyse/call-metrics (x-api-secret)      → 200, background task runs
[ ] POST /call-metrics-sync (x-api-secret)         → 200, callMetrics stored
[ ] Check MongoDB Contact.callMetrics              → full metrics object present
[ ] Check MongoDB Contact.session_report           → session report stored

Call Analysis Metrics Developer Guide ​

Glossary ​

Architecture Overview ​

End-to-End Flow: Call End → Metrics Generation ​

Phase 1: Call Ends Agent Sends Data ​

Phase 2: Frontend Triggers Metrics Computation ​

Phase 3: MetricsEngine Computes Everything ​

Phase 4: Results Stored and Available ​

Data Sources Where MetricsEngine Reads From ​

All Files Changed ​

Client Backend (TypeScript) ​

Analysis Service (Python) ​

Agent Server (Python) ​

MongoDB Schema Changes ​

MetricsEngine What Each Method Computes ​

compute_per_turn_latency(transcription) ​

compute_conversation_flow(transcription) ​

compute_model_usage(observability, agent_turns) ​

compute_user_behavior(transcription, provider_transcription) ​

compute_latency_alerts(latency) ​

compute_agent_state_metrics(session_events) ​

compute_token_growth(session_events) ​

compute_session_metadata(observability, latency, provider_transcription) ​

compute_pipeline_metrics(metrics_collected) ​

compute_interruption_metrics(interruption_data) ​

evaluate_agent_performance(transcript, agent_prompt) (async) ​

compute_call_score(latency, agent_eval, flow, contact_doc) ​

API Endpoints ​

Client Backend (JWT-protected, for frontend) ​

Client Backend (internal webhooks, for inter-service) ​

Analysis Service ​

Agent Telemetry Collection (agent.py) ​

1. Session Report (via LiveKit SDK) ​

2. Per-Plugin Metrics (via event handlers) ​

Test Coverage ​

How to Test Step by Step ​

Prerequisites ​

Step 1: Run Automated Unit Tests ​

Analysis Service (Python 123 tests) ​

Client Backend (TypeScript) ​

Step 2: Test the Transcription Sync (session_report storage) ​

Step 3: Test Metrics Trigger (Frontend → Analysis) ​

Step 4: Test Metrics Retrieval ​

Step 5: Test the Analysis Service Directly (bypass Client Backend) ​

Step 6: Test with Real Call Data ​

Step 7: Test Edge Cases ​

Step 8: Test the call.metrics_ready Webhook ​

Quick Smoke Test Checklist ​

Call Analysis Metrics Developer Guide

Glossary

Architecture Overview

End-to-End Flow: Call End → Metrics Generation

Phase 1: Call Ends Agent Sends Data

Phase 2: Frontend Triggers Metrics Computation

Phase 3: MetricsEngine Computes Everything

Phase 4: Results Stored and Available

Data Sources Where MetricsEngine Reads From

All Files Changed

Client Backend (TypeScript)

Analysis Service (Python)

Agent Server (Python)

MongoDB Schema Changes

MetricsEngine What Each Method Computes

`compute_per_turn_latency(transcription)`

`compute_conversation_flow(transcription)`

`compute_model_usage(observability, agent_turns)`

`compute_user_behavior(transcription, provider_transcription)`

`compute_latency_alerts(latency)`

`compute_agent_state_metrics(session_events)`

`compute_token_growth(session_events)`

`compute_session_metadata(observability, latency, provider_transcription)`

`compute_pipeline_metrics(metrics_collected)`

`compute_interruption_metrics(interruption_data)`

`evaluate_agent_performance(transcript, agent_prompt)` (async)

`compute_call_score(latency, agent_eval, flow, contact_doc)`

API Endpoints

Client Backend (JWT-protected, for frontend)

Client Backend (internal webhooks, for inter-service)

Analysis Service

Agent Telemetry Collection (agent.py)

1. Session Report (via LiveKit SDK)

2. Per-Plugin Metrics (via event handlers)

Test Coverage

How to Test Step by Step

Prerequisites

Step 1: Run Automated Unit Tests

Analysis Service (Python 123 tests)

Client Backend (TypeScript)

Step 2: Test the Transcription Sync (session_report storage)

Step 3: Test Metrics Trigger (Frontend → Analysis)

Step 4: Test Metrics Retrieval

Step 5: Test the Analysis Service Directly (bypass Client Backend)

Step 6: Test with Real Call Data

Step 7: Test Edge Cases

Step 8: Test the call.metrics_ready Webhook

Quick Smoke Test Checklist