All Projects

Case Study

SAGE — Speech Analysis And Guidance Engine

Turning thousands of support calls into structured, scoreable insights

Whisper AIGPT-4o-miniPrompt EngineeringLLM Parallelization
Private project — details on request

What it is

SAGE (Speech Analysis And Guidance Engine) ingests recorded customer-support calls and evaluates each one across 140+ parameters: problems the customer faced, emotional tone across the call's arc, resolution satisfaction, compliance points, and more. Built on Whisper for transcription and GPT-4o-mini for analysis, with LLM parallelization to make the volume tractable. (Private project — details shared on request.)

The problem

Manual call QA reviews a tiny random sample of calls with hours of listening per call, and consistency depends entirely on which reviewer you get. I wanted near-total coverage, parameter-level scoring, and a feedback loop fast enough to change agent behaviour the same week.

How it works

Whisper transcription

Calls are transcribed with Whisper, preserving speaker turns and timestamps so every downstream analysis can cite the exact moment it refers to.

140+ parameter rubric

Each call is scored against a structured rubric covering problem categories, emotional tone progression, resolution quality and process compliance. The rubric is encoded as structured prompts with explicit scoring criteria, so parameter definitions are configuration — not prompt folklore.

LLM parallelization

Scoring parameters are grouped and evaluated in parallel across calls, turning a serial hours-per-batch problem into one that scales out — the throughput trick that makes near-100% call coverage economically sensible.

GPT-4o-mini economics

GPT-4o-mini hit the accuracy/cost balance for high-volume scoring: cheap enough to run on every call, strong enough at structured classification that parameter-level results stayed consistent across reviewers and re-runs.

Challenges & solutions

Asking one prompt for 140+ parameters produces sloppy, inconsistent scoring deep into the list.

Split the rubric into focused groups evaluated by separate prompts, then merged downstream — each call gets narrow attention, and parallel execution keeps total latency flat.

Emotional tone isn't a point-in-time label — calls arc, and a summary label loses the story.

Tone is evaluated per segment of the transcript, so the output is a progression (frustrated → heard → satisfied) rather than a single adjective, which is what makes the coaching feedback actionable.

What I learned

  • At scale, prompt architecture is a throughput problem — rubric partitioning and parallelization matter as much as model choice.
  • Structured output schemas are the difference between LLM output you can aggregate and output you can only read.
  • Coverage changes the product: sampling 2% of calls finds anecdotes, scoring 100% finds patterns.