Real-Time Audio Intelligence Pipeline

Author: Bandaluppi Sai Venkata Ganesh

A production-grade pipeline that transcribes call-centre audio, scores sentiment, checks compliance, generates per-call coaching recommendations, and surfaces session-level trend insights - all through a five-agent LangGraph workflow exposed via a FastAPI backend and Streamlit dashboard.

What It Does

Upload a call recording or paste a transcript. The pipeline runs five sequential AI agents:

1. Transcription — converts audio to a timestamped transcript via faster-whisper (or accepts raw text directly)
2. Sentiment Monitor — scores sentiment 1-10, classifies trajectory, detects escalation signals
3. Compliance Checker — validates the transcript against a rule set, scores compliance 0-100, and surfaces violations with severity
4. Coaching Advisor — generates specific, actionable coaching recommendations for the agent
5. Trend Reporter — compares this call against session averages and produces data-driven insights

Results are stored in PostgreSQL and surfaced through four Streamlit dashboard pages.

Features

• Five-agent LangGraph workflow with deterministic fallbacks when Groq is unavailable
• JWT authentication with BOLA prevention on every call record endpoint
• SlowAPI rate limiting on all endpoints (60/min general, 5/min auth)
• Compliance rule engine covering common, billing, technical, sales, and complaint call types
• Audio file validation by magic bytes and extension (.wav .mp3 .m4a, max 50 MB)
• Full input sanitisation including HTML stripping and prompt injection blocking
• Async SQLAlchemy with PostgreSQL in production and SQLite for tests
• 110-test suite with 85% coverage and 70% enforcement threshold
• Bandit security scanning and pip-audit in CI
• Docker Compose deployment with PostgreSQL 16, Redis 7, and non-root container user

Tech Stack

Layer	Technology
API	FastAPI 0.135, Uvicorn
Agents	LangGraph 1.0, Groq (llama-3.3-70b-versatile)
Transcription	faster-whisper
Database	PostgreSQL 16 / SQLAlchemy 2.0 async
Cache	Redis 7
Auth	PyJWT, bcrypt
Rate Limiting	SlowAPI
Dashboard	Streamlit 1.56, Plotly
CI	GitHub Actions, Ruff, Bandit, pip-audit

Prerequisites

• Python 3.11
• Docker and Docker Compose (recommended for full deployment)
• A Groq API key (free tier — agents fall back gracefully without one)
• PostgreSQL 16 and Redis 7 (provided via Docker Compose)

Setup

1. Clone the repository

git clone <repository-url>
cd real-time-audio-intelligence-pipeline

2. Configure environment variables

cp .env.example .env

Edit .env and set:

DATABASE_URL=postgresql+asyncpg://pipeline_user:yourpassword@localhost:5432/audio_pipeline
SECRET_KEY=your-long-random-secret-key-at-least-32-chars
GROQ_API_KEY=your_groq_api_key_here
DB_PASSWORD=yourpassword

3. Start with Docker Compose

docker compose up

The API is available at http://localhost:8008 and the dashboard at http://localhost:8503.

4. Local development without Docker

Install dependencies:

pip install -r requirements.txt
pip install -r requirements-dev.txt

Run the API (uses SQLite by default, no PostgreSQL needed):

python -m uvicorn app.main:app

Run the dashboard in a second terminal:

streamlit run streamlit_app.py

5. Generate sample data

python data/generate_sample_calls.py

6. Generate the project logo

python assets/generate_logo.py

Usage

API Endpoints

All endpoints except /health, /auth/register, and /auth/login require a Bearer token.

Register and login:

curl -X POST http://localhost:8008/auth/register \
  -H "Content-Type: application/json" \
  -d '{"username": "demo", "email": "demo@example.com", "password": "SecurePass123!"}'

curl -X POST http://localhost:8008/auth/login \
  -H "Content-Type: application/json" \
  -d '{"username": "demo", "password": "SecurePass123!"}'

Process a call:

curl -X POST http://localhost:8008/calls/process \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -F "call_type=billing" \
  -F "transcript=Hello my name is Sarah. This call is recorded. Let me help you with your billing concern."

List calls:

curl http://localhost:8008/calls/ \
  -H "Authorization: Bearer YOUR_TOKEN"

Dashboard

Open http://localhost:8503 in your browser and create an account or log in. Navigate between:

• Process Call — submit transcripts or audio files
• Call Library — browse, filter, and delete call records
• Analytics — Plotly charts for volume, compliance, sentiment, and violations
• Coaching Hub — aggregate performance overview and improvement recommendations

Running Tests

python -m pytest tests/ -v

Coverage report:

coverage run -m pytest tests/ -v
coverage report

Environment Variables

Variable	Description	Default
DATABASE_URL	Async SQLAlchemy connection string	SQLite (dev)
SECRET_KEY	JWT signing key (min 32 chars)	Dev placeholder
GROQ_API_KEY	Groq cloud inference key (optional)	Empty
ENVIRONMENT	development or production	development
ALLOWED_ORIGINS	Comma-separated CORS origins	localhost
ACCESS_TOKEN_EXPIRE_MINUTES	JWT expiry in minutes	15
REDIS_URL	Redis connection string	localhost:6379

License

MIT License. Copyright (c) 2026.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files, to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software.