oMLX Exporter

Prometheus exporter for oMLX monitoring. Exposes metrics about OMLX models, memory pressure, health, and request statistics via a /metrics endpoint. Also provides ready-to-go Grafana dashboard.

Quick Start

Running with Docker Compose

# Create .env file from the example
cp .env.example .env
# Edit .env with your OMLX connection details

docker compose up -d

This starts three services:

• omlx-exporter - The metrics exporter on port 8001
• prometheus - Prometheus server on port 9090
• grafana - Grafana dashboard on port 3000

Configuration

Configuration is loaded from config.yaml and can be overridden via environment variables.

Parameter	Description	Default
address	HTTP listen address	0.0.0.0:8001
logLevel	Log level (debug, info, warn, error)	info
omlx.targetUrl	OMLX API endpoint	http://localhost:8000
omlx.apiKey	OMLX API key	(empty)
omlx.scrapeIntervalInSec	Scrape interval in seconds	5

For running in Docker on MacOS you should use host.docker.internal:8000

Environment variable names match the YAML keys (e.g., omlx.targetUrl).

Dashboard

Grafana Dashboard configuration lives here

Endpoints

Endpoint	Description
GET /metrics	Prometheus-format metrics
GET /metrics/description	Structured metric descriptions

Metrics

The exporter exposes the following categories of metrics:

Global Server Stats

Metric	Type	Description
omlx_stats_total_prompt_tokens	Gauge	Cumulative number of prompt tokens served by the server
omlx_stats_total_completion_tokens	Gauge	Cumulative number of completion tokens generated by the server
omlx_stats_total_cached_tokens	Gauge	Cumulative number of tokens served from cache
omlx_stats_total_requests	Gauge	Cumulative number of requests served by the server
omlx_stats_cache_efficiency	Gauge	Percentage of tokens served from cache (0-100)
omlx_stats_avg_prefill_tps	Gauge	Average prefill tokens per second across all requests
omlx_stats_avg_generation_tps	Gauge	Average generation tokens per second across all requests
omlx_stats_uptime_seconds	Gauge	Server uptime in seconds

Active Models (Global)

Metric	Type	Description
omlx_active_models_total_active_requests	Gauge	Total number of active requests across all models
omlx_active_models_total_waiting_requests	Gauge	Total number of requests waiting in queue across all models
omlx_active_models_memory_pressure_level	Gauge	Memory pressure level: 0=ok, 1=warning, 2=critical
omlx_active_models_memory_current_bytes	Gauge	Current memory usage in bytes
omlx_active_models_memory_soft_bytes	Gauge	Soft memory limit in bytes (warning threshold)
omlx_active_models_memory_hard_bytes	Gauge	Hard memory limit in bytes (eviction threshold)

Per-Model Request Counts

Metric	Labels	Description
omlx_model_waiting_requests	model	Number of requests waiting in queue for the model
omlx_model_prefilling_requests	model	Number of requests currently in prefill phase for the model
omlx_model_generating_requests	model	Number of requests currently generating tokens for the model
omlx_model_active_requests	model	Total active requests (prefilling + generating) for the model

Per-Model Generating Metrics

Metric	Labels	Description
omlx_model_generating_tokens_per_second	model	Tokens per second for generating requests (sum across all generating requests per model)
omlx_model_generating_generated_tokens	model	Total generated tokens across all generating requests per model
omlx_model_generating_elapsed_seconds	model	Total elapsed time across all generating requests per model

Per-Model Prefilling Metrics

Metric	Labels	Description
omlx_model_prefilling_tokens_per_second	model	Tokens per second for prefilling requests (sum across all prefilling requests per model)
omlx_model_prefilling_prompt_tokens	model	Total prompt tokens across all prefilling requests per model
omlx_model_prefilling_elapsed_seconds	model	Total elapsed time across all prefilling requests per model

Per-Model Waiting Metrics

Metric	Labels	Description
omlx_model_waiting_queue_position	model	Queue position for waiting requests (sum across all waiting requests per model)
omlx_model_waiting_elapsed_seconds	model	Total elapsed time across all waiting requests per model

Health

Metric	Type	Description
omlx_scrape_duration_seconds	Histogram	Duration of each scrape from oMLX API in seconds
omlx_scrape_failures_total	Counter	Total number of failed scrapes from oMLX API

Adding New Metrics

1. Define the metric in the metrics/ package
2. Register it with metrics.DefaultRegistry
3. Update the scraper in exporter/ to collect and set metric values
4. Add corresponding API types in client/models.go if needed

License

MIT