Model Router + OpenTelemetry Metrics
Major feature release adding intelligent model routing and production-grade observability.
- Model Router: Automatic model selection by task type (coding → Llama-3.1-70B, reasoning → GPT-4o, fast → Llama-3.1-8B). Configurable fallback chains with health checks.
- OpenTelemetry: Full OTel integration — traces, metrics, logs exported to Jaeger, Datadog, or your collector.
OTEL_EXPORTER_OTLP_ENDPOINTconfiguration. - API Keys v2: Scoped keys (read-only, admin, model-specific), expiration, rotation API, usage analytics per key.
- Performance: 40% faster cold starts via model pre-warming, connection pooling for vLLM/TGI backends.