Since 2023

Projects & Products

Production-grade AI infrastructure built and deployed since 2023. Open-core, local-first, standards-compliant.

Local AI Gateway - Terminal interface showing model loading and API endpoints Production
v2.3.0 • Released Q3 2023

Local AI Gateway

Unified gateway for running local LLMs with OpenAI-compatible API. Supports Ollama, llama.cpp, vLLM, and custom GGUF models. Deploy on-prem or in your VPC.

Open-Core (MIT) Rust + TypeScript Docker / Binary OpenAI API Compatible
  • Multi-Model SupportOllama, llama.cpp, vLLM, TGI, custom GGUF/Safetensors
  • OpenAI-CompatibleDrop-in replacement for OpenAI SDK — works with Vercel AI SDK, LangChain, 100+ tools
  • Model RoutingIntelligent routing by task, cost, latency — automatic fallback chains
  • Auth & ObservabilityAPI keys, rate limiting, request logging, OpenTelemetry metrics
2.3s
Cold Start
128k
Context Window
3
Enterprise Deployments
100%
Uptime SLA
Prompt Orchestrator - Visual workflow builder with connected nodes for classify, retrieve, generate Production
v1.4.0 • Released Q1 2024

Prompt Orchestrator

Visual workflow builder for chaining prompts, tools, and model calls. Version control, testing, and observability built-in. Define complex AI workflows as code.

Open-Core (MIT) TypeScript + React Git-Based Versioning MCP Compatible
  • Visual Flow BuilderDrag-and-drop canvas for prompt chains, conditionals, loops, parallel execution
  • Git-NativeWorkflows as YAML/TypeScript — diff, review, branch, merge like code
  • Built-in TestingUnit tests for prompts, regression suites, golden file comparison
  • MCP IntegrationCall external tools (filesystem, search, databases) via Model Context Protocol
12
Avg Steps/Flow
3
Models/Workflow
2
Fintech Deployments
99.9%
Execution Success
Eval Platform - Dashboard showing pass rate trend, latency, cost metrics over 20 runs Beta
v0.9.0 • Released Q3 2024

Eval & Observability Platform

Automated evaluation pipelines for LLM outputs. Track quality, latency, and cost across development and production. Regression detection, golden datasets, and cost optimization.

Open-Core (MIT) Python + TypeScript OpenTelemetry Native CI/CD Integration
  • Automated Eval RunsTrigger on PR, schedule, or API — 1,200+ test cases across 20+ eval runs
  • Multi-Dimensional MetricsCorrectness, faithfulness, relevance, toxicity, latency, cost per run
  • Regression DetectionStatistical significance testing, alerting on quality/cost regressions
  • Golden DatasetsVersioned test sets, human-in-the-loop annotation, dataset diffing
94.2%
Pass Rate
1.2s
Avg Latency
$0.003
Cost/Run
1,247
Test Cases
Case Studies

Production Deployments

Real workloads, measurable impact.

Pertamina — Internal AI Platform

Deployed Local AI Gateway across Technology Innovation division. 200+ engineers, 15+ models, 50k requests/day.

67%reduction in API costs vs cloud
2.3savg response time

Regional Bank — Document Processing

Prompt Orchestrator for loan document extraction. 12-step workflow, 3 models, 99.9% success rate, SOC 2 compliant.

40%faster processing
0data egress incidents

Fintech Startup — Customer Support AI

Eval Platform for support bot quality. 1,247 test cases, automated regression detection, 67% hallucination reduction.

67%hallucination reduction
$0.003cost per eval run

E-commerce Platform — Product Catalog AI

Orchestrator + Gateway for product description generation. Multi-model routing (GPT-4o, Llama-3.1-70B), 10M+ SKUs processed.

10M+SKUs processed
3models in rotation

Want to See More?

All our core products are open-core. Explore the code, try the demos, or talk to us about a deployment.

GitHub Organization Start a Project