Product Updates

Changelog

Every release, fix, and improvement across our products. Subscribe to RSS or follow GitHub releases for real-time updates.

gateway v2.3.0 January 15, 2025
Minor Gateway

Model Router + OpenTelemetry Metrics

Major feature release adding intelligent model routing and production-grade observability.

  • Model Router: Automatic model selection by task type (coding → Llama-3.1-70B, reasoning → GPT-4o, fast → Llama-3.1-8B). Configurable fallback chains with health checks.
  • OpenTelemetry: Full OTel integration — traces, metrics, logs exported to Jaeger, Datadog, or your collector. OTEL_EXPORTER_OTLP_ENDPOINT configuration.
  • API Keys v2: Scoped keys (read-only, admin, model-specific), expiration, rotation API, usage analytics per key.
  • Performance: 40% faster cold starts via model pre-warming, connection pooling for vLLM/TGI backends.
orchestrator v1.4.0 December 3, 2024
Minor Orchestrator

MCP Server Support + Workflow Templates

Model Context Protocol integration and a new template marketplace for common AI workflows.

  • MCP Servers: Connect to filesystem, GitHub, PostgreSQL, Redis, and custom MCP servers directly from workflow nodes. Zero-config for stdio/SSE transports.
  • Template Library: 12 built-in templates (RAG pipeline, code review, data extraction, SQL generation, summarization). One-click deploy to your workspace.
  • Parallel Execution: parallel block type for fan-out/fan-in patterns. Automatic concurrency control with maxParallel.
  • Secret Management: Encrypted secrets store with environment-specific values. Reference via {{ secrets.API_KEY }} in any node.
eval v0.9.0 November 12, 2024
Beta Eval

Regression Detection + Golden Datasets

Statistical regression testing and versioned golden datasets for continuous quality assurance.

  • Regression Detection: Welch's t-test + Bonferroni correction on pass-rate, latency, cost metrics. Configurable sensitivity (p < 0.05 default). Slack/PagerDuty alerts.
  • Golden Datasets: Versioned test sets with bakanohito eval dataset push. Diff viewer for dataset changes. Human annotation UI with inter-annotator agreement (Cohen's κ).
  • Custom Evaluators: Register Python/TypeScript evaluators via plugin system. Built-ins: factuality, faithfulness, toxicity, JSON schema, regex, semantic similarity.
  • CI/CD: bakanohito eval run --gate exits non-zero on regression. GitHub Actions, GitLab CI, CircleCI examples in docs.
gateway v2.2.1 October 28, 2024
Patch Gateway

Hotfix: llama.cpp 0.3.0 Compatibility

Fixed breaking changes in llama.cpp 0.3.0 API. Updated GGUF loading, context handling, and tokenization endpoints.

  • Fixed /v1/models endpoint returning empty list for llama.cpp backends
  • Resolved context window truncation bug with long conversations (>32k tokens)
  • Updated tokenization to match new llama_tokenize signature
  • Added integration test matrix for llama.cpp 0.2.x → 0.3.x
orchestrator v1.3.0 September 18, 2024
Minor Orchestrator

TypeScript SDK + Visual Debugger

First-class TypeScript SDK and in-browser workflow debugger with time-travel inspection.

  • TypeScript SDK: npm install @bakanohito/orchestrator. Full type inference for workflow definitions, node configs, and runtime values.
  • Visual Debugger: Step-through execution, inspect node inputs/outputs at each step, time-travel to previous states, breakpoint support.
  • Conditional Nodes: if/else and switch nodes with expression language (JMESPath).
  • Workflow Sharing: Export/import workflow bundles (.bako files) with dependencies. Share via URL or file.
gateway v2.1.0 August 5, 2024
Minor Gateway

vLLM + TGI Backend Support

Added support for vLLM and Text Generation Inference as high-throughput backends alongside Ollama and llama.cpp.

  • vLLM: PagedAttention, continuous batching, tensor parallelism. --backend vllm --tensor-parallel-size 4
  • TGI: Optimized for Hugging Face models. Safetensors loading, quantized weights (GPTQ, AWQ, bitsandbytes).
  • Backend Health: Automatic health checks, circuit breaker, graceful degradation. GET /health/backends
  • Benchmarks: vLLM 3.2x throughput vs llama.cpp on H100 for Llama-3.1-70B. Docs: benchmarks.
orchestrator v1.2.0 June 24, 2024
Minor Orchestrator

GitOps Workflow Deployment

Deploy workflows directly from Git. ArgoCD/Flux compatible manifests. Preview environments for PRs.

  • GitOps: bakanohito orchestrator deploy --gitops generates Kustomize/Helm manifests. ArgoCD ApplicationSet examples included.
  • Preview Environments: Auto-deploy PR workflows to preview-.bakanohito.yourdomain.com. Cleanup on merge/close.
  • Rollback: One-click rollback to previous workflow version. bakanohito orchestrator rollback
  • RBAC: Role-based access for workflow deploy (admin, developer, viewer). SSO group mapping.
gateway v2.0.0 March 12, 2024
Major Gateway

Rust Rewrite + OpenAI API Parity

Complete rewrite in Rust for performance, memory safety, and concurrency. Full OpenAI API compatibility.

  • Rust Core: 10x throughput, 90% memory reduction vs Node.js v1.x. tokio + axum + tower.
  • OpenAI Parity: /v1/chat/completions, /v1/completions, /v1/embeddings, /v1/models, /v1/tokenize. Streaming, function calling, logprobs.
  • Plugin System: Dynamic WASM plugins for custom routing, auth, transformation. SDK in Rust/Go/TypeScript.
  • Breaking Changes: Config format v2 (YAML → TOML). Migration guide: docs.
orchestrator v1.0.0 January 8, 2024
Major Orchestrator

Initial Public Release

First stable release of Prompt Orchestrator. Visual builder, Git-native workflows, local execution.

  • Visual canvas with nodes: LLM call, tool call, condition, loop, transform, output
  • Workflows as TypeScript/YAML — version control, code review, CI/CD ready
  • Local runtime (bakanohito orchestrator run) and server mode
  • Built-in test runner with snapshot testing
  • VS Code extension for workflow authoring (syntax highlight, intellisense, run/debug)
gateway v1.0.0 September 15, 2023
Major Gateway

Initial Release — Local AI Gateway

First production release. Unified API for local LLMs. Deployed at Pertamina Technology Innovation division.

  • Ollama + llama.cpp backends with OpenAI-compatible /v1/chat/completions
  • Model management: pull, list, remove, info via API
  • Request logging, basic auth, rate limiting
  • Docker image (ghcr.io/bakanohito/gateway:v1.0.0) and static binaries
  • Systemd service, health endpoint, Prometheus metrics

Stay Updated

Subscribe to release notifications — no spam, only product updates.

Subscribe