Ogentic-RouterRouting
0/ 100
Deciding the route on-device000%
ogentic·router
Open source · Apache-2.0 · v0.1.0

The routing decision
happens on-device,
before data leaves

Privacy-aware LLM routing. Shield classifies on your machine, a policy picks the backend, and only cleared or redacted content crosses the network. A library, not a hosted service — nothing of yours ever reaches our infrastructure.

Apache-2.0PyPI · v0.1.0Local-firstPython 3.10+alpha
Scroll to route
Live

See the decision, before anything is sent.

Three prompts, one policy. A privileged memo stays on a local model. A contract with names goes to the cloud — redacted first. A benign prompt goes to the cloud model you already use. Same code path; the sensitivity decides.

route() returns a decision, not a completion — the privacy-relevant choice is made, and auditable, before a single byte leaves the device.

$ ogentic-router route --prompt "…"

Draft a note about the privileged settlement memo.

sensitivity
87
PRIVILEGE
your device──▶ollama-localon device
ruleprivilege-stays-localtransform
✓ Kept on-device · ollama-local
Sensitive → on-device. deny_cloud makes a cloud route impossible, even if a rule were misordered.
{"prompt_hash":"sha256:9f2c1a…","sensitivity_score":87,"route_decision":"ollama-local","rule_id":"privilege-stays-local","backend_is_local":true}
// faithful illustration of the library's RouteDecisionRun the demo ↗
Why it exists

Five guarantees a cloud gateway can't make.

A SaaS proxy routes on cost — after your content has already left the machine. These invariants are load-bearing, and they only hold because the decision runs in your process, before dispatch.

01no middleman

In-process, on your machine

Classification, policy, and redaction are library calls, not network calls. There is no Ogentic service in the path — no infrastructure that ever sees your content. You import it; it runs in your process.

02decide → dispatch

Decision before dispatch

route() returns a RouteDecision — the choice of backend and why — not a completion. The privacy-relevant decision exists, and is recordable, before a single byte is sent anywhere. You dispatch second.

03loopback-only

Local means local

The Ollama and llama.cpp adapters are loopback-only, enforced in code: they reject any non-loopback host. Content that stays local can only reach a model on your own machine.

04deny_cloud

Fail-closed

Content Shield flags as privilege, PHI, or MNPI can never resolve to a cloud backend — the router raises CloudRouteDeniedError before dispatch, even if a rule is misconfigured or mis-ordered. On by default.

Locality is authoritative when the config declares its backends — a misordered rule can't leak a privileged prompt to the cloud.

05shape-only

Shape-only audit

Every route() emits one decision row — a prompt fingerprint, the sensitivity score, category labels, and the chosen backend — never the raw prompt text. Error paths are recorded too. Fire-and-forget: a full or misconfigured log never crashes the router.

The policy

One YAML policy, read top to bottom.

Not a config console — a plain, versioned file. Each rule tests the on-device classification; the first match picks a backend and an optional transform. Anything that misses every rule falls back to default_backend.

policy.yaml
version: 1
default_backend: ollama-local

deny_cloud:                      # fail-closed guard (on by default)
  enforce: true
  groups: [PRIVILEGE, PHI, MNPI]

budget:                          # per-call cost ceiling (on by default)
  enforce: true
  ceiling_usd: 1.00

rules:                           # first match wins
  - id: privilege-stays-local
    when: { groups_include: [PRIVILEGE, PHI, MNPI] }
    route: ollama-local
  - id: medium-redact-then-cloud
    when: { sensitivity_score_gte: 30 }
    route: openai-cloud
    transform: shield_redact     # mask before it leaves
  - id: low-cloud
    when: { sensitivity_score_gte: 0 }
    route: openai-cloud
The DSL
Match
first rule that matches wins; else default_backend
Groups
groups_include / groups_exclude
Score
sensitivity_score_gte / _lt (0–100)
Category
category_in / category_not_in
Transform
shield_redact — mask before it leaves
Guards
budget ceiling · deny_cloud (fail-closed)

Guards run on every call — a per-call budget ceiling (default $1.00) and the fail-closed deny_cloud. Full reference: POLICY_REFERENCE ↗.

Scenarios

Wherever some content simply can't leave.

Regulated teams already run local models for the sensitive 10% and cloud models for the rest. The router makes that split a policy — automatic, auditable, and impossible to fat-finger.

Legal

Privileged matters stay on-device

Attorney-client privilege and work product never touch a cloud model. The privilege rule pins them to a local backend, and the fail-closed guard makes overriding it impossible.

PRIVILEGE → local
Healthcare

PHI never leaves the building

Patient data is classified on-device and routed to a local model by policy. deny_cloud refuses a cloud backend even if a rule is misordered — HIPAA-shaped by construction.

PHI → local
Finance

MNPI stays inside the wall

Material non-public information is pinned local. Medium-sensitivity content gets shield_redact applied before any cloud call — the names are masked while the analysis still ships.

MNPI → local
AI agents

Drop-in for existing OpenAI code

Point base_url at the local router — one line. Benign prompts still reach the cloud model you already use; the privacy layer is invisible until the content actually matters.

base_url swap
Lifecycle

Classify, decide, guard, dispatch.

Five steps from a prompt to a dispatched call. The one that changes everything is the second — the decision is made, and recorded, before the third step lets any byte leave.

1 · classifyclassify

Shield classifies

The prompt is scored on-device for privilege / PHI / MNPI and an overall sensitivity score. A library call — nothing has left the machine yet.

2 · decideroute()

Policy picks a backend

First-match-wins rules choose a backend and an optional transform. route() returns the RouteDecision — the choice, made and recordable before dispatch.

3 · guardguard

Budget + deny-cloud

The estimated cost is checked against the per-call ceiling, and the fail-closed guard refuses any cloud route for privileged content. No partial sends.

4 · dispatchdispatch

Send the cleared payload

Only now is a backend called — a loopback-only local adapter, or a host-allowlisted cloud one — carrying only cleared or redacted content.

5 · recordaudit

Shape-only audit row

One JSON-lines row per call: a prompt hash, the score, the labels, the decision — never the raw text. Error paths included.

classify route() guard dispatch audit

// route() returns a decision, not a completion — the privacy-relevant choice is made, and auditable, before a single byte is sent.

Backends

One async chat(), four backends.

Every adapter implements a single method — chat(messages, …) — and passes provider-native responses through unchanged. Two live on your machine (loopback-only); two reach the cloud (host-allowlisted). Swap where a route goes without touching the policy.

local[local]

OllamaAdapter

kindollama
modelllama3.2:3b
hostlocalhost:11434

Loopback-only — rejects any non-loopback host.

local[local]

LlamaCppAdapter

kindllama.cpp
modelserver-pinned
hostlocalhost:8080

Loopback-only; model pinned server-side.

cloud[cloud]

OpenAIAdapter

kindopenai
modelgpt-4o-mini
hostapi.openai.com

Host-allowlisted; key read from env, never inlined.

cloud[cloud]

AnthropicAdapter

kindanthropic
modelclaude-sonnet-4-5
hostapi.anthropic.com

Allowlisted; defaults max_tokens=1024.

How it compares

The axis everyone competes on — inverted.

Every AI control plane routes on cost, latency, and quality, and decides after the content has left. That's the right tool for unregulated traffic. For regulated content, the axis and the timing are both wrong.

SaaS gateway
OpenRouter · Portkey · Merge · LiteLLM
Form factor
SaaS proxy
Routing axis
Cost / latency / quality
Where it decides
Vendor servers
Sensitive path
Transits vendor + provider
Audit
Vendor dashboard
Trust model
Trust the vendor
ogentic-router
local library · your process
Form factor
Local library
Routing axis
Sensitivity first, then cost
Where it decides
Your process
Sensitive path
Stays on-device
Audit
Shape-only, user-held
Trust model
Open source — verify it

Zero-data-retention is not local-first. “Deleted after we processed it” ≠ “never left the device.” A retention promise still ships your content to a third party who can be compelled to produce it while they hold it. Full sourcing: COMPARISON ↗.

Code

Route a prompt in five lines.

The library makes the full per-prompt decision today; the OpenAI-shaped server is the drop-in for code you already have. Same policy, same classifier — Python 3.10+, Apache-2.0.

route.pyPython
from ogentic_router import Policy, Router

policy = Policy.from_yaml("policy.yaml")
router = Router(policy)          # default classifier: ogentic-shield

decision = router.route(
    "Draft a note about the privileged settlement memo."
)

print(decision.backend_id)   # -> ollama-local   (privilege stays local)
print(decision.rule_id)      # -> privilege-stays-local
print(decision.reasoning)    # -> why this rule fired

# route() returns a decision, not a completion — you dispatch to
# decision.backend_id yourself, after the choice is already recordable.
openai_swap.pyOpenAI SDK
# 1. start the local router (one terminal)
#    ROUTER_CONFIG=router.yaml ogentic-router serve   # 127.0.0.1:8080

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:8080/v1",   # <- the only change
    api_key="not-used-by-the-router",       # server holds real keys
)

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Say hello."}],
)

# Inspect what the router will do: GET /v1/policy
policy.yamlPolicy
version: 1
default_backend: ollama-local

deny_cloud:                      # fail-closed (on by default)
  enforce: true
  groups: [PRIVILEGE, PHI, MNPI]

rules:                           # first match wins
  - id: privilege-stays-local
    when: { groups_include: [PRIVILEGE, PHI, MNPI] }
    route: ollama-local
  - id: medium-redact-then-cloud
    when: { sensitivity_score_gte: 30 }
    route: openai-cloud
    transform: shield_redact
  - id: low-cloud
    when: { sensitivity_score_gte: 0 }
    route: openai-cloud
terminalCLI
pip install "ogentic-router[shield,cloud,local]"

# route one prompt; print the decision (never calls a backend)
ogentic-router route --policy policy.yaml \
  --prompt "attorney work product"

# the safe "what would happen" inspector — no server, no model call
ogentic-router policies dry-run policy.yaml --prompt "..."

# boot the OpenAI-shaped server (or the stdio MCP surface with --mcp)
ogentic-router serve --config router.yaml --port 8080
The ogentic stack

Shield classifies. Router decides. Audit proves.

Composable Apache-2.0 primitives for regulated AI. Shield reads sensitivity; Router makes the on-device routing decision; Audit records tamper-evident evidence. Three are live today; the rest are on the way.

classify → route → prove · Router is the decision layer between ogentic-shield ↗ and ogentic-audit ↗ — part of the OgenticAI ↗ OSS suite.

Get started

Install it. Route in five lines.

The base install is slim — just the policy DSL. Add the extras you need: the classifier, the adapters, the drop-in server. No account, no telemetry, no service in the path.

pip install "ogentic-router[shield,cloud,local,server]"