Model pool
Candidates
Kimi K3
0.841
C78 · Q90 · S72 · Selected
GPT 5.6 Sol
0.808
C58 · Q84 · S88 · Fast · Costly
DeepSeek V4 Flash
0.792
C88 · Q80 · S75 · Cheap
Gemini 3.6
0.776
C70 · Q78 · S82
Opus 5
—
C52 · Q91 · S68 · Excluded
Marker size is proportional to route score.
Cost axis is inverted: higher is cheaper.
Create a composite model profile that outperforms any single component model
Routing separates into stable pieces — explainable and portable across runtimes.
01
Requests
Normalized intent from apps and agents.
02
Roles & tasks
Taxonomy the router uses to narrow.
03
Endpoints
Model candidates with capabilities.
04
Policy
Budget, eligibility, and scope bounds.
05
Observability
Inspectable decisions after the fact.
Decision flow
Deterministic enough to inspect later.
Normalize
Request intent and policy
Narrow
By role, task, and scope
Eligibility
Capability and budget checks
Score
Measured evidence first
Emit
Explainable RouterDecision
Taxonomy
Domains, roles, tasks, and scopes that keep decisions inspectable.
LayerExamples
Domain
Engineering
Role
coder · reviewer · planner
Task
implement · summarize · extract
Scope
file · repo · workspace
Route between endpoints based on declared and observed performance
One base model can appear as many endpoints — different providers, costs, tools, latency, reliability, cache hit rate.
01
Same model, different endpoints
Providers, runtimes, regions, and local devices change the real candidate — not just the label on the box.
02
Declared ≠ measured
Capability claims establish compatibility. Observed profiles decide which eligible endpoint actually wins.
03
Hidden conditionals don’t travel
Ad-hoc if/else model pickers break when providers, hosts, or policies change. Protocol objects stay portable.
Local and remote in one pool
Connect on-device backends and cloud providers as concrete endpoints, then let the router pick across both for every request — same alias, no client-side if/else.
01
Add the machines you already run
Wire llama-swap peers, local runtimes, and private hosts as first-class endpoints — with locality the router can prefer when they stay eligible.
02
Add the providers you already pay for
Bring OpenAI-compatible cloud endpoints into the same inventory — declared capability plus measured latency, cost, and reliability.
03
One route across both shapes
Clients call an alias. Hard eligibility + evidence-first scoring choose local or remote per request; scored fallbacks keep the posture available when the top pick can’t.
A composite that beats any single member
An alias is a routable posture across a pool — not one fixed provider model. For each request, the router picks the best eligible member; over time the alias behaves like a profile stronger than any one endpoint alone.
01
Pool spans local + remote members
An alias is a routable posture across a pool — not one fixed provider model.
02
Winner per request from measured + declared signals
For each request, the router picks the best eligible member.
03
Conservative alias limits so clients don’t overshoot the pool
Over time the alias behaves like a profile stronger than any one endpoint alone.
04
Fallbacks keep the posture available when the top pick fails
Selected, fallback, and excluded markers stay visible on the decision.
Quality envelope
Higher effective quality than locking to one model
Role and task eligibility first, then score with measured quality before declared defaults.
Cost & latency control
Same pool, different strategies
Save balanced, quality, latency, or cost after the benchmark — weight the same eligible set differently.
Fallbacks & continuity
Keep serving when the top pick can't
Scored fallbacks stay on the decision — so a cold, excluded, or over-budget member doesn't strand the request.
Trust & ops
Decisions you can defend
Every route leaves a RouterDecision — winner, fallbacks, exclusions, and reasons — plus Observe ledgers.
Keep cache hit rate high while you route
Locality-aware routing prefers warm endpoints when they stay eligible — so the composite pool can raise quality without throwing away cache efficiency.
01
Prefer warm local endpoints
When a local member is eligible, routing can keep work on the warm path instead of always bouncing remote.
02
Watch hit tokens and rate together
Observe tracks cacheHitTokens and cacheHitTokenRate so efficiency stays visible as volume grows.
03
Avoided cost shows up in Observe
Cache savings roll into cost avoided — routing quality and cache efficiency are one ops story.
Cache efficiency trend
Cache-hit token volume and cache-hit rate for the filtered slice.
120k
90k
60k
30k
0
100%
75%
50%
25%
0%
00:00
04:00
08:00
12:00
16:00
20:00
24:00
cacheHitTokens
cacheHitTokenRate
Run benchmarks and set routing strategy
Strategy is evidence-based, not guessed up front — grade the endpoints you actually activated, write observed profiles, then save the scoring mode.
01
Connect endpoints
02
Assign roles
03
Full benchmark
04
Observed profiles
05
Save strategy
06
Validate live
What you get back
A RouterDecision you can read later — not a hidden runtime guess.
chosen
Winning endpoint for this request
fallbacks
Ordered backups when the winner fails
exclusions
Who was removed and why
reasons
Stable reason codes for selection and reject
Where you look
Operator surfaces that turn the protocol into daily habit.
Router
Live decision review — winner, fallback order, eligibility diagnostics.
Observe
Volume, cost avoided, cache hit rate, and ranking mix across the pool.
Runtime operators
Configure the pool
Connect endpoints, assign roles, benchmark, save strategy, and validate live routes.
App / agent builders
Call aliases, not models
Discover routable postures through OpenAI-compatible surfaces — Pi and other clients join after setup.
Protocol readers
Portable contracts
Request, endpoint, policy, and RouterDecision objects that travel across providers and hosts.
Pi extension improves routing accuracy
@try-works/pi-role-model sends taxonomy-aware intent with each request — role, task, capabilities, and confidence — so the router scores the right candidates instead of guessing from the prompt alone.
Why accuracy goes up
Progressive disclosure classifies work before the route runs.
01
Classify the request
Compact groups → likely roles → task details only for those roles.
02
Attach advisory intent
role_model.intent carries role, task, modalities, tool classes, confidence, and alternatives.
03
Router stays the authority
Runtime validates against active taxonomy; stale or uncertain metadata falls back to policy — never fails the request.
Connect from Pi
Runtime first — the package never owns lifecycle.
installpi install npm:@try-works/pi-role-model
/role-model setup
/role-model alias recommended
/role-model alias use <alias>
Choose a Role-Model alias posture in Pi. The runtime still picks the concrete endpoint.