Get Started
First launch and connect models
Launch the runtime, connect the local or remote models you actually plan to use, and assign their roles before benchmarking.
After installation:
role-model-routerThe packaged runtime opens the local operator UI and serves the router on your machine.
The first-time setup sequence
Use this order:
- launch the runtime
- connect the local backend or remote provider accounts you plan to use
- activate the actual models or endpoints that should compete in routing
- assign roles to those models
- only then run the full benchmark
That order matters because the benchmark should grade the real candidate set you intend to route across.
Install and launch the runtime
Start the packaged router and open the operator UI on the local machine.
Connect providers and local backends
Wire in the exact local and remote execution paths you plan to route across.
Activate models and assign roles
Publish the real endpoint inventory and bind each model only to the roles it should serve.
Run the full benchmark
Grade the configured candidate set and write measured quality into observed profiles.
Choose and save routing strategy
Set balanced, quality, latency, or cost after the benchmark evidence exists.
Validate a real routed request
Inspect Router and Observe to confirm the winner, fallbacks, and reasons match the evidence.
What to configure in the UI
The operator shell is organized around a few core areas:
- Connect for the first-run handoff into the local and remote setup paths
- Local for local backends, local models, and local endpoint state
- Remote for provider accounts and remote execution paths
- Models for benchmark and model-level routing context
- Router for candidate, config, and decision visibility
- Observe for request and telemetry evidence
- System for readiness and runtime health diagnostics
Which roles to assign
The baseline role IDs include:
general.chatcoder.patchcoder.reviewtool.agentembedderclassifierlanguage.detector
Only assign the roles you genuinely want a model to serve. The benchmark and later routing decisions should reflect your real operator intent, not a maximal test matrix.
What “ready” means before benchmarking
Before you move on, you should have:
- the providers or local backends connected
- the endpoint or model activations in place
- the intended role bindings assigned
- at least two plausible candidates for any route family you want to compare
Next
Continue to Run the full benchmark.