[jacob-liner]

What

• Router: liner-router. Routing is decided from the query text alone: the router labels the query's task type, and for some types it also asks a judge model (typesafe/jev-1.13, one question: "will google/gemini-3-flash-preview answer this correctly?") before choosing among four models: google/gemini-3-flash-preview, google/gemma-4-31b-it, anthropic/claude-haiku-5.5, deepseek/deepseek-v4-flash.
• JEV is called on 2,565 of 8,400 full-split queries; each of those rows carries router_usage with the judge's recorded token counts. (The robustness file is routing-only and carries no usage.)
• The judge is priced at its OpenRouter list price, input $0.042/1M tokens, output $0 (https://openrouter.ai/typesafe/jev-1.13). The model is on its OpenRouter model page and endpoints API but not in the /models list, so the profile is added with sync: false.
• Every answer was generated by the predicted model through Liner's gateway with default parameters (no temperature, no max_tokens), streamed.
• Files: router_inference/config/liner-router.json, router_inference/predictions/liner-router.json (8,400 rows with generations), router_inference/predictions/liner-router-robustness.json (420 routing-only rows), and the judge's price entries (model_cost/model_profiles.yaml, model_cost/model_cost.json, universal_model_names.py).

Local evaluation

Metric	Value
Arena	76.32
Accuracy	78.58
Cost / 1K queries	$0.346 (judge cost included; scripts/pricing/compute_cost.py: answers $2.833144, router calls $0.074211)
Robustness	89.76

Notes

• Calibration: we used external calibration data to estimate each model's capabilities and to tune the routing prompt. We checked this data for overlap with RouterArena questions and found none. No RouterArena questions, ground-truth answers, or scores were used in calibration.
• Routing reads only the query text; it never uses a dataset name or index.
• LiveCodeBench answers were graded locally by running each answer on the test cases (offline sandbox); 0 grader failures.

Try it

This router is available in Liner's product (mark-1.1) at https://platform.liner.com. If you'd like to try it, please let us know and we'll be happy to help you get started.

🤖 Generated with Claude Code

[jacob-liner]

/evaluate

[github-actions]

Router Evaluation Results

Router: liner-router
Dataset Split: full
Evaluator: main @ 44cc1ff

RouterArena Metrics

Metric	Value
RouterArena Score	0.7630
Accuracy	78.55%
Total Cost	$2.907355
Avg Cost per Query	$0.000346
Avg Cost per 1K Queries	$0.3461
Number of Queries	8400
Abnormal Entries	0
Robustness Score	0.8976

---

Evaluation completed by RouterArena automated workflow