[XM-Dong]

DT-Router is a proprietary router developed by Ant Digital Technologies. This submission includes 8,400 full-split results and 420 robustness routing decisions.

Prices below are USD per million tokens, based on official public pricing checked on 2026-10-29. All discounted or introductory rates are publicly advertised by the official providers.

Model	Input	Output	Official source
lingdt-3.0-flash	$0.021	$0.062	[Ant Digital MaaS](https://maas.antdigital.com/api/v1/model-service/public/page-list)
anthropic/claude-haiku-5.5	$0.10	$0.50	[Anthropic](https://platform.claude.com/docs/en/models/haiku-5-5/overview#pricing)
glm-5.3-flash	$0.15	$0.50	[Z.AI](https://docs.z.ai/guides/overview/pricing)
google/gemini-3.8-flash	$0.75	$3.75	[Google](https://ai.google.dev/gemini-api/docs/pricing#gemini-3.8-flash)

LingDT uses the official discounted rate of CNY 0.14/0.42, converted at 6.7208 CNY/USD and rounded to three decimal places. Gemini uses Google's published introductory pricing through December 31, 2026. Haiku uses the applicable ≤100K-input-token tier.

RouterArena data were not used for training. Reasoning tokens are included in output_tokens.

/evaluate

Router Evaluation Results

Router: DT-Router
Dataset Split: full

RouterArena Metrics

Metric	Value
RouterArena Score	0.7790
Accuracy	81.02%
Total Cost	$4.029790
Avg Cost per Query	$0.000480
Avg Cost per 1K Queries	$0.4797
Number of Queries	8400
Abnormal Entries	7
Robustness Score	0.7881

⚠️ 7 of 8400 queries (0.1%) had no valid generation (inference failed / empty answer) and were scored as incorrect (0). These queries still count toward the denominator, so accuracy and cost reflect the full query set. Please regenerate predictions for these queries and resubmit for a complete evaluation.

---

Evaluation completed by RouterArena automated workflow

Thanks for the submission, @XM-Dong. We've gone through the prediction files, and they look like genuine model calls: the prompts match RouterArena's exactly, token usage tracks prompt length, and where other submissions used the same models (gemini-3.8-flash, glm-5.3-flash, claude-haiku-5.5), your answers score in line with theirs on the same queries.

Before we can review further, we need a description of the router, since neither the PR nor github.com/antgroup/DTRouter says how it works. In particular:

1. Method. What signal does the router use to pick a model for each query, and how was the policy trained or calibrated? Which datasets were used, and how were their items selected and formatted?
2. RouterArena data. The README rule covers any use of RouterArena data in building the router: its queries, labels, prompt templates, or its dataset composition (for example, sizing a calibration set to match RouterArena's per-source counts). Please confirm that none of these was used.
3. Runs on RouterArena. How many router configurations were scored on RouterArena, locally or here, before this one, and how was the submitted one chosen?
4. lingdt-3.0-flash. It handles 6,845 of the 8,400 queries. Since it's your own model, could you confirm that RouterArena's queries weren't part of its training data?

Seven rows have empty answers with zero token usage (ArcMMLU_114, ArcMMLU_158, ArcMMLU_315, GeoBench_866, MMLUPro_health_4907, QANTA_Literature_40, QANTA_History_1616). If those were provider errors, you're welcome to regenerate them once with the same model. The new price entries will be reviewed separately, together with the pending pricing update.

We'd like to hear back on these before we continue the review.

/evaluate

Router Evaluation Results

Router: DT-Router
Dataset Split: full

RouterArena Metrics

Metric	Value
RouterArena Score	0.7793
Accuracy	81.06%
Total Cost	$4.047159
Avg Cost per Query	$0.000482
Avg Cost per 1K Queries	$0.4818
Number of Queries	8400
Abnormal Entries	2
Robustness Score	0.7881

⚠️ 2 of 8400 queries (0.0%) had no valid generation (inference failed / empty answer) and were scored as incorrect (0). These queries still count toward the denominator, so accuracy and cost reflect the full query set. Please regenerate predictions for these queries and resubmit for a complete evaluation.

---

Evaluation completed by RouterArena automated workflow

@yl231 Thank you for carefully reviewing our submission and checking the prediction files. We appreciate the opportunity to clarify the method and evaluation process.

1. Routing method
   We use a draft-and-escalate approach. LingDT first generates a draft answer. The router then assesses the query and draft for task type, difficulty, and answer adequacy, and considers candidate models’ capabilities and costs to decide whether to retain the draft or escalate. Most queries retain LingDT’s draft under this capability–cost trade-off.
   External calibration data were used to characterize candidate-model capabilities and calibrate the routing prompt. The detailed prompts and decision rules remain proprietary.
2. Calibration data and RouterArena
   We used multiple external datasets, including MMLU, MMLU-Pro, LiveCodeBench, MATH, AIME, and xRouteBench, alongside internal datasets. Examples overlapping with RouterArena were excluded. These data supported capability profiling and prompt calibration, rather than fitting routing decisions against RouterArena’s per-question correctness labels.
3. Evaluation runs and submission selection
   We evaluated multiple configurations locally on RouterArena, primarily to compare candidate-model combinations. We did not inspect RouterArena’s per-question correctness or error breakdowns to adjust routing rules.
4. LingDT training data
   LingDT-3.0-Flash is an enterprise variant of the publicly available Ling-3.0-Flash, with additional improvements for agent workloads. The public base model is available through OpenRouter. Although the enterprise variant’s weights are not public, it is accessible through a public MaaS platform.
   No RouterArena queries were used in the enterprise-specific training or tuning. We cannot independently verify question-level overlap in the original base model’s pretraining corpus. This limitation on independent verification applies to all candidate models in the pool.
   We also evaluated both versions individually:
   | Model | Accuracy | Total generation cost |
   |---|---:|---:|
   | LingDT-3.0-Flash | 71.33% | $0.8236 |
   | Ling-3.0-Flash | 71.45% | $0.8765 |
   Both use the same input and output token prices. Their aggregate accuracies differ by approximately 0.12 percentage points, providing a supplementary comparison showing similar performance on RouterArena.
5. Missing answers
   We reran each of the seven missing generations once using its originally selected model. Five now have answers. GeoBench_866 and QANTA_History_1616 were again blocked by GLM’s content-safety filter. We retained these two failed generations and counted them as incorrect.
   The updated predictions are already included in the result files submitted with this PR.
6. Updated generation costs
   After these updates, accuracy is 81.0630%. Final-answer generation costs $4.047159499. Including the preceding LingDT drafts for escalated queries brings the answering-model generation cost to $4.438232758, and the Arena score will be 77.78.
   We initially reported final-answer costs based on the accounting convention we observed in similar leaderboard submissions. If you need an itemized cost breakdown covering all LingDT draft-generation calls, we would be happy to provide it.
   Thank you again for your time.

/evaluate

Router Evaluation Results

Router: DT-Router
Dataset Split: full
Evaluator: main @ 9f6ae25

RouterArena Metrics

Metric	Value
RouterArena Score	0.7821
Accuracy	81.40%
Total Cost	$4.047159
Avg Cost per Query	$0.000482
Avg Cost per 1K Queries	$0.4818
Number of Queries	8400
Abnormal Entries	2
Robustness Score	0.7881

⚠️ 2 of 8400 queries (0.0%) had no valid generation (inference failed / empty answer) and were scored as incorrect (0). These queries still count toward the denominator, so accuracy and cost reflect the full query set. Please regenerate predictions for these queries and resubmit for a complete evaluation.

---

Evaluation completed by RouterArena automated workflow

Hi @yl231 , the merge conflicts and line-ending issue have been resolved. The evaluation has also completed successfully with the updated workflow. Could you please take another look when convenient? We’re happy to provide any further details. Thank you!

Thanks for the detailed answers, @XM-Dong. The submission is merged. Comparing configurations as you described doesn't stand in the way.

One thing remains: cost. The score should include every model call the router makes for a query, not just the final answer. For DT-Router that means the LingDT drafts on escalated queries and the router's own calls that judge the draft and choose the model. We're adding a per-row field for these calls, and we'll post the format here once it's ready. Meanwhile, the itemized breakdown you offered would help: for each query, the model, input tokens and output tokens of every call.

The leaderboard row will appear in the next ranking update, after we rebuild the price table and re-price every router.

Hi @XM-Dong, one more step before the leaderboard row. As mentioned above, every model call the router makes for a query has to be charged, not just the final answer. For DT-Router that means:

• the LingDT draft on escalated queries;
• the call that judges the draft and chooses the model.

The evaluator now supports this (#251). Add a router_usage list to each prediction row, with one entry per call:

"router_usage": [
  {"model": "lingdt-3.0-flash", "input_tokens": 512, "output_tokens": 230, "purpose": "draft"},
  {"model": "lingdt-3.0-flash", "input_tokens": 840, "output_tokens": 40, "purpose": "judge"}
]

• output_tokens should count every generated token, reasoning included.
• Each call is charged at its model's price.
• uv run python scripts/pricing/compute_cost.py DT-Router shows the answer cost, the router-call cost and the total per 1K queries.
• The README's submission section has the details.

This PR is already merged, so please open a new pull request that updates router_inference/predictions/DT-Router.json, and comment /evaluate there. We'll add the row once it is evaluated. All leaderboard routers were re-priced today at current OpenRouter prices (#262), and DT-Router will be priced on the same table.

Hi @yl231 , thank you for your guidance. Could you please help clarify a couple of points about routing cost accounting?
How would self-hosted models used for routing or judging be priced under the proposal in #266? Since models with similar parameter counts can have substantially different published prices, could you explain how those differences would be reflected in a unified pricing standard?
We checked the 23 prediction-backed leaderboard entries at main@44cc1ff and did not find separate router_usage or router_cost fields. For methods that make additional routing or judging model calls, would they also be required to report that usage and have their total costs and Arena Scores recalculated before their results are considered valid leaderboard scores? It would also be helpful to understand how existing entries would be brought under the same accounting standard to ensure a fair comparison across submissions.
Thank you for your time and clarification.