All pull requests

Pull requests

• fix(cache): persist regenerated responses after invalid cache entries

  #1474·Cookie0103 opened Nov 3, 2026·

  ·

• Fill in the translation literals for Latvian, Lithuanian, Maltese and Slovenian

  #1473·inchang-ing opened Nov 3, 2026·

  ·
  1

• fix(litellm): log and back off on non-filter BadRequestError

  #1472·pcbeingused333 opened Nov 3, 2026·

  ·

• fix(hle): scale confidence to 0-1 before computing calibration error

  #1470·damienen opened Nov 2, 2026·

  ·
  1

• Combine subtask stderrs in quadrature for the _average and all rows

  #1468·Andreas-Hum opened Nov 1, 2026·

  ·
  1

• Fix double BOS when using chat templates (vllm, transformers, sglang backends)

  #1467·EvanZhuang opened Nov 1, 2026·

  ·

• fix: give the LCB sandbox's fake stdout a binary buffer

  #1465·abhishekcpp007 opened Nov 1, 2026·

  ·
  1

• fix(cache): stabilize solver task hashes

  #1462·jovial-liu opened Nov 1, 2026·

  ·

• Fix Probability metric to aggregate over gold choices, not all choices

  #1461·Sivadheeraj5356 opened Oct 31, 2026·

  ·

• Limit the memory a generated LiveCodeBench solution can add

  #1460·williamcaban opened Oct 29, 2026·

  ·
  1

• Fix typos in docs, docstrings, comments and log messages

  #1458·sarankumar1325 opened Oct 29, 2026·

  ·

• Bump doc-builder to get the docs machine-translated

  #1457·mishig25 opened Oct 29, 2026·

  ·
  1

• Strip reasoning section when only the closing </think> tag is generated

  #1454·selimfedakar opened Oct 28, 2026·

  ·
  1

• Fix aimo_progress_prize_1 exact match on integer gold answers

  #1452·mottopanikeiku opened Oct 27, 2026·

  ·

• fix(metrics): compare mathematical equivalence in Inspect math_scorer (#1430)

  #1451·pfolex opened Oct 24, 2026·

  ·
  1

• Keep Recall@k tie-breaking consistent with Recall@1

  #1450·yunyv opened Oct 24, 2026·

  ·

• Fix source offsets in extracted fragment strings

  #1449·yunyv opened Oct 24, 2026·

  ·

• Skip batch size search when model parallel has several processes

  #1447·chetana1410 opened Oct 23, 2026·

  ·
  1

• Restore TriviaQA prediction normalization

  #1446·mottopanikeiku opened Oct 22, 2026·

  ·
  1

• Use actual gold labels for TruthfulQA MC2

  #1444·salma2vec opened Oct 20, 2026·

  ·
  1

• Count repeated tokens in F1 scores

  #1443·salma2vec opened Oct 20, 2026·

  ·
  1

• Keep explicit model parallel on one multi-GPU machine

  #1439·chetana1410 opened Oct 20, 2026·

  ·
  1

• Fix invalid type annotations (callable, np.array) in the package

  #1438·Michael-Duck opened Oct 20, 2026·

  ·
  1

• docs: point task guides at paths that exist

  #1436·xlb3055 opened Oct 20, 2026·

  ·

• Report truncated_tokens_count in tokens on the transformers backend

  #1435·isHeSatoshi opened Oct 19, 2026·

  ·
  1