Skip to content

ENH: Add tF confidence intervals at multiple levels for IV2SLS - #860

Open
soodoku wants to merge 8 commits into
bashtage:mainfrom
finite-sample:feature/tf-confidence-intervals
Open

soodoku wants to merge 8 commits into
bashtage:mainfrom
finite-sample:feature/tf-confidence-intervals

Conversation

@soodoku

@soodoku soodoku commented Nov 2, 2026 •

Copy link
Copy Markdown

Adds IVResults.tf_conf_int(level=0.95, *, method="table") for IV2SLS with one endogenous regressor and one excluded instrument. The method reuses the fitted coefficient, covariance convention and first-stage regression. Included controls, constant weights, unadjusted or robust covariance (including aliases), and debiased=False are supported. Unsupported configurations raise ValueError; conventional inference is unchanged.

Both methods delegate to the public API of the optional tf-inference package. That package owns table loading, interpolation, integrity checks and calibration caching. This PR contains no calibration binaries or copied metadata. The adapter maps confidence levels to the package's exact significance-level keys and caches that mapping.

Using tF inference requires installing tf-inference 0.1.0 from PyPI with Python 3.12+, NumPy 1.26+, SciPy 1.12+ and PyArrow 18+. Installation instructions and compatible CI jobs pin the published release. Core linearmodels requirements are unchanged, and a missing optional package produces installation guidance when tF inference is requested.

The default table method covers 18 confidence levels from 75% to 99.99%, with no interpolation across levels or implicit numerical calibration. Explicit method="compute" supports other levels within that range. At 95%, F strictly above approximately 3.84145882 gives finite bounds, including F between that boundary and 4. For an estimate of 1, SE of 0.4 and F=10, the interval is approximately [-0.37214, 2.37214]. Each confidence level has its own calibrated plateau.

Calibration uses deterministic integration, a finite nuisance-parameter grid and local searches. Numerical validation is not a uniform size proof, a conservative rounding guarantee, or a finite-sample coverage guarantee. The procedure remains asymptotic under the single-instrument IV assumptions; no p-value API is introduced.

The dependency uses the independently reviewed 0.1.0 release, which stores its calibrations in one validated Parquet file. All 109 tF feature tests pass against the PyPI installation; changed-file style checks and all 15 guide examples pass. The published package separately passes all 137 tests in a clean environment, and its installed source/data files match the reviewed candidate.

Earlier full-suite and artifact validation at b4a3497cd1b3e623dab9f6193c50ae688b6e12ac:

  • All 109 feature tests pass locally and against the built wheel installed outside the checkout. Tests use seeded Python-generated data and cover numerical landmarks, boundaries, covariance calculations, supported levels, explicit calibration, caching, labels, transformations, eligibility checks and missing dependencies.
  • 18,108 comparisons across all 18 levels reproduce the previous bundled-table reader exactly, including boundary and plateau inputs.
  • Python 3.10 without tf-inference: 40 tests pass and 69 numerical tests skip. Direct checks confirm normal fitting and conventional intervals work, while both tF methods provide installation guidance.
  • Full local suite: 23,387 passed, 2,473 skipped and 47 expected failures. The remaining 13 failures were reproduced against the original PR head: one existing RandomEffects constant-outcome case and 12 notebooks blocked by sandbox socket restrictions.
  • Flake8, Black, isort, Ruff, changed API numpydoc checks and actionlint pass. Mypy output is identical to the original PR head: 17 existing errors in five files.
  • All 15 guide examples pass. The updated guide and API render without warnings under a standard Sphinx theme. The full site's existing theme is blocked by its socket-based multiprocessing manager in this sandbox.
  • Source distribution and compiled wheel builds pass. Both artifacts contain no tF calibration files and declare no new mandatory dependency.

References: Valid t-Ratio Inference for IV; author manuscript, Appendix B.3; calibration package release and validation.

AI assistance was used for implementation, testing and documentation.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-11-02T03:49:18.855472Z
We couldn’t complete this review. Comment @codex review to try again.
39ba585 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@codecov

codecov Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.08537% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 99.68%. Comparing base (7fa390b) to head (62ce675).
⚠️ Report is 25 commits behind head on main.

Files with missing lines Patch % Lines
linearmodels/tests/iv/test_tf.py 98.92% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #860      +/-   ##
==========================================
- Coverage   99.68%   99.68%   -0.01%     
==========================================
  Files         199      203       +4     
  Lines       37875    38500     +625     
  Branches     2436     2455      +19     
==========================================
+ Hits        37756    38378     +622     
- Misses         51       54       +3     
  Partials       68       68              
Flag Coverage Δ
adder 99.67% <99.08%> (-0.01%) ⬇️
subtractor 99.67% <99.08%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread linearmodels/iv/results.py Resolved
@soodoku soodoku changed the title ENH: Add conservative 95% tF confidence intervals for IV2SLS ENH: Add tF confidence intervals at multiple levels for IV2SLS Nov 2, 2026
@bashtage

bashtage commented Nov 6, 2026

Copy link
Copy Markdown
Owner

While I'm not fundamentally opposed to this PR or the direction, it is too heavy. It has many large binary files. Retaining results is fine, but I would suggest putting them in a single file. If the data is needed, it should be simulated internally as part of the process. If you need the data to confirm R results, simply simulate it in Python, then check R and record results as needed, and then use the same simulation in the test suite.

soodoku commented Nov 6, 2026 •

Copy link
Copy Markdown
Author

Thanks, Kevin. I agree that the PR was carrying too much data. I’ve pushed b4a3497 to remove the duplicated calibration files and simplify the implementation.

The binaries contained precomputed tF critical-value tables used during inference. The test inputs are already generated in Python with a fixed seed. The revised implementation calls the public API of tf-inference, which owns the tables, interpolation, integrity checks and caching. All 18 binary files and their accompanying metadata are removed; the PR now changes 9 files rather than 46. Numerical reference values remain in the test module.

Both tF methods now require the optional tf-inference package, now released on PyPI as 0.1.0. It stores the calibrations in a single Parquet file and requires Python 3.12+, NumPy, SciPy and PyArrow. Core linearmodels requirements are unchanged. I’ve documented pip install tf-inference==0.1.0, added a clear error when the package is missing, and pinned the published release in compatible CI jobs. Table lookup still requires no network access or numerical calibration after installation.

All 109 feature tests pass against the PyPI installation. I also compared 18,108 inputs across all 18 confidence levels with the previous reader, including boundary and plateau cases, and obtained identical results. The full suite has 23,387 passing tests; its 13 failures reproduce on the previous PR head (one existing RandomEffects case and 12 notebook socket restrictions in the local sandbox). Style checks pass, both distributions build without the calibration files, and remote CI is running.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants