Skip to content

Reject covariance estimates with fewer than two return observations - #775

Open
nitishxbt wants to merge 3 commits into
PyPortfolio:mainfrom
nitishxbt:codex/fix-insufficient-covariance-data
Open

nitishxbt wants to merge 3 commits into
PyPortfolio:mainfrom
nitishxbt:codex/fix-insufficient-covariance-data

Conversation

@nitishxbt

@nitishxbt nitishxbt commented Oct 31, 2026 •

Copy link
Copy Markdown

Summary

  • Raise ValueError in exp_cov and CovarianceShrinkage when fewer than two non-empty return observations are available.
  • Add regression tests for insufficient data and the minimum valid input.

Related issue

Fixes #773.

Testing

  • Full suite: 318 passed, 15 skipped, zero failures across all 333 collected tests, run in batches of 25 to avoid memory limits.
  • All 42 risk-model tests pass, including 16 new cases.
  • Ruff lint and formatting checks pass.

AI disclosure

Partially AI-generated; reviewed and tested before submission.

@nitishxbt
nitishxbt marked this pull request as ready for review November 1, 2026 15:41
@Kiril1Shevtsov

Copy link
Copy Markdown

Thanks for working on this. I tested a related sparse-data case against this PR's head (397aa621): with returns_data=True, A=[0.01, 0.02, 0.04] and B=[NaN, 0.03, NaN] pass the global nonempty-row check, but exp_cov still reports zero variance for B, which has only one observed return. Checking each column separately would also leave pairs with fewer than two joint observations.

Would you be interested in extending this PR to require at least two jointly nonmissing returns for every asset pair, including the diagonal? I have a small incremental patch that preserves the existing covariance formula and still accepts valid constant-return samples. This is a stricter missing-data policy, so I would appreciate your view before proceeding.

The 17 targeted cases give 6 failures / 11 passes on the unmodified head and all pass with the patch; combined with the existing risk-model tests, 59 pass. The full local run is not entirely green: 313 passed, 32 skipped, and 5 HRP failures involving SciPy's removed _LINKAGE_METHODS, also reproduced on the unmodified head. Happy to contribute the regression tests and patch here rather than open a competing PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CovarianceShrinkage / exp_cov return an all-zero matrix with < 2 returns, giving volatility 0 and Sharpe inf

2 participants