Skip to content

Enhance backlog assessment automation and documentation - #485

Merged
robgruen merged 10 commits into
mainfrom
dev/robgruen/typechat_factory
Oct 27, 2026
Merged

robgruen merged 10 commits into
mainfrom
dev/robgruen/typechat_factory

Conversation

@robgruen

@robgruen robgruen commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR makes the read-only backlog assessment easier to understand and keeps TypeChat's project direction under explicit human owner control. It also raises the main assessment budget so the next run has more room to investigate the backlog.

What changed

Owner-controlled project direction

  • Adds GOALS.md as the primary repository policy for product scope. Changes require approval from a human member of @microsoft/typechat.
  • Separates approved goals and non-goals from ideas under consideration, including additional languages and the relationship with the existing TypeChat.NET project. These ideas are not approved implementation work or roadmap commitments.
  • Connects the policy to contributor guidance, documentation, and assessments. Each issue and PR receives an In scope, Needs owner decision, or Conflicts with direction assessment, with a policy citation and a plain-language explanation.
  • Reads the policy at the trusted default-branch commit used by the run, not from proposed policy edits in an unmerged PR. Missing or unclear policy does not authorize implementation.
  • Adds CODEOWNERS coverage for project-direction and automation controls, without assigning all library maintenance to the direction owner.

Clearer reports and explicit budgets

  • Uses plain English, short paragraphs, explained terminology, and user-focused summaries. Evidence, uncertainty, prior decisions, and contributor credit remain visible.
  • Raises the main assessment budget from 200 to 400 AI credits. Threat detection retains its separate default of 400; this is not a 400-credit cap for the whole workflow.
  • Keeps GPT-6.1-sol and maximum reasoning effort for the main assessment. Detection's effort setting, permissions, time/turn limits, and seven-day default report retention are unchanged.
  • Updates the README and automation history with dated milestones, first-run findings, report-retention guidance, and the distinction between implemented files and verified GitHub activation.

Regression coverage and review feedback

  • Adds policy, ownership, trusted-revision, and report-guidance checks, and makes policy/control-file changes trigger the existing workflow validation.
  • Addresses the CodeQL review comment: the test now compares the complete Markdown link destination with the TypeChat.NET project URL, rather than looking for a project-name substring. No alert suppression or application-runtime change is involved.

Validation

  • All 23 repository-automation tests passed locally on Node.js 24.22.1 after the review fix.
  • git diff --check passed.
  • The existing CI validates strict workflow compilation, generated-file consistency, and CodeQL. Checks must pass on the final PR commit before merge.

Approval and rollout

This remains a manually started, report-only pilot. It does not accept or close issues, start implementation, create PRs, merge changes, or enable automatic Copilot reviews.

CODEOWNERS alone does not enforce approval. The initial policy PR needs deliberate human owner review because GitHub uses the CODEOWNERS file on the base branch. Team access, required code-owner reviews, stale-approval rules, and bypass permissions still need GitHub-side verification; this PR does not change those settings.

After merge, start a new manual Backlog assessment run on main using Actions > Backlog assessment > Run workflow. Re-running an older run uses its original commit and would not pick up these changes. Review the new draft's language, scope assessments, coverage, and usage before taking any action.

robgruen and others added 8 commits October 26, 2026 17:15
Add a manually dispatched, report-only backlog assessment with scoped repository reads and shared Copilot guidance. Pin GPT-6.1-sol with max reasoning effort for the main assessment, preserve detector effort defaults, and add strict compilation and contract checks with dated rollout documentation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…_factory

# Conflicts:
#	AUTOMATION.md

Co-authored-by: robgruen <25374553+robgruen@users.noreply.github.com>
Co-authored-by: robgruen <25374553+robgruen@users.noreply.github.com>
Use plain-language reports while preserving evidence and human review. Raise the main assessment allowance to 400 AI credits, document report retention, and update the generated workflow and contract tests. Detection, permissions, and retention settings remain unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread tools/tests/backlog-assessment.test.mjs Fixed
Compare the complete Markdown link destination against the existing TypeChat.NET project URL. This addresses the CodeQL PR comment by ensuring the assertion validates the full destination, not merely link text or a partial match.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@robgruen
robgruen merged commit 3a4907e into main Oct 27, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants