Skip to content

fix(trace): session code changes sum the diff of each commit instead of the net diff #3674

Closed

Description

@jiparis

Problem

chainloop trace records the code changes of an AI coding session as the sum of the diff of each session commit. It does not record the net diff of the session. So the line counts and the file status are wrong when two or more session commits change the same file.

Example: a branch with two session commits on one new file.

Lines added Lines removed
Commit 1 (creates the file) 225 0
Commit 2 (edits the file) 18 4
Net diff against the base branch 239 0
Recorded in the evidence 243 4

The recorded file status is also modified, but the file is new.

Cause

  • sessionCodeChanges (app/cli/pkg/action/trace_hook_handler.go) calls CodeChangesForCommits with the set of session commits.
  • CodeChangesForCommits diffs each commit against its first parent (app/cli/internal/trace/git/gogit.go, exec.go).
  • addCommitStats (app/cli/internal/trace/git/aggregate.go) adds the lines of each commit. The file status is the status of the last commit.
  • TestGoGitClient_CodeChangesForCommits (gogit_test.go) and the test in exec_test.go expect the sum.

This behavior has been there since the trace command was open sourced (#3388).

Since #3658, the AI and human split of added lines is net (blame at the pushed commit). lines_added is still the sum. So in one file, ai_lines_added + human_lines_added (239) is not equal to lines_added (243). The line ranges of #3665 also follow the net count.

Expected behavior

  • lines_added and lines_removed of each file, and the totals, are the net change of the session commits.
  • A file that the session creates has the status created, also when a later session commit changes it.
  • ai_lines_added + human_lines_added is equal to lines_added for each file.

Proposed fix

Compute the line counts and the file status from one net diff, from the first parent of the first session commit to the pushed commit. Or derive them from the blame at the pushed commit, as netAdded does. Keep the set of session commits only to select the files and lines that belong to the session. This keeps the case of commits that are not contiguous, which the set-based API supports.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions