A Review Belongs to a Commit

A reviewer sees a change and raises a useful concern. The author pushes another commit.

The comment is still there. The code it described may not be.

Human reviewers handle this imperfectly too. An automated reviewer makes the state-management problem easier to notice because it can repeat the same concern every time it runs, or confidently act on a conclusion from the previous revision.

The earlier PR-reviewer article focused on useful language and visibility into a personal tool. This article is about a narrower engineering problem: what a finding, check, or review decision refers to as the code changes.

I have developed commit-aware tracking, duplicate-finding safeguards, dry-run checks, and guarded actions in personal review tooling with AI assistance. That describes development of mechanisms, not current continuous operation, organization-wide adoption, or a replacement for human approval.

A PR number is not the reviewed object

The PR is a container for an evolving proposal. The head commit identifies one version of that proposal. The base revision and comparison mode also affect what the reviewer sees.

If an automated tool records only "PR reviewed," it cannot distinguish unchanged work from a new commit. If it records only head SHA, it may miss a changed base or integration result that matters to its conclusion.

A useful record identifies what was reviewed and under which assumptions:

type ReviewSnapshot = {
  headSha: string;
  baseSha: string;
  comparison: 'pull-request-diff';
  rulesRevision: string;
  checkedAt: string;
};

This is an illustrative type, not an internal implementation. The rules revision matters because the same code can be assessed differently after the checking policy changes.

I would not let a convenient database row imply more history than it contains. "Reviewed" should be explainable as a particular observation, not a permanent property of the PR.

CI has a revision too

A green check from an earlier commit does not establish the state of the current one. A required integration check may run on a merge result rather than the branch head. Those are different artifacts.

Before review actions, identify which checks are relevant, which revision they exercised, and whether they are finished. A skipped check or an optional success is not necessarily equivalent to every required gate passing.

Even then, CI evidence has the scope limits discussed in Green Checks Only Cover What You Checked. It can support a reviewable state without proving the absence of semantic defects.

I want a tool to report that distinction rather than promote "green CI" into "safe to approve."

A finding needs identity and evidence

Line numbers are useful for placement. They are poor long-term finding identities.

An author can move the same risky expression to another line. The finding remains relevant, but a line-based key changes and the bot may publish it again. Conversely, new code at the old line can have a different risk that a naive key suppresses.

A more semantic identity can include the rule, affected symbol, risk category, and relevant normalized evidence. Keep the exact source revision alongside it.

Normalization introduces its own trade-off. Strip too much context and different findings collapse into one fingerprint. Keep every character and harmless formatting changes produce duplicates.

Treat a fingerprint as a useful matching mechanism, not proof that two concerns are identical. Re-evaluate the current evidence before deciding the old finding still applies or no new comment is needed.

Suppressed does not mean resolved

A repeated finding can be current and intentionally not republished. A missing finding can mean it was fixed, became unobservable in the selected diff, or was skipped because the analysis failed.

Those states need different names.

current: the concern is supported by the latest evidence
unchanged: current, but no repeated comment is needed
resolved: evidence supports that the concern no longer applies
not-observed: this run did not establish either outcome
needs-human: the tool cannot settle the question safely

The exact storage model can vary. The distinction keeps duplicate suppression from quietly becoming automatic resolution.

An unresolved human thread is another state the tool should not overwrite simply because its own detector no longer sees the pattern. The thread may concern context or a decision that the detector never understood.

Recheck state before publication

Analysis can take time. While it runs, the author can push new code. A comment created from the original diff may now point to invalid lines or describe a risk that has already changed.

Re-read the relevant state before publishing. If it no longer matches the reviewed snapshot, stop, discard the pending action, or schedule another review. Do not silently attach the finding to a different revision because that makes the API call easier.

The same applies to stronger actions. A guarded action should carry the expected revision and use a platform-supported condition where available:

type GuardedReviewAction = {
  expectedHeadSha: string;
  snapshotId: string;
  action: 'publish-review';
};

This type expresses the requirement; it does not implement an atomic guarantee. Reading head and then calling an API leaves a time-of-check/time-of-use interval. If the platform cannot condition the action atomically, handle that limitation explicitly and avoid unsupported claims about stale-action prevention.

A fresh check reduces risk. It does not freeze another person's branch.

A dry run has its own evidence boundary

A dry run can show which findings and actions a tool would produce. It is useful for inspecting scope, tone, duplicates, and targeting without creating external effects.

It does not prove that the later API call will be authorized, attach to the intended line, or observe the same PR state. Nor does a successful API call prove the review content was correct.

Use the dry run to inspect proposed actions. Use publication checks to verify current state and delivery. Keep the two results separate in the control interface.

If the inline attachment fails, a body-level review may preserve the concern, but should still identify the source revision. A fallback should not turn a precisely scoped observation into an unexplained general warning.

Human decisions need context, not silent reassignment

A person may approve a particular revision. New code can change the basis for that decision. The tool should make that change visible and follow the platform's review policy rather than assume an earlier approval means the new code is accepted.

Whether an approval is dismissed, still valid, or requires a new decision depends on the repository's settings and accountable reviewers. The automation should not invent its own release authority.

Likewise, an AI suggestion that a dependency update appears routine is evidence, not permission to merge it. Any permitted automatic action needs its own narrowly defined authorization and technical controls. It should not be inferred from the absence of findings.

A review result describes observed code. It does not transfer responsibility for whatever code arrives next.

Preserve enough history to explain the tool

The control surface should answer ordinary questions: which revision was analyzed, what was published, what was suppressed, why a run stopped, and what remains undecided.

Record identifiers and outcomes with appropriate retention. Avoid storing raw credentials, private payloads, or unnecessary code copies just to make a log feel complete.

When the tool is wrong, the history should help reproduce the decision. Was the input stale? Did normalization merge two findings? Did a missing detector result get treated as resolution? Did the action target change between analysis and publication?

Those are repairable mechanisms. Without the history, every correction becomes a guess about the model's intentions.

The quieter reviewer is not necessarily the better reviewer

Duplicate suppression makes the conversation less noisy. That is useful. It can also hide an unresolved issue if the lifecycle is poorly modelled.

I would evaluate the tool on the relevance and traceability of its findings, not merely on how few comments it emits. Check unchanged code, moved code, corrected code, a new risk at the old location, and a head update during analysis.

Keep a human in charge of interpretation and approval. A review assistant should make that person's evidence easier to find, not create a reassuring status that no one can explain.

The next article, What a Skill Changed, and What It Didn't, examines the same attribution problem at workflow level: what a reusable skill changed, and what it did not.

If your review automation remembers that it saw a PR, what exactly does it remember about the code? That is a useful place to inspect before adding another action to it.