Skip to content

feat(review): cap/reject precondition (null-deref) findings whose parameter source isn't in the diff #107

Description

@devops-thiago

Problem Statement

A null-dereference / precondition finding about a method parameter can only be judged by seeing the caller. Both the generator and the verifier see only the diff (PrReviewPrompts USER feeds {{diff}} + base comparison; FindingVerifierPrompts USER feeds {{diff}}). When a change touches a callee whose guarding caller is unchanged — and therefore absent from the diff — the model cannot establish nullability and tends to assume the worst.

The existing self-checks do not catch this. PrReviewPrompts.java:82-87 ("when the other place is not visible in the provided material at all, do not claim the comparison") and the verifier's "the diff already guards against the condition … reject" (FindingVerifierPrompts.java:53-54) both fail to fire, because the guard lives in an unchanged caller that is not in the material, and neither rule is framed around the caller of the changed method.

Dogfood evidence — PR #101: MEDIUM "Potential NullPointerException when accountOwner is null in installedRepos()" (DashboardAccessChecker.java:222). accountOwner is provably non-null — checkAccess early-returns on an empty owner (:126-127) and then passes owner.get(), and evaluateAccess dereferences it at :176 before calling installedRepos — but checkAccess/evaluateAccess are unchanged and absent from the diff. The finding even stated "the caller's contract is not visible in the diff" and posted anyway. Thread: PR #101 discussion_r3410353985.

Proposed Solution

Add a claim-class rule to both PrReviewPrompts and FindingVerifierPrompts (the same shape #97 introduces for a different class):

  • A finding asserting a parameter may be null / violates a precondition is at most low confidence unless the calling code is present in the provided material and shown to pass such a value.
  • The verifier rejects the finding when the parameter's source (caller) is not in the provided material — the nullability is unestablished, not confirmed.
  • Add a verifier-prompt regression example derived from the PR fix(dashboard): key repo-snapshot cache by owner to prevent cross-owner reuse (#91) #101 miss.

This is the cheap interim guard. The deep fix is #55 (codebase-aware context): once the caller is pulled into the prompt, these claims become genuinely verifiable, and the rule relaxes from "reject because unseen" to "confirm/refute against the caller."

Alternatives Considered

Priority

Nice to have

Additional Context

Implementation order

  1. feat(review): gate inline posting on confidence — route low-confidence findings to the summary #105 — gate inline posting on confidence. Do first: it makes "cap to low" actually reduce noise, which is the lever this rule pulls.
  2. fix(review): FindingQuoteValidator checks only suggestion_old, so fabricated code in the description passes #106 (deterministic) and feat(review): cap/reject precondition (null-deref) findings whose parameter source isn't in the diff #107 (this, prompt rule) — independent near-term guards; either order.
  3. feat(review): codebase-aware context beyond the diff (cross-file analysis) #55 — codebase-aware context: the structural fix that makes these precondition claims verifiable and subsumes this rule. Larger effort / its own spike.

Code of Conduct

  • I have searched for existing issues and this is not a duplicate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestjavaPull requests that update java code

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions