You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A null-dereference / precondition finding about a method parameter can only be judged by seeing the caller. Both the generator and the verifier see only the diff (PrReviewPrompts USER feeds {{diff}} + base comparison; FindingVerifierPrompts USER feeds {{diff}}). When a change touches a callee whose guarding caller is unchanged — and therefore absent from the diff — the model cannot establish nullability and tends to assume the worst.
The existing self-checks do not catch this. PrReviewPrompts.java:82-87 ("when the other place is not visible in the provided material at all, do not claim the comparison") and the verifier's "the diff already guards against the condition … reject" (FindingVerifierPrompts.java:53-54) both fail to fire, because the guard lives in an unchanged caller that is not in the material, and neither rule is framed around the caller of the changed method.
Dogfood evidence — PR #101: MEDIUM "Potential NullPointerException when accountOwner is null in installedRepos()" (DashboardAccessChecker.java:222). accountOwner is provably non-null — checkAccess early-returns on an empty owner (:126-127) and then passes owner.get(), and evaluateAccess dereferences it at :176before calling installedRepos — but checkAccess/evaluateAccess are unchanged and absent from the diff. The finding even stated "the caller's contract is not visible in the diff" and posted anyway. Thread: PR #101discussion_r3410353985.
Proposed Solution
Add a claim-class rule to bothPrReviewPrompts and FindingVerifierPrompts (the same shape #97 introduces for a different class):
A finding asserting a parameter may be null / violates a precondition is at most low confidence unless the calling code is present in the provided material and shown to pass such a value.
The verifier rejects the finding when the parameter's source (caller) is not in the provided material — the nullability is unestablished, not confirmed.
This is the cheap interim guard. The deep fix is #55 (codebase-aware context): once the caller is pulled into the prompt, these claims become genuinely verifiable, and the rule relaxes from "reject because unseen" to "confirm/refute against the caller."
Problem Statement
A null-dereference / precondition finding about a method parameter can only be judged by seeing the caller. Both the generator and the verifier see only the diff (
PrReviewPromptsUSER feeds{{diff}}+ base comparison;FindingVerifierPromptsUSER feeds{{diff}}). When a change touches a callee whose guarding caller is unchanged — and therefore absent from the diff — the model cannot establish nullability and tends to assume the worst.The existing self-checks do not catch this.
PrReviewPrompts.java:82-87("when the other place is not visible in the provided material at all, do not claim the comparison") and the verifier's "the diff already guards against the condition … reject" (FindingVerifierPrompts.java:53-54) both fail to fire, because the guard lives in an unchanged caller that is not in the material, and neither rule is framed around the caller of the changed method.Dogfood evidence — PR #101: MEDIUM "Potential NullPointerException when accountOwner is null in
installedRepos()" (DashboardAccessChecker.java:222).accountOwneris provably non-null —checkAccessearly-returns on an empty owner (:126-127) and then passesowner.get(), andevaluateAccessdereferences it at:176before callinginstalledRepos— butcheckAccess/evaluateAccessare unchanged and absent from the diff. The finding even stated "the caller's contract is not visible in the diff" and posted anyway. Thread: PR #101discussion_r3410353985.Proposed Solution
Add a claim-class rule to both
PrReviewPromptsandFindingVerifierPrompts(the same shape #97 introduces for a different class):This is the cheap interim guard. The deep fix is #55 (codebase-aware context): once the caller is pulled into the prompt, these claims become genuinely verifiable, and the rule relaxes from "reject because unseen" to "confirm/refute against the caller."
Alternatives Considered
PrReviewPrompts.java:67— insufficient: PR fix(dashboard): key repo-snapshot cache by owner to prevent cross-owner reuse (#91) #101 shows the model rationalizing a definitive-sounding MEDIUM despite its own stated uncertainty.Priority
Nice to have
Additional Context
PrReviewPrompts.java:82-91;FindingVerifierPrompts.java:47-54;FindingVerifier.java(the same-modality verifier that re-makes the generator's error).Implementation order
Code of Conduct