Today a reader who thinks a finding is wrong has exactly one route: open a pull request against a JSON
rule pack. That works for the handful of people who have GitHub and .NET. It does not work for the
person this project is built for, who is a teacher looking at a paragraph that got flagged and thinking
"that is just how I write".
That reaction is the single most valuable signal this project can receive, and there is nowhere for it
to go.
The rule that decides whether this helps or ruins the project
A report can change a rule. It can never change the number.
The published false-positive rate — 0 of 90, interval 0%–4.1% — is the one thing this project is
entitled to be believed on, and it moves only by measuring against the calibration corpus. If a
rule is ever loosened because complaints accumulated, this becomes a detector tuned by popularity,
which is precisely what it criticises elsewhere.
So the loop must end in evidence, not in votes:
someone disagrees → triage → reproduce it against the corpus → measure → republish the rate
A report is a hypothesis. The corpus is the referee. The machinery already exists
(tools/SignsOfAI.Calibration); this issue is about the front half.
What may leave the device
The README promises that nothing leaves the device to run the analysis, with four named
exceptions — a sentence that was corrected on 5 August precisely because it used to say "the only
one" and was wrong. A button that uploads a flagged sentence would be the fifth, and the most
sensitive of them: by construction that text is a student's writing that a teacher was examining.
So the design question is not "how do I ask for consent", it is what is the smallest thing that is
still useful:
{
"ruleId": "lex.underpin",
"language": "es",
"kind": "misfire",
"matched": "…", // optional
"sentence": "…", // optional
"build": "0.4.0",
"host": "desktop"
}No account, no identifier, no telemetry, one report at a time. matched and sentence are opt-in
per report, shown verbatim before sending, and editable — the reader sees the exact bytes and can
delete them. That is not a consent checkbox; it is the same gesture the product makes everywhere
else, look at the evidence before acting on it, turned on itself.
Misfires only — not misses
Deliberately do not accept "you missed this one".
This project has never measured how much machine writing it catches, and says so. A false-negative
channel creates pressure to chase recall, which is the arms race it declined to enter. The direction
that matters is the one where the harm is: a person flagged for writing formally.
Someone with a blatant miss still has this issue tracker.
Publish the dispute rate
Next to the false-positive rate, per rule:
rhet.rule-of-three — fired 412 times; 38 people said it was wrong.
Same move as publishing 0 of 90: uncomfortable, checkable, and nobody else in this category will do
it. When a rule is tightened as a result, its evidence line says so — the way translators are
credited on the language switch.
What this does not solve
It does not unblock #59, #61 or #66. Those need pre-2022 corpora — short complete texts,
Spanish academic writing from more than one source — which are fetched from PLOS, SciELO or
Wikipedia, not donated. A text a user sends has no verifiable provenance or date, so it can never
enter the corpus.
What the loop gives is real counter-examples for tightening a regex, and priority: which rules
annoy actual readers rather than the maintainers. That is worth a lot, and it is a different thing.
Two halves, in this order
1. The client, with no server. The disagree control next to a finding, in the panel and in the
report. It composes a prefilled GitHub issue URL — the reader reviews the content and submits it
under their own account. Zero infrastructure, a public audit trail, and the deliberation lands where
the review committee already works.
2. The endpoint, only if half 1 proves it is needed. The likely blocker is that a teacher has no
GitHub account. If that shows up as a real, observed drop-off, an append-only endpoint on the
PeopleWorks server is justified by a measured reason rather than an assumption — and it brings
retention, a privacy policy, abuse moderation and something to keep running, none of which the
project carries today.
Order
#59 goes first. It is a wrong answer being given to teachers in production right now. This is a
strong second.
Today a reader who thinks a finding is wrong has exactly one route: open a pull request against a JSON
rule pack. That works for the handful of people who have GitHub and .NET. It does not work for the
person this project is built for, who is a teacher looking at a paragraph that got flagged and thinking
"that is just how I write".
That reaction is the single most valuable signal this project can receive, and there is nowhere for it
to go.
The rule that decides whether this helps or ruins the project
The published false-positive rate — 0 of 90, interval 0%–4.1% — is the one thing this project is
entitled to be believed on, and it moves only by measuring against the calibration corpus. If a
rule is ever loosened because complaints accumulated, this becomes a detector tuned by popularity,
which is precisely what it criticises elsewhere.
So the loop must end in evidence, not in votes:
A report is a hypothesis. The corpus is the referee. The machinery already exists
(
tools/SignsOfAI.Calibration); this issue is about the front half.What may leave the device
The README promises that nothing leaves the device to run the analysis, with four named
exceptions — a sentence that was corrected on 5 August precisely because it used to say "the only
one" and was wrong. A button that uploads a flagged sentence would be the fifth, and the most
sensitive of them: by construction that text is a student's writing that a teacher was examining.
So the design question is not "how do I ask for consent", it is what is the smallest thing that is
still useful:
{ "ruleId": "lex.underpin", "language": "es", "kind": "misfire", "matched": "…", // optional "sentence": "…", // optional "build": "0.4.0", "host": "desktop" }No account, no identifier, no telemetry, one report at a time.
matchedandsentenceare opt-inper report, shown verbatim before sending, and editable — the reader sees the exact bytes and can
delete them. That is not a consent checkbox; it is the same gesture the product makes everywhere
else, look at the evidence before acting on it, turned on itself.
Misfires only — not misses
Deliberately do not accept "you missed this one".
This project has never measured how much machine writing it catches, and says so. A false-negative
channel creates pressure to chase recall, which is the arms race it declined to enter. The direction
that matters is the one where the harm is: a person flagged for writing formally.
Someone with a blatant miss still has this issue tracker.
Publish the dispute rate
Next to the false-positive rate, per rule:
Same move as publishing 0 of 90: uncomfortable, checkable, and nobody else in this category will do
it. When a rule is tightened as a result, its
evidenceline says so — the way translators arecredited on the language switch.
What this does not solve
It does not unblock #59, #61 or #66. Those need pre-2022 corpora — short complete texts,
Spanish academic writing from more than one source — which are fetched from PLOS, SciELO or
Wikipedia, not donated. A text a user sends has no verifiable provenance or date, so it can never
enter the corpus.
What the loop gives is real counter-examples for tightening a regex, and priority: which rules
annoy actual readers rather than the maintainers. That is worth a lot, and it is a different thing.
Two halves, in this order
1. The client, with no server. The disagree control next to a finding, in the panel and in the
report. It composes a prefilled GitHub issue URL — the reader reviews the content and submits it
under their own account. Zero infrastructure, a public audit trail, and the deliberation lands where
the review committee already works.
2. The endpoint, only if half 1 proves it is needed. The likely blocker is that a teacher has no
GitHub account. If that shows up as a real, observed drop-off, an append-only endpoint on the
PeopleWorks server is justified by a measured reason rather than an assumption — and it brings
retention, a privacy policy, abuse moderation and something to keep running, none of which the
project carries today.
Order
#59 goes first. It is a wrong answer being given to teachers in production right now. This is a
strong second.