Commit cbcd4c3
authored
test(signing): vendor the complete AdCP 3.1.8 request-signing vector set and pin it against drift (#980)
* test(signing): vendor the complete 3.1.1 request-signing vector set and pin it
The vendored request-signing conformance vectors were an incomplete copy of
the spec set, and the loaders could not tell. Both graders globbed the vector
directory and asserted only that the result was non-empty
(test_verifier_vectors.py, test_canonicalization.py), so any non-empty subset
passed and a missing vector was indistinguishable from a complete set.
Against AdCP 3.1.1 -- the version src/adcp/ADCP_VERSION already targets -- the
vendored copy was missing:
- 4 of 12 positive vectors (009/010 percent-encoding edges, 011/012 IPv6
authority)
- 8 of 28 negative vectors (021-028)
- canonicalization.json entirely, despite the vendored README documenting
it and devoting a section to it
Three further vectors (016, 017, 020) had silently drifted from their upstream
contents. 017 is the notable one: it had been vendored at a revision that
revoked test-ed25519-2026, the primary signing key, rather than the dedicated
test-revoked-2026 keypair the spec now uses -- and keys.json had never been
refreshed to carry that key.
Changes:
- Vendor all 12 missing files plus canonicalization.json, and re-sync the
three drifted vectors, keys.json and README.md. The tree is now
byte-identical to dist/compliance/3.1.1/test-vectors/request-signing/.
- Add tests/conformance/signing/vectors.py: one shared loader whose
parametrization is gated on a completeness pin, replacing the per-module
glob-and-assert-non-empty helpers.
- Add vector_manifest.json, pinning the SHA-256 of every vendored file. It
fails on a missing file, an unexpected extra file, and on content drift.
A filename-only manifest would not have caught what happened to 016/017/020,
so the pin is over content, not names.
- Add test_vector_completeness.py, which also ties the pin to
get_adcp_spec_version(), so bumping the targeted spec version without
re-vendoring fails loudly instead of leaving the SDK graded against a
stale contract.
- Wire canonicalization.json into test_canonicalization.py. It earns its own
grader: it exercises canonicalize_target_uri/canonicalize_authority over
31 cases including 6 malformed-authority rejections that no signed vector
covers at all.
The newly graded vectors surface 12 real conformance failures, marked
strict-xfail with the gap each one tracks so a fix XPASSes and forces the
entry to be retired. They fall into two groups:
- The verifier has no step-1 strict-parse stage, so malformed or ambiguous
input flows on to a later check and is rejected under the wrong error code
(negative 021, 022, 023, 026). Conformance grades the code byte-for-byte,
so "rejected anyway" is not a pass.
- @target-uri canonicalization applies no UTS-46 A-label conversion, drops a
trailing empty query, and accepts five malformed authority shapes the spec
requires be rejected. adcp.signing ships canonicalize_host(), but
canonical.py never calls it.
Each group is filed separately; this change deliberately adds no production
behavior, only the vectors and the guard that keeps them complete.
* test(signing): cite tracking issues in the strict-xfail ledgers
* test(signing): avoid private _pytest import in vector parametrization
* test(signing): note why _write_manifest replaces files instead of mutating
* test(signing): assert the reject code, not just the refusal
The reject branch asserted a bare ValueError -- but urlsplit raises exactly
that for `https://[::1/p`, so `malformed-ipv6-missing-closing-bracket` passed
on someone else's exception while five sibling cases sat in the xfail ledger.
Six reject cases ship in the data; only five were recorded as gaps.
No case asserted `expected_error_code` at all, so the vector set could not
detect its own incompleteness -- the defect this PR exists to close.
Capture the exception and assert the code the vector ships. All six reject
cases now xfail together under #978 and retire together when the typed error
lands.
* test(signing): re-pin the vendored vectors at AdCP 3.1.8
Review follow-up. main moved to 3.1.8 (#992) while this branch pinned 3.1.1,
so test_manifest_pins_the_spec_version_the_sdk_targets failed once the two met.
Only spec_version and source change: the 3.1.1 and 3.1.8
dist/compliance/.../test-vectors/request-signing trees are byte-identical, and
regenerating the manifest reproduces all 43 file hashes unchanged. That is the
pin doing its job -- it forced an explicit re-vendor decision on a spec bump
rather than letting the old set ride silently.
Note for the next bump: `python -m tests.conformance.signing.vectors --write`
regenerates `files` only and carries spec_version/source forward, so those two
fields need editing before the regen, not by it.
Refs #975.1 parent b5c6f16 commit cbcd4c3
23 files changed
Lines changed: 1050 additions & 29 deletions
File tree
- tests/conformance
- signing
- vectors/request-signing
- negative
- positive
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| 13 | + | |
13 | 14 | | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
| 19 | + | |
| 20 | + | |
18 | 21 | | |
19 | 22 | | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
20 | 28 | | |
21 | | - | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
22 | 55 | | |
23 | 56 | | |
24 | 57 | | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
30 | 63 | | |
31 | | - | |
32 | | - | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
33 | 74 | | |
34 | 75 | | |
35 | 76 | | |
| |||
57 | 98 | | |
58 | 99 | | |
59 | 100 | | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
60 | 135 | | |
61 | 136 | | |
62 | 137 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
| |||
21 | 22 | | |
22 | 23 | | |
23 | 24 | | |
| 25 | + | |
24 | 26 | | |
25 | | - | |
26 | 27 | | |
27 | 28 | | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
28 | 55 | | |
29 | 56 | | |
30 | 57 | | |
| |||
73 | 100 | | |
74 | 101 | | |
75 | 102 | | |
76 | | - | |
77 | | - | |
78 | | - | |
79 | | - | |
| 103 | + | |
80 | 104 | | |
81 | 105 | | |
82 | | - | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
83 | 115 | | |
84 | 116 | | |
85 | | - | |
| 117 | + | |
86 | 118 | | |
87 | 119 | | |
88 | 120 | | |
| |||
97 | 129 | | |
98 | 130 | | |
99 | 131 | | |
100 | | - | |
| 132 | + | |
101 | 133 | | |
102 | 134 | | |
103 | 135 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
0 commit comments