Source
Scouted from headroom branch feat/lossless-provider-seam (mature: 148-line test suite, near-merge). The architectural idea: external code can propose compactions, but the core never trusts a proposal — every one is verified before acceptance.
Proposal
A pluggable provider seam in ctxzip:
type LosslessProvider interface {
// Propose returns a candidate compaction for content, or nil to pass.
Propose(content string, ctx ProposeContext) *Compaction
}
type Compaction struct {
Text string // the proposed compact form
Recover func() (string, error) // reproduces the original
Equivalence Equivalence // ByteEqual | CanonicalJSON
Label string // savings attribution name
}
Core behavior (the safety gate, non-negotiable):
- On every proposal, call
Recover() and verify: byte equality for ByteEqual; canonical-JSON value equality for CanonicalJSON — type-strict, rejecting true→1 style mutations.
- A provider that returns nil, errors, or fails verification is silently skipped; a broken provider can never make output lossy or break a request.
- Among built-in folds and provider proposals, keep whichever verified fold shrinks the most.
Marginal savings attribution
Record each winning provider's marginal savings (bytes/tokens beyond the best built-in fold) under its Label, surfaced through the host's metrics (in forge: additive fields on context_compressed / rollup on invocation_complete). This answers "what is this extension actually buying us" separately from total strategy savings.
Related idea for forge (out of scope here, noting for the record)
The same headroom branch enables cross-turn deduplication by default (identical content blocks across consecutive turns, keep-earliest, prefix-monotonic, cache-safe). In forge this would live in the llm.Client wrapper — but it interacts with prompt-cache prefix stability, so it should be designed together with initializ/forge#294.
Priority
Lower urgency than the densification work — the seam pays off when there is a second compactor author (enterprise extension, host-specific folds). File now so the interface can be considered before v1 API freeze.
Source
Scouted from headroom branch
feat/lossless-provider-seam(mature: 148-line test suite, near-merge). The architectural idea: external code can propose compactions, but the core never trusts a proposal — every one is verified before acceptance.Proposal
A pluggable provider seam in ctxzip:
Core behavior (the safety gate, non-negotiable):
Recover()and verify: byte equality forByteEqual; canonical-JSON value equality forCanonicalJSON— type-strict, rejectingtrue→1style mutations.Marginal savings attribution
Record each winning provider's marginal savings (bytes/tokens beyond the best built-in fold) under its
Label, surfaced through the host's metrics (in forge: additive fields oncontext_compressed/ rollup oninvocation_complete). This answers "what is this extension actually buying us" separately from total strategy savings.Related idea for forge (out of scope here, noting for the record)
The same headroom branch enables cross-turn deduplication by default (identical content blocks across consecutive turns, keep-earliest, prefix-monotonic, cache-safe). In forge this would live in the
llm.Clientwrapper — but it interacts with prompt-cache prefix stability, so it should be designed together with initializ/forge#294.Priority
Lower urgency than the densification work — the seam pays off when there is a second compactor author (enterprise extension, host-specific folds). File now so the interface can be considered before v1 API freeze.