RFC-0061: Information-Preserving Degradation for ModalityFilter (design discussion) - #377
Open
Million-mo wants to merge 1 commit into
Conversation
…ty filter * docs(rfc): design discussion for modality_filter degradation Proposes replacing the bare MIME placeholder ([image/png]) that ModalityFilterCapability emits when degrading unsupported multimodal content with an information-preserving placeholder carrying retrievable metadata (filename/mime/source), and optionally a session-scoped persisted reference so a text-only model can delegate the image to a vision-capable subagent or file tool. The problem is grounded in opencode ecosystem findings (HEAD 040b856): bare placeholders were rejected in PR #29279 on hallucination risk, are reported as a defect in issue #42758 (no way to access actual content), and the community consensus (#29216) is to preserve the reference rather than destroy it. Auto model-switching is deliberately out of scope (NOT_PLANNED upstream). Surveys 4 options (status quo / metadata-only / persist+reference / hybrid) with an evaluation matrix and recommends the hybrid: upgrade describe() to honest metadata now, add an opt-in reference strategy for manifests that declare vision-capable consumers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
RFC design discussion PR (no code change). Proposes fixing a defect in
ModalityFilterCapability'sdescribestrategy: when the active model does not support a modality, the capability degrades an image to a bare MIME placeholder ([image/png]), which:Motivation (real deployment)
Encountered while debugging a text-only model (
glm52/ kimi-k2) failing on pasted images through the opencode server — the same downstream scenario as opencode issue #42758.What this RFC does
Surveys 4 options with an evaluation matrix:
[image/png]placeholderreferencestrategyRecommends Option 4: upgrade
describe()to honest metadata immediately (cheap, in-core), add an opt-inreferencestrategy for manifests that declare a vision-capable subagent/tool consumer.Grounding: opencode ecosystem (HEAD
040b856)Discussion ask
referencestrategy (differentiating from opencode's unresolved PRs)?Files
docs/rfcs/draft/RFC-0061-modality-filter-information-preserving-degradation.md(new, 445 lines)