[Proposal] UltraRAG “diagnostic mode” powered by the WFGY 16-problem map

[onestardao]

Hi UltraRAG team,

UltraRAG is a very interesting low-code MCP framework for building complex RAG workflows.:contentReference[oaicite:10]{index=10}

I'm the author of WFGY, an MIT-licensed framework that focuses on systematically classifying RAG failures into 16 concrete modes (Input/Retrieval, Reasoning, State, Infra).
The full checklist is documented here:

https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md

This checklist has already seen adoption in:

• ToolUniverse (Harvard MIMS Lab) as a robustness / debugging reference,
• Multimodal RAG Survey (QCRI LLM Lab),
• Rankify (University of Innsbruck) as part of its troubleshooting docs.

Proposal: “diagnostic mode” preset

Since UltraRAG already defines RAG workflows as configurations, it seems natural to ship a preset like:

1. Diagnostic preset

   • A config that adds an extra branch / tool which, when the user flags an answer as suspicious, calls a diagnostic agent using the WFGY 16-problem map.
   • The agent returns: problem_id, region, and a short explanation; these can be logged in the same store as traces.

2. Docs addition

   • Page or subsection “Debugging RAG pipelines with WFGY ProblemMap” explaining how this preset works and when to enable it.

3. No coupling required

   • WFGY remains external; UltraRAG only provides an example configuration and prompts.

I’d be happy to:

• draft a sample UltraRAG configuration for this diagnostic mode,
• adapt the wording of the 16-problem prompts to match your style,
• keep all changes small and optional.

If this direction fits UltraRAG’s goals, I can open a PR with a concrete proposal.