Suggestion: WFGY as a system-level debugging layer for LLM / RAG pipelines

[onestardao]

Hi, thank you for maintaining this great list on Software Engineering for ML.

I would like to suggest WFGY as a resource that sits at the system level, focusing on LLM and RAG robustness.

• Project: WFGY (All Principles Return to One)
• Repo: https://github.com/onestardao/WFGY
• License: MIT
• Stars: about 1.4k+ on GitHub

Short summary:

• WFGY is a text based framework. You load the TXT pack into an LLM and get a structured “debugging assistant” that speaks in terms of failure modes and system components.
• It does not change infra. It is meant to be an overlay on top of existing vector stores, retrievers and agents.

Key piece for SE4ML readers:

WFGY 2.0 ProblemMap
https://github.com/onestardao/WFGY/tree/main/ProblemMap

• 16 named failure modes for RAG and LLM systems, grouped by retrieval, chunking, vector store, semantic coherence, deployment ordering, etc.
• Each problem has: description, anti patterns, diagnosis checklist, and concrete actions (for example rebalancing retrieval weights, re indexing, changing chunk to embedding contracts).
• Used as a sort of “semantic firewall” before people start changing infrastructure.

There is also:

• WFGY 1.0 PDF for the formal math and evaluation setup.
• WFGY 3.0 Singularity Demo with 131 S class questions for long horizon reasoning, for people who want to explore system evaluation.

If this feels relevant, I can open a PR and propose a small entry under the debugging / reliability part of the list.

[onestardao]

Hi, following up briefly.

The WFGY 2.0 RAG 16 Problem Map is now referenced or adopted by 20+ repos across the ecosystem, including RAGFlow, LlamaIndex, ToolUniverse (Harvard MIMS Lab), Rankify, and the Multimodal RAG Survey (QCRI LLM Lab), along with multiple curated awesome lists.

Given that traction, I wanted to reconfirm whether it would make sense to include it here as a debugging / evaluation resource.

If aligned, I can submit a minimal one-line PR in your preferred format.

Appreciate the work you’re doing on this list.