Skip to content

fix: address PR review for LLM hardening and API Docker import - #39

Closed
omakszewska wants to merge 4 commits into
Solvro:mainfrom
omakszewska:cypher-hardening
Closed

fix: address PR review for LLM hardening and API Docker import#39
omakszewska wants to merge 4 commits into
Solvro:mainfrom
omakszewska:cypher-hardening

Conversation

@omakszewska

@omakszewska omakszewska commented May 16, 2026

Copy link
Copy Markdown
Contributor

Review follow-up (fix: address PR review)

  • Share timeout messages via src/config/messages.py (fix API image import)
  • Implement DeepSeek/Google provider fallback; fix provider selection without OpenAI key
  • Use accurate_model for Cypher, fast_model for guardrails
  • Strip quoted strings before Cypher write-keyword validation

This PR improves safety and reliability in the knowledge-graph RAG pipeline:

Cypher write-operation guardrail
Validates generated Cypher before Neo4jGraph.query().
Blocks mutation keywords such as CREATE, MERGE, DELETE, DETACH, SET, REMOVE, and DROP.
Strips comments first and inspects uppercase tokens, so unsafe queries fail before execution with a controlled error.
LLM call timeouts
Adds explicit timeout values to LangChain LLM clients (BaseChatOpenAI / ChatGoogleGenerativeAI).
Wraps async pipeline entry points with asyncio.wait_for() so a single request cannot hang indefinitely.
Uses a separate wall-clock budget for the full LangGraph run where applicable.
LLM retry with exponential backoff
Wraps chain.invoke() / chain.ainvoke() used by generate_cypher and guardrails_system with Tenacity.
Uses 3 attempts with exponential backoff and jitter for transient failures such as rate limits and temporary API errors.
Provider fallback is intentionally left as follow-up TODO; startup-time provider selection remains unchanged.
Guardrail output as strict JSON
Guardrail prompt now requires strict JSON output: {decision: generate_cypher} or {decision: end}.
Parser strips optional code fences, extracts JSON from noisy text, parses it safely, and normalizes decisions via aliases.
Parse failures default to end, keeping LangGraph routing aligned with guardrail_decision.

@omakszewska
omakszewska force-pushed the cypher-hardening branch 2 times, most recently from af74e0d to 0620c3d Compare May 16, 2026 19:03
@omakszewska

Copy link
Copy Markdown
Contributor Author

Superseded by the unified PR on fix/llm-cypher-hardening-unified (same hardening scope: Cypher guards, timeouts, provider fallback without tenacity, JSON guardrails, tests, client/API wiring). Closing in favor of that PR.

@omakszewska omakszewska closed this Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant