feat(assistant): integrity core + RAG foundation for the AI assistant#80
Conversation
First implementation increment of docs/spec/ai-assistant.md, focused on the highest-priority §9 hardening items plus the RAG layer. Pure, tested and deploy-independent — no new deps/bindings — so it can land and be reviewed before the agent loop / dock UI (which need BGGPT_API_KEY + cloud bindings). - report-schema.ts: server-owned report values (§9.1) — the model references result handles, the server re-binds real numbers; tables take rows wholesale; prose is HTML-sanitized (closes stored-XSS on the public /reports/:id). - sql-guard.ts: read-only structural guard + injected LIMIT + byte cap (§7/§9.4) with the AST-parser + read-only-binding primary guards documented as next. - describe-schema.ts: curated data dictionary encoding the real data traps (amount vs amount_eur, value_flag, ocid≠UNP, lots grain) (§9.2). - rag.ts: Vectorize + Workers-AI (bge-m3) — schema-grounding retrieval and a semantic_search tool. Deliberate addition over the SQL-only spec. - tests for the value-binding and SQL guard (17 new; web suite 75 green). See apps/web/app/lib/assistant/README.md for the architecture and roadmap.
Continues the foundation with two pure, tested modules that feed the agent loop: - system-prompt.ts: encodes the runtime policies — emit_report policy (§9.10), values-by-reference (§9.1), data-trust / no-instructions-in-data (§7), the editorial skeleton (§4) and per-source freshness (§9.7); injects RAG schema context with a static-dictionary fallback. - tool-results.ts: bridges D1 .all() rows to handled, byte-capped QueryResults (R1, R2 …) that the report binder re-binds from (§7). 10 new tests; web suite 85 green; typecheck 0; prettier clean.
…ize cap) Live ЦАИС ЕОП day-query tool, hardened per spec §9.7: takes only a validated date (never a model-supplied URL → no SSRF), reuses the verified eopSource URL builder, caps each file before it reaches the model context, and treats the payload as untrusted. Network call injected for testability. 7 new tests; web suite 92 green; typecheck 0; prettier clean.
CI `pnpm audit --audit-level=high` began failing repo-wide on a newly published undici advisory (GHSA-vmh5-mc38-953g / -vxpw-j846-p89q / -hm92-r4w5-c3mj) — DoS via WebSocket fragment-count bypass and SOCKS5 proxy pool reuse — pulled in transitively via wrangler→miniflare (undici 7.24.8). Dev/build-time only; never ships to the Worker runtime. Add an `undici: ^7.28.0` override alongside the existing ws/vite pins. After the bump `pnpm audit --audit-level=high` exits 0 (1 low remains, below the gate). Lockfile updated to undici@7.28.0; nothing else changes.
Implements spec §4's two renderer rules as pure helpers the /reports/:id renderer will consume: - formatCell(value, hint) → delegates to @sigma/shared money/count/pct/date, so reports match native page formatting (no drift). - entityHref(kind, id) → canonical internal href via @sigma/db hrefForEntity. 4 new tests; web suite 96 green; typecheck 0; prettier clean.
Two more pure helpers: - source-link.ts: grounded official deep links (ЦАИС ЕОП procedure + open-data day files via the verified eopSource helper). Trade Register / АОП links are deferred — won't ship an unverified "official source" URL for a gov tool. - emit-report-schema.ts: validateEmitShape (structural guard that runs before bindReport's handle resolution) + the model-facing EMIT_REPORT_JSON_SCHEMA. 10 new tests; web suite 106 green; typecheck 0; prettier clean.
Starts the agent-loop layer with its substance, kept dependency-free and tested: - tools.ts: describe_schema, run_sql (guard → LIMIT → D1, retains the result under a handle), semantic_search, eop_fetch, source_link — each runs server-side and returns a compact string; data tools retain full results in ctx.results for binding. Plus runTool dispatcher and finalizeReport (validateEmitShape → bindReport against THIS turn's results only — §9.1/§9.3). Remaining: the thin Vercel-AI-SDK wiring (streamText + /assistant/chat + provider via AI Gateway) — adds the deps/bindings and carries no logic. 9 new tests; web suite 115 green; typecheck 0; prettier clean.
The thin SDK layer that turns the tested foundation into a working chat endpoint: - agent.ts: BgGPT through @ai-sdk/openai (createOpenAI.chat) routed via the AI Gateway (§9.5); maps ASSISTANT_TOOLS → SDK tools (jsonSchema), adds emit_report wired to finalizeReport, runs streamText with stopWhen: stepCountIs(MAX_STEPS), returns the streamed Response. - routes/assistant.chat.tsx: stateless resource route — POST UIMessages, RAG-ground the prompt (best-effort), run one turn, stream back. - wrangler.jsonc: AI + VECTORIZE + REPORTS (R2) bindings + config vars; BGGPT_API_KEY stays a secret. - deps: ai ^6, @ai-sdk/openai ^3. Not runtime-verifiable here (needs BGGPT_API_KEY + the bindings), but typecheck 0, 115 tests, audit clean (no new high/moderate), prettier clean.
|
@nedda76 супер работа — чист, тестван fundament (integrity core + RAG) по §9 от спецификацията, при това deploy-независим (без нови bindings/ключове), така че може да се ревюира преди agent loop-а и облачните части. На практика това може да е Фаза 1 от асистента. Една бележка по координацията. Искаме да доставим асистента като един общ, дълготраен branch — Предложение: пренасочи #80 към За следващите фази — нека координираме в Discord-а на МИДТ, да си разпределим обхвата и да не дублираме. Благодаря много за усилията! |
|
Благодаря за прегледа и за рамкирането като Фаза 1! 🙏 Превключих #80 на ready for review и обнових README-то да отразява текущото състояние: backend-ът е опроводен от край до край (13 модула + tool registry + agent loop през Vercel AI SDK → AI Gateway → BgGPT + Една уговорка по координацията. Проверих съдържанието на Понеже #79 е docs-only и не пипа нито един файл, който #80 променя, бих предложила най-простия и чист път: и двете PR-та да влязат директно в След като и двете са в Благодаря отново — радвам се, че foundation-ът е полезен! 🙌 |
Extend the guarantees-vs-limits section with concrete guardrails that harden the residual data-correctness gaps, building on PR midt-bg#80's system-prompt/describe-schema foundation: default filters, reconcile- with-rollup self-check, explicit CPV interpretation, mandatory methodology callout, Verifier trap-compliance checks, and a golden- reports CI harness. Honesty (watermark + methodology) stays load-bearing.
|
@Bozhidar-Vangelov @teodorkirkov |
|
Сега ще погледна кода и сайта и ще напиша ако забележа нещо |
|
@Alben13579 Ако може да прегледаш дизайна на този ПР и като цяло дизайна на проекта. 🙏 |
|
@nedda76 прегледах кода
|
|
@experiment-bg Тук сме - един преглед и от теб моля и проверка на сайта. В момента доста ПР-и са се натрупали и може би е добра идея да изчакаме да се мърджнат, иначе голям рибейз ще падне. |
|
@nedda76 разгледах кода подробно Решението е премислено и се чете леко. Това, че единствено сървърът изписва числа — моделът само посочва хендъли, а
Иначе посоката е вярна и ядрото за достоверност е силно. За мен единствено спиращо е разменения таван в |
Clamp MAX_STEPS to [1, 20] so a misconfigured deploy can neither stall the tool loop (0/negative) nor uncap BgGPT calls (a huge value), and degrade gracefully on failure: a mid-stream BgGPT outage now surfaces as a readable message via the stream's onError, and a setup failure returns a 503 instead of an unhandled 500. Addresses review notes on PR midt-bg#80.
Exercise the boundary teodorkirkov raised on PR midt-bg#80: a tool/EOP/DB value carrying a fake instruction (e.g. "игнорирай предишните инструкции") must be treated as DATA, never as a command. Lock the system-prompt data-trust clause, that forModel serialises a poisoned cell verbatim inside the data payload, and that bindReport keeps it as a plain table cell. (Model-level resistance itself stays an eval concern — golden-report CI, §9.9.)
Make explicit that the new AI/Vectorize/R2 bindings reference resources that must exist before `wrangler deploy` — deploying first fails the deploy and blocks the team's CD (review note on PR midt-bg#80). Re-stage rate-limiting + circuit-breaker from the launch gate into Phase 2, note the v1 graceful degradation now in place, and refresh the test count.
Layer node-sql-parser (SQLite-only build) over the structural read-only check in run_sql: parse the statement and reject anything that is not a single read-only SELECT, failing closed on parse errors. Closes the gap flagged on PR midt-bg#80 that a regex/keyword guard is inherently incomplete. The structural guard stays the cheap first pass; the parser is the real gate. Deliberately fails closed — valid-but-unparsed SQLite (e.g. window functions without PARTITION BY) is refused, steering the model to the canonical ORDER BY + LIMIT pattern, which all canonical queries use and a regression test covers. A read-only D1 binding remains the open §9.4 layer.
|
@teodorkirkov Благодаря за внимателния преглед! 🙏 Адресирах бележките в четири commit-а на branch-а: 1. 2. 3. Rate-limiting / circuit-breaker. Преместих го от launch gate във Фаза 2, както предложи. За v1 добавих базова graceful degradation: mid-stream грешка от BgGPT се показва като четим текст през 4. 5. Проверено: typecheck 0, 127 теста минават, Prettier чист, |
|
Здравей @nedda76! Направихме задълбочен security ред-тийм на #80, спрямо самата спецификация в #79 (§9 hardening + agent-team addendum-а на @cefothe) — спрямо нашата обща висока летва. Нa първо място: имплементацията е силна. Проверих какво е изпълнимо (прекарах adversarial SQL през самите guard функции) и write-защитата издържа на всичко — Ред-тиймът извади набор от gap-ове — и важното: почти всеки е точно launch-gate изискване, което спецификацията вече описва, просто още не е достигнато в кода. По тежест, с file:line + repro + препратка към спецификацията: 🔴 Launch-gate (спецификацията ги изисква преди публично излагане)1. 2. → моделът може да изброи схемата и да чете всяка таблица в 3.
🟠 Бъгове / hardening4. 5. Bound data cells не се санитизират. 6. „Числа в prose" — точно guardrail E2. Binding-ът пази стойностните слотове, но 7. Без таван на историята/тялото + retry мултипликатор. 🟡 Low
Накратко: ядрото е добро и покрива „Guaranteed by construction" от addendum-а. Останалото е launch-gate слоят, който спецификацията ми описва (read-only път/allowlist, rate-limit + circuit-breaker, query timeout, E2) + два бъга (4, 5) — и всичко е в backend integrity core-а, който е твоят lane ( От наша страна мислим да поемем lane-овете, които консумират схемите ти, без да ги пипат — renderer-ът на |
Three integrity-core fixes from the midt-bg#80 security red-team, all at the binding layer where the guarantee belongs: - Resolve entity-link ids per row (ResolvedRow.links) and require the link idCol to exist. Without this an immutable R2 report could not rebuild its /companies/:eik links — the block-spec contract needs it. - Tag-strip submitter-influenceable string data cells at bind, not just prose, so no markup reaches the public /reports/:id even if a renderer forgets to escape (defence-in-depth, §7). Fixes the over-claiming comment in report-schema. - Add guardrail E2: a deterministic no-material-number-in-prose gate on text/callout (currency, млн/млрд, grouped numbers, 5+ digit integers; years/small counts/ordinals pass). The model must put numbers in value slots the server binds — closes the unbound-number defamation vector.
Freeze the three contracts the team agreed to lock first so FE and BE can work in parallel against fixtures: the block-spec (ResolvedReport, incl. the new per-row link ids), the immutable R2 report object (§5), and the SSE chat protocol (AI SDK UI message stream + the emit_report tool output that carries the report). Adds machine-readable JSON/SSE fixtures under app/lib/assistant/fixtures/. Source of truth stays the BE types.
The EOP_MAX_BYTES cap was a no-op: when the body exceeded the cap the code still JSON.parsed the FULL body and returned every row (the cap only set a 'truncated' flag). An oversized untrusted EOP file therefore reached the model in full. Now an over-cap body is refused with a soft error and never parsed. Strengthens the test to assert rows are withheld. (review midt-bg#80)
Чеклист за merge и активиранеDeferred launch-gate / eval елементите от прегледа вече се проследяват като отделни issues: Launch gate (блокират прод ключа):
Eval (§9.9, преди публичен launch):
Преди merge
Преди прод
|
|
📌 TL;DR провизиониране (пълните стъпки: чеклистът по-горе + Преди deploy създай ресурсите (Vectorize |
|
Благодаря за това — направих обстоен security преглед (SQL guard-овете, SSRF, интегритет на стойностите/XSS, edge/DoW и supply-chain/malicious-code скан), със сваляне на branch-а и PoC-ове срещу пиннатия Няколко неща, които бих добавил към тракера (нито едно не блокира merge-а): Струва си преди активиране
SQL guard — две крехки fail-closed разминавания (безопасни днес само защото SQLite случайно дава грешка):
Гейт за стойностите / рендиране
Egress / DoW (дребно)
Много добра основа — с удоволствие ще отворя issue-та за всяко от горните, ако е полезно. 🙌 |
|
@nedda76 Изглежда добре — съгласен съм, че merge-ът е безопасен при неактивиран Едно нещо, което не видях да е повдигано (не блокира merge, по-скоро за #135): CSRF / липса на same-origin проверка на |
Without an origin check, a cross-site page could make a victim's browser POST text/plain (a CORS simple request, no preflight) and start a paid BgGPT turn under the victim's IP. Reject non-POST / non-JSON / explicit cross-site requests before buffering the body — requiring application/json forces a preflight on cross-origin fetch and blocks <form> CSRF. Also pre-check Content-Length to reject an oversized body before buffering. Pure firstPartyRejection helper + tests; error matrix in assistant-contracts.md §3 updated. (review midt-bg#80, lyubomir-bozhinov; Content-Length: ydimitrof)
decodeNumericEntities ran a single pass, so a double-encoded number (1&midt-bg#38;midt-bg#50;000 -> 1&midt-bg#50;000) passed guardrail E2 while a renderer would decode it the rest of the way to a fabricated 12000 — the §9.1 vector the gate exists to close. Loop to a fixpoint (bounded). (review midt-bg#80, ydimitrof)
node-sql-parser accepts LIMIT 1e9 (type 'bigint') and 1.5, but enforceLimit's regex does not, so it appended a second LIMIT (LIMIT 1e9 LIMIT 500) — a SQLite syntax error that only failed closed by accident. Reject any LIMIT/OFFSET that is not a plain non-negative integer so the AST and regex text models agree. (review midt-bg#80, ydimitrof)
|
Благодаря на @lyubomir-bozhinov и @ydimitrof за двата security re-review! Адресирах трите изпълними бележки от 26.06 — head вече е Имплементирано (с тестове):
Проследено (non-blocking, pre-activation) → #155: fail-closed лимитер от runtime сигнал вместо build-time |
Both this PR and midt-bg#118 (/health) independently claimed namespace_id 1004. A duplicate silently disables one limiter (no deploy error) and git auto-merges the two bindings without conflict, so the dup would ship unnoticed. Take 1005 here (1004 stays with midt-bg#118's reviewed health limiter) to deconflict regardless of merge order.
Mirror the assistant-contract seam (report.ts, stream.ts, fixtures + spec) onto feat/ai-assistant. It re-exports ResolvedReport from app/lib/assistant/report-schema, which is present here now that midt-bg#80 has landed in main and feat is caught up. Closes the seam parity hole between feat/ai-assistant and feat/ai-assistant-contracts.
Adopt fork-main's canonical app/lib/assistant foundation (upstream midt-bg#80 is strictly ahead of contracts' midt-bg#80-head base) and deploy layer. wrangler limiter namespace_id 1004->1005 fixes a silent collision with the /health limiter. Keeps contracts' seam spec and .dev.vars assistant docs. seed-endpoint/reindex (contracts-only) preserved.
Mirror of #10 (approved) onto feat/ai-assistant-contracts. Identical lane content — L0-L3 dedup keys + freshness (dedup.ts), in-isolate single-flight coordinator (single-flight.ts), AI SDK v6 data-dedup/data-progress stream parts (dedup-stream.ts) — pure, self-contained, not yet wired. The only difference from #10 is the base: midt-bg#80 head here vs upstream main there. Part of midt-bg#97 (two identical fixed-period questions must never diverge).
…ontract - format AssistantPanel and AssistantTranscript after rebase conflict resolution - revert cell() to hard errors for missing column and out-of-range/non-integer row (facts/totals precision slots; model must retry with correct reference) - keep table display columns as warnings (graceful null rendering) - add hard-error check for missing link idCols in table blocks (structural requirement — immutable report cannot reconstruct entity links without them)
… meta canonical - contractSlug now encodes %, /, ?, # so any AOP contract number is safe as a URL path segment - entityHref in render-format.ts uses contractSlug directly for contracts (removes double-encode risk) and encodeURIComponent on the slug part for authorities/companies (closes path-traversal concern from review midt-bg#80) - meta canonical/og:url in contract.tsx re-encodes params.id via contractSlug(contractIdFromSlug()) — React Router decodes the param before the loader runs, so the raw id was being used previously - identity.test.ts covers the new ? and # cases and round-trips
Remove the bare '(review ...)' attribution notes I left in contract.json.tsx and sort-index-sentinel-sync.test.ts; the explanations stay. The pre-existing '(review midt-bg#80)' issue references elsewhere are an established convention and are untouched. Comment-only.
Какво е това
Backend основата на AI асистента на Sigma — имплементация на
docs/spec/ai-assistant.mdс хардунирането от §9 (PR #79). За разлика от ранната чернова, бекендът вече е опроводен от край до край: чисти тествани модули → tool registry → agent loop → ресурс route/assistant/chat(BgGPT през Cloudflare AI Gateway). Потребителският слой (dock UI, renderer на справките, глас) и provisioning-ът остават за следващи PR-та.Какво има (имплементирано и проверено)
emit_reportблоковете реферират хендъли към сървърно изпълнени резултати, аbindReport()пре-свързва реалните стойности. Прозата е markdown-санитизирана; guardrail E2 отказва едри числа в прозата (вкл. Unicode цифрови форми).node-sql-parser) с table-allowlist, забрана на TVF/cross-join/recursion, AST-достоверен LIMIT и scalar-fn denylist.bge-m3) — schema grounding +semantic_search.describe_schema,run_sql(+ Denial-of-Wallet rows-read бюджет, [Идея]: Denial-of-Wallet чрез run_sql: D1 таксува прочетени редове, не върнати (§7/§9.4 #122),semantic_search,eop_fetch(валидирана дата, fixed base — без SSRF),source_linkи терминалнияemit_report.abortSignal, drop на client-подадениsystem/toolсъобщения.Нови зависимости:
ai@^6,@ai-sdk/openai@^3,node-sql-parser@^5.Нови bindings/vars (wrangler.jsonc):
AI,VECTORIZE,REPORTS(R2),ASSISTANT_RATE_LIMITER+ config vars (BGGPT_*,MAX_STEPS,D1_ROWS_READ_BUDGET).Проверено:
pnpm --filter web typecheck→ 0; пълният web test suite минава;pnpm audit --audit-level=highчист; Prettier чист. (CI е авторитетът за актуалния брой тестове.)Това PR добавя bindings към Cloudflare ресурси, които трябва да съществуват преди
wrangler deploy. Освен това route-ът изпълняваrun_sqlв момента, в койтоBGGPT_API_KEYе наличен, докато D1 binding-ът е read-write — затова не задавайте продBGGPT_API_KEY, докато (1)run_sqlне работи срещу read-only D1 binding/реплика и (2) не е наложен глобален budget/circuit-breaker. Двуслойният SQL guard е defense-in-depth, не единствената бариера. Пълни детайли:apps/web/app/lib/assistant/README.md.Какво НЕ е тук (следва)
UI слоят (dock,
emit_reportrenderer +/reports/:id, глас), provisioning-ът, и launch-gate hardening: read-only D1 data path, глобален budget/circuit-breaker, freshness wiring, golden-report CI.eop_fetchзасега връща само брой редове на ден, не самите данни (probe за наличие/свежест).Имплементира части от спецификацията в #79.