fix(extract): guard against large SQL file stack overflow (#691)

[lg320531124]

Summary

The tree-sitter SQL grammar (39 MB parser.c) uses deeply recursive non-terminals that overflow the C stack on files with many statements (e.g. schema dumps >10K lines, stored procedures). The stack overflow kills the thread before any timeout callback can fire, so a pre-parse guard is the only safe mitigation.

Changes

• Add a line-count guard (CBM_SQL_MAX_LINES = 5000) in cbm_extract_file() that returns has_error for SQL files exceeding the threshold
• The pipeline skips the file gracefully instead of crashing

Why line-count, not byte-size?

• tree-sitter's recursive descent parser pushes a stack frame per statement
• A 500KB SQL dump with 10K CREATE TABLE lines overflows the default 512KB macOS thread stack
• A 2MB compressed binary blob with few newlines does NOT overflow
• Line count is the correct proxy for SQL stack depth

Why 5000 lines?

• Default macOS thread stack: 512 KB
• tree-sitter SQL frame size: ~100 bytes per recursion
• 5000 lines → ~500 KB stack usage (safe margin)
• Captures virtually all crash-inducing workloads

Why in cbm_extract_file(), not the discover layer?

• The discover layer only checks byte size and knows nothing about SQL
• The guard must be where tree-sitter is invoked — that is where the crash happens
• Keeps the change minimal and colocated with the dangerous call

Testing

• All 5720 existing tests pass
• Small SQL files (<5000 lines) parse normally
• Large SQL files (6000 lines) are skipped gracefully with has_error=true

Fixes #691

Related: #668 (original report)

[DeusData]

Huge thanks for opening this PR and for the work you put into it.

The maintainer shop is currently full, so this may sit for a bit before it gets a proper review. We will come back to this as soon as possible with real feedback; I wanted to make sure it did not sit unacknowledged in the meantime.

[DeusData]

Thank you @lg320531124 — you're right that a large .sql file crashing the whole index run (#668, of which #691 is a duplicate) is a real, high-priority problem, and reaching for a pre-parse guard is the right instinct.

After analysis we're not going to take the line-count cap, because line count is the wrong signal for this crash: a recursive/GLR parser overflows on depth, not size — a 20-line but thousands-deep-nested statement still crashes and sails under 5000 lines, while a legitimately large-but-flat 6000-line migration parses fine and would be dropped by this cap. It happens to stop the one reported schema dump, but by coincidence (that file being both big and a real crasher), not by addressing the mechanism.

Instead we've planned a proper, general fix (reproduce-first): (1) root-cause the actual tree-sitter SQL crash and fix it at the grammar/scanner level so it stops crashing on all platforms; and (2) a general resilience layer so that any file that fails during indexing — any language, parse or LSP — is skipped-and-reported (per-run logfile + surfaced via MCP/CLI) instead of aborting the run, with a cross-platform process supervisor so a hard crash can never take the whole run down. That resolves #668 and the whole class of similar issues.

Closing this in favor of that effort — genuinely appreciate the report and the push on it. 🙏