feat(postgres): support pgvector columns (vector, halfvec, sparsevec)

[jonatannietoa]

What

Columns created with the pgvector extension were unusable in Tabularis: they showed up with type USER-DEFINED and every value rendered as null. This adds first-class support for vector, halfvec and sparsevec across reading, metadata, writing and the data-grid UI.

Why it was broken

pgvector types are extension base types with dynamic OIDs, so they matched none of the built-in Type::* arms in the value extractor and fell through to JsonValue::Null. Separately, information_schema.columns.data_type reports any extension type as the literal string "USER-DEFINED" (the real name lives in udt_name), so the column header showed that instead of vector.

How

Backend (src-tauri/src/drivers/postgres)

• extract/advanced_types.rs — decoders for vector, halfvec and sparsevec following pgvector's *_send binary layouts (incl. an IEEE-754 half-precision decode for halfvec). Values render as their canonical text form ([1,2,3], {1:1.5,3:2}/5), so a 1536-dim embedding round-trips exactly.
• extract/simple.rs — name-guarded match arms route the three types to the new decoders instead of the null fallthrough. Arrays (vector[]) work automatically via the existing element dispatch.
• mod.rs — get_columns / get_all_columns_batch fall back to udt_name (CASE WHEN data_type = 'USER-DEFINED' THEN udt_name … END) so columns report vector.
• binding.rs — on write, vector values are inlined as '<value>'::vector. pgvector registers no text -> vector cast, so a bound TEXT parameter is rejected; the value must reach the type's input function as an unknown-typed literal. Input is validated against a strict numeric/bracket character allow-list before inlining.

Frontend

• src/utils/text.ts — new isVectorColumn(); isLongTextCellTarget now treats vector columns as long-text so the grid routes them to the readable, expandable TextCell preview instead of dumping the raw string into a CSS-clipped <td> (which collapsed to a bare …).
• src/components/ui/FieldEditor.tsx — vector columns get the multiline text editor.

Testing

• Rust: 4 new unit tests in extract/simple.rs (vector, empty vector, halfvec, sparsevec). Full drivers::postgres suite: 250 passed.
• Frontend: new isVectorColumn / vector isLongTextCellTarget cases in tests/utils/text.test.ts (23 passed); tsc --noEmit and ESLint clean.
• Manual: verified against a live DB with CREATE EXTENSION vector — a vector(1536) embedding column now shows type vector and a readable, expandable preview instead of null.

Notes

• Float formatting uses Rust's shortest round-trippable representation; values are numerically identical to pgAdmin, though very small magnitudes render in decimal (0.000019073486) rather than scientific notation (1.9073486e-05).
• No dependencies added; Cargo.lock left untouched.

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (8 files)

• src-tauri/src/drivers/postgres/binding.rs — pgvector value binding with strict character allow-list validation
• src-tauri/src/drivers/postgres/extract/advanced_types.rs — binary decoders for vector, halfvec, and sparsevec
• src-tauri/src/drivers/postgres/extract/simple.rs — dispatch arms and unit tests for pgvector types
• src-tauri/src/drivers/postgres/mod.rs — metadata queries fall back to udt_name for USER-DEFINED types
• src-tauri/src/drivers/postgres/tests.rs — round-trip binding tests for pgvector literals
• src/components/ui/FieldEditor.tsx — vector columns routed to multiline text editor
• src/utils/text.ts — isVectorColumn helper and isLongTextCellTarget integration
• tests/utils/text.test.ts — unit tests for vector column detection and long-text routing

Previous Review Summary (commit 5030586)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 5030586)

Status: No Issues Found | Recommendation: Merge

Files Reviewed (7 files)

• src-tauri/src/drivers/postgres/binding.rs — pgvector value binding with strict character allow-list validation
• src-tauri/src/drivers/postgres/extract/advanced_types.rs — binary decoders for vector, halfvec, and sparsevec
• src-tauri/src/drivers/postgres/extract/simple.rs — dispatch arms and unit tests for pgvector types
• src-tauri/src/drivers/postgres/mod.rs — metadata queries fall back to udt_name for USER-DEFINED types
• src/components/ui/FieldEditor.tsx — vector columns routed to multiline text editor
• src/utils/text.ts — isVectorColumn helper and isLongTextCellTarget integration
• tests/utils/text.test.ts — unit tests for vector column detection and long-text routing

---

_{Reviewed by step-3.7-flash · Input: 84.5K · Output: 36.4K · Cached: 1M}

[NewtTheWolf]

Really nice work — the *_send binary decoders are careful (the f16 zero/subnormal/inf/nan branches and the 1-based sparsevec index printing are all correct and well-tested), and inlining behind a strict character allow-list is the right way to handle the missing text -> vector cast. 🙏

One blocking issue inline: the write path never reaches bind_pg_vector_string, so editing a vector value doesn't actually work yet (reading does). Details + fix inline.

Non-blocking:

• Plain views still show USER-DEFINED: get_view_columns (mod.rs:1112) selects bare c.data_type, so a vector column in a regular view won't get the CASE fallback. Materialized views are fine (they use format_type). Consistency nit.
• Cross-PR heads-up: #427 (hstore) also branches on data_type == "USER-DEFINED"; once this CASE lands that value becomes "hstore", so whichever of the two merges second will need to reconcile that seam. Flagging since they touch the same spot.

Also: the branch currently has a merge conflict against main (mergeStateStatus: DIRTY) — could you rebase/resolve that when you get a chance? 🙏

[NewtTheWolf]

Re-reviewed after the latest push (9e2ed25). All blocking feedback is addressed:

• Write path now reaches the vector binding: get_column_data_type applies the USER-DEFINED → udt_name fallback (mod.rs:529), so update/insert bind typed vector literals.
• Plain views resolve the real type via the CASE in get_view_columns (mod.rs:1194).
• Round-trip binding tests through bind_pg_value added, including rejection of non-numeric/injection literals.
• Cleanly rebased on current main.

Follow-up (non-blocking): #427 (hstore) also branches on data_type == 'USER-DEFINED' — whichever of the two merges second needs to reconcile that seam. LGTM.