-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Open
Labels
Storage & DatabasesenhancementNew feature or requestNew feature or requesthelp wantedMaintainers would appreciate community help on this.Maintainers would appreciate community help on this.launch-schooltrio-tnt
Description
Problem Statement
Currently @mastra/pg only supports vector (full precision) and halfvec (half precision) types. pgvector 0.7.0+ also supports two additional storage types:
bit- Binary vectors using PostgreSQL's native bit type, useful for binary quantization which significantly reduces storage and improves search speedsparsevec- Sparse vectors that only store non-zero elements, useful for BM25/TF-IDF representations and other sparse embeddings
Proposed Solution
- Add bit and sparsevec to the
VectorTypeunion type - Implement new distance metrics:
- For
bit: Hamming distance (<~>) and Jaccard distance (<%>) - For
sparsevec: L2, cosine, and inner product (same as vector)
- Add appropriate operator classes:
bit_hamming_ops,bit_jaccard_ops
sparsevec_l2_ops,sparsevec_ip_ops,sparsevec_cosine_ops - Implement dimension/element limits:
bit: up to 64,000 dimensions for indexessparsevec: up to 1,000 non-zero elements for indexes
- Handle index type restrictions:
IVFFlat: supportsbitwith Hamming distance only (bit_hamming_ops), does NOT support Jaccard orsparsevecHNSW: supportsbit(both Hamming and Jaccard) andsparsevec(L2, cosine, inner product)
Component
RAG
Additional Context
The recent PR #11002 added halfvec support and provides a pattern for implementation
Verification
- I have searched the existing issues to make sure this is not a duplicate
- I have provided sufficient context for the team to understand the request
coderabbitai
Metadata
Metadata
Assignees
Labels
Storage & DatabasesenhancementNew feature or requestNew feature or requesthelp wantedMaintainers would appreciate community help on this.Maintainers would appreciate community help on this.launch-schooltrio-tnt