Skip to content

feat: Add context-hub haystack connector #651

@AlexMikhalev

Description

@AlexMikhalev

Summary

Add a new haystack connector for context-hub (Andrew Ng's curated API docs for coding agents) to complement the existing grepapp_haystack and query_rs haystacks.

Motivation

Terraphim's haystack architecture already covers two code knowledge sources:

Source Status What it searches
grep.app Implemented (grepapp_haystack) Code across ~1M GitHub repos -- "How do others use this API?"
query.rs Implemented (query_rs haystack) Rust std lib, crates, type signatures, error codes, Clippy lints, books -- "What is the Rust API for X?"
context-hub Proposed Curated, versioned API docs (Stripe, OpenAI, etc.) with agent annotations -- "Give the agent correct, hallucination-free API docs"

Context-hub fills a gap neither grep.app nor query.rs covers: curated, language-aware API documentation specifically designed for AI coding agents. It provides incremental doc fetch (token-efficient), community-maintained quality, language variants (Python/JS), and persistent annotations.

How context-hub differs from existing haystacks

Dimension grep.app query.rs context-hub
Data Source code Rust docs, types, lints Curated API docs (multi-language)
Query model Text/regex Prefixed search + type signatures Hierarchical IDs + chub search
Freshness Live index of GitHub Synced with Rust releases Community-maintained PRs
Privacy Public repos only Public docs Public repo
Best for Usage examples Rust API reference, error lookup Agent-friendly API context for any language
Stack Web API Web API (scraping + JSON) CLI (chub) + markdown repo

Proposed Design

chub_haystack

  • Implement IndexMiddleware trait (same pattern as grepapp and query_rs haystacks)
  • Wrap chub search and chub get CLI or replicate the search/fetch protocol directly via HTTP
  • Map context-hub markdown + YAML frontmatter to Terraphim Document types
  • Preserve language variants (Python/JS) as document metadata
  • Optionally bridge annotations back to Terraphim's learning capture system
  • Feature-gate behind context-hub flag in terraphim_middleware

Acceptance Criteria

  • chub_haystack module in crates/terraphim_middleware/src/haystack/
  • Implements IndexMiddleware trait
  • Feature flag context-hub in middleware Cargo.toml
  • Integration tests with real queries (not mocks)
  • Config support in role configs (similar to grepapp/query_rs entries)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions