test(server-test): CLI e2e tests + field-specific search examples in generated docs#916
Merged
pyramation merged 9 commits intomainfrom Mar 28, 2026
Merged
Conversation
Tests generated CLI commands (codegen → transpile → execute) against a running PostgreSQL database with a real GraphQL server. - Uses ts.transpileModule to strip types without resolving imports - Uses async spawn (not execFileSync) to keep event loop unblocked - Sets up appstash context pointing at test server endpoint - Resolves NODE_PATH for pnpm's strict module isolation 5 focused corner-case tests: 1. Paginated list with --where (dot-notation) + --fields 2. Cursor-based forward pagination (--after) 3. find-first with --where.name.equalTo 4. Combined --where + --orderBy + --fields 5. Empty result set handling
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
…SE_DIR, bump appstash 0.7.0 - Type buildAnimalsTable() with proper Table interface from codegen - Replace internal path.join require hacks with proper package exports - Use APPSTASH_BASE_DIR env var instead of overriding HOME - Bump appstash to ^0.7.0 for APPSTASH_BASE_DIR support - Fix cliEntryPoint -> entryPoint config property
Suite 2 — Search CLI (6 tests): - tsvector search via --where.tsvTsv (dot-notation passthrough) - trgm fuzzy matching via --where.trgmTitle - composite fullTextSearch filter - search + pagination (--limit) - pgvector similarity (conditional, skip if unavailable) - _meta query from live server (MetaSchemaPlugin verification) Uses search-seed fixture (5 articles with tsvector, pg_trgm, optional pgvector). All search tests use list --where dot-notation to pass filter field names directly to the server, testing the full pipeline: codegen -> transpile -> spawn child process -> ORM findMany -> GraphQL -> real PostgreSQL.
- pgvector test: vector arrays can't be passed via CLI dot-notation (they become strings, not JSON arrays). Changed test to verify the CLI reports a clear GraphQL type error rather than crashing silently. - _meta test: replaced with schema introspection test since the search-seed server doesn't load MetaSchemaPlugin (enableServicesApi is false). New test verifies Article type exposes expected search fields (tsvRank, titleTrgmSimilarity, bodyTrgmSimilarity, searchScore, and conditionally pgvector fields).
The CLI exits with code 0 even on GraphQL errors, returning
{ ok: false, errors: [...] }. Updated the test to check the
response content instead of expecting the promise to reject.
…eral for runner script - Moved runCli() from inline per-suite to a shared module-level function that takes (distDir, tmpHome, ...args) — eliminates 60 lines of duplication - Replaced string[] array joined with newlines with a readable template literal (RUNNER_SCRIPT constant) - Fixed duplicate JSDoc comment on setupAppstashContext - Removed stale Suite 3 reference from file header
When a table has search-capable fields (tsvector, trgm, BM25, pgvector), the generated README and skill references now include concrete CLI examples showing the exact dot-notation flags for each search type: - tsvector: --where.<field> "query" - trgm: --where.trgm<Base>.value "query" --where.trgm<Base>.threshold 0.3 - BM25: --where.bm25<Base>.query "query" - pgvector: --where.<field>.vector '[...]' --where.<field>.distance 1.0 - composite: --where.fullTextSearch "query" Also adds a combined search + pagination example. Field-name derivation mirrors buildSearchHandler so examples always match the generated code. Integrated into all four generators: single-target README, single-target skills, multi-target README, and multi-target skills.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds 11 end-to-end tests (2 suites) that exercise the full CLI pipeline: codegen → transpile → execute as child process against a running PostgreSQL database with a real GraphQL server.
Also adds field-specific search examples to all generated CLI docs so that agents and users can see concrete dot-notation flags for each search type (tsvector, trgm, BM25, pgvector, composite fullTextSearch).
CLI E2E Tests
Approach (Approach A):
getConnections()spins up a real Postgres DB + GraphQL HTTP servergenerateCli()andgenerateOrm()produce TypeScript source filests.transpileModulestrips types without resolving imports (avoids needing all type packages in a temp dir)APPSTASH_BASE_DIRpointing at a temp directory with the test server endpointspawn, notexecFileSync) runs the compiled CLI against the live serverWhy
spawninstead ofexecFileSync: The GraphQL server runs in the same Node.js process. Synchronous child execution blocks the event loop, preventing the server from responding — causing a deadlock/timeout.Suite 1 — Animals (simple-seed, 5 tests):
--where(dot-notation) +--fieldsprojection--limit+--after)find-firstwith--where.name.equalTo--where+--orderBy+--fieldsSuite 2 — Search CLI (search-seed, 6 tests):
--where.tsvTsv(dot-notation passthrough to server's actual filter field)--where.trgmTitle.value+--where.trgmTitle.thresholdfullTextSearchfilter via--where.fullTextSearch--where.tsvTsv+--limit){ok: false, errors: [...]}when vector arrays are passed via dot-notation (string coercion, not JSON arrays — a known CLI limitation for complex types)Package.json changes: Added
@0no-co/graphql.web,gql-ast,appstash(^0.7.0),inquirerer, andnested-objas devDependencies inserver-test— these are ORM/CLI runtime deps needed by the child process.Generated Search Examples in CLI Docs
New
buildSearchExamples()andbuildSearchExamplesMarkdown()helpers indocs-utils.ts. When a table has search-capable fields, generated README and skill references now include concrete examples:--where.<field> "search query" --fields title,tsvRank--where.trgm<Base>.value "query" --where.trgm<Base>.threshold 0.3--where.bm25<Base>.query "search query"--where.<field>.vector '[...]' --where.<field>.distance 1.0(with CLI limitation note)--where.fullTextSearch "query"(dispatches to all text adapters)Field-name derivation mirrors
buildSearchHandlerintable-command-generator.tsso examples always match the generated code. Integrated into all four generators: single-target README, single-target skills, multi-target README, and multi-target skills. Replaces the previous single genericsearch "query text"example.Updates since last revision
runClihelper: MovedrunCli()from inline per-suite to a shared module-level function taking(distDir, tmpHome, ...args)— eliminates ~60 lines of duplication between Suite 1 and Suite 2.[...].join('\n')array-of-strings approach with a readableRUNNER_SCRIPTtemplate literal constant.setupAppstashContext.Review & Testing Checklist for Human
buildSearchExamplesfield-name derivation duplicatesbuildSearchHandlerlogic: The BM25 (bodyBm25Score → bm25Body) and trgm (titleTrgmSimilarity → trgmTitle) name-mapping regexes indocs-utils.tsare copy-pasted fromtable-command-generator.ts. If one changes, the other will silently produce wrong examples. Consider whether this should be a shared utility.title,tsvRankin generated examples:buildSearchExamplesusestitleas the example display field, which may not exist on all tables. Verify this is acceptable for example-quality docs or whether it should derive a field name from the table.{ok: false, errors: [...]}with exit code 0 when the server rejects a query. This is the current behavior being tested, not a bug introduced by this PR — but it may be worth considering whether the CLI should exit non-zero on server errors in a follow-up.buildArticlesTable()is hand-crafted (lines 609–751): Manually mirrors the search-seed schema. If the search plugin's naming conventions change or the schema.sql fixture evolves, this Table object will silently produce wrong generated code. Cross-check field names againstsearch-seed/schema.sqland the live introspection output.node.id === 'a0000001-0000-0000-0000-000000000001'— tied tosimple-seed/test-data.sql. Confirm this ID exists and won't change.Recommended test plan: Run
cd graphql/server-test && npx jest cli-e2e --verbosein CI (requires PostgreSQL) to confirm all 11 tests pass. Also verify the existing test suite still passes (pnpm test). For the doc generation changes, runcd graphql/codegen && pnpm testand inspect a generated skill reference for a search-enabled table to confirm the new examples render correctly.Notes
pnpm-lock.yamldiff is large but is purely formatting changes (single-line vs multi-line resolution objects) from a lockfile refresh — no dependency version changes beyondappstash@0.7.0.graphile-cacheERROR log during teardown (PostGraphile instance has been released) is pre-existing in other server-test suites and is harmless.APPSTASH_BASE_DIRenv var support in appstash itself was merged and published as appstash@0.7.0 via dev-utils PR #75.Link to Devin session: https://app.devin.ai/sessions/c92c3a11450342f8875625a60fa1be28
Requested by: @pyramation