BE-614: HashQL: Generate synthetic closure bodies for intrinsics used as first-class values#8895
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
2 Skipped Deployments
|
PR SummaryMedium Risk Overview Reification now routes qualified variables through Small supporting changes: Reviewed by Cursor Bugbot for commit 7801f77. Bugbot is set up for automated code reviews on this repo. Configure here. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure #8895 +/- ##
======================================================================================================
- Coverage 40.65% 24.83% -15.82%
======================================================================================================
Files 791 663 -128
Lines 70641 50119 -20522
Branches 3992 3379 -613
======================================================================================================
- Hits 28719 12448 -16271
+ Misses 41590 37514 -4076
+ Partials 332 157 -175
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Merging this PR will improve performance by 38.94%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ❌ | pattern_match_constant |
150.8 ns | 180 ns | -16.2% |
| ⚡ | runtime |
58.9 ns | 29.7 ns | +98.13% |
| ⚡ | constant |
58.9 ns | 29.7 ns | +98.13% |
| ⚡ | type_checker_simulation |
497.5 ns | 439.2 ns | +13.28% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing bm/be-614-hashql-generate-synthetic-mir-bodies-for-intrinsics-in-value (7801f77) with bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure (c836d70)
538afac to
88472a7
Compare
| /// Constructs a `&'static [ConstantSymbol]` path from `::` separated segments. | ||
| /// | ||
| /// Each segment maps to the corresponding `sym::` constant. | ||
| /// | ||
| /// ```ignore | ||
| /// T![::core::math::add] // expands to &[sym::core, sym::math, sym::add] | ||
| /// T![::core::cmp::eq] // expands to &[sym::core, sym::cmp, sym::eq] | ||
| /// ``` |
88472a7 to
d3b03b8
Compare
ac8ed9e to
4aacfd8
Compare
4aacfd8 to
f2b45cb
Compare
d3b03b8 to
015b737
Compare
f2b45cb to
489dd1a
Compare
015b737 to
0694d8d
Compare
489dd1a to
5628792
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 4580488. Configure here.
| /// Constructs a `&'static [ConstantSymbol]` path from `::` separated segments. | ||
| /// | ||
| /// Each segment maps to the corresponding `sym::` constant. | ||
| /// | ||
| /// ```ignore | ||
| /// T![::core::math::add] // expands to &[sym::core, sym::math, sym::add] | ||
| /// T![::core::cmp::eq] // expands to &[sym::core, sym::cmp, sym::eq] | ||
| /// ``` |
| /// ```ignore | ||
| /// T![::core::math::add] // expands to &[sym::core, sym::math, sym::add] | ||
| /// T![::core::cmp::eq] // expands to &[sym::core, sym::cmp, sym::eq] | ||
| /// ``` |
| /// Constructs a `&'static [ConstantSymbol]` path from `::` separated segments. | ||
| /// | ||
| /// Each segment maps to the corresponding `sym::` constant. | ||
| /// | ||
| /// ```ignore | ||
| /// T![::core::math::add] // expands to &[sym::core, sym::math, sym::add] | ||
| /// T![::core::cmp::eq] // expands to &[sym::core, sym::cmp, sym::eq] | ||
| /// ``` |
| impl From<ConstantSymbol> for Symbol<'_> { | ||
| #[inline] | ||
| #[expect(unsafe_code)] | ||
| fn from(value: ConstantSymbol) -> Self { | ||
| // SAFETY: The constant symbol is already interned, so the repr is valid. | ||
| unsafe { Symbol::from_repr(Repr::constant(value.repr)) } | ||
| } | ||
| } |
| /// ```ignore | ||
| /// T![::core::math::add] // expands to &[sym::core, sym::math, sym::add] | ||
| /// T![::core::cmp::eq] // expands to &[sym::core, sym::cmp, sym::eq] | ||
| /// ``` |
| /// Constructs a `&'static [ConstantSymbol]` path from `::` separated segments. | ||
| /// | ||
| /// Each segment maps to the corresponding `sym::` constant. | ||
| /// | ||
| /// ```ignore | ||
| /// T![::core::math::add] // expands to &[sym::core, sym::math, sym::add] | ||
| /// T![::core::cmp::eq] // expands to &[sym::core, sym::cmp, sym::eq] | ||
| /// ``` |
Benchmark results
|
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2002 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 1002 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 3314 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 1527 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 2078 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 1033 | Flame Graph |
policy_resolution_medium
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 102 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 269 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 108 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 133 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 63 | Flame Graph |
policy_resolution_none
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 2 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 8 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 3 | Flame Graph |
policy_resolution_small
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| resolve_policies_for_actor | user: empty, selectivity: high, policies: 52 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: empty, selectivity: medium, policies: 26 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: high, policies: 94 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: seeded, selectivity: medium, policies: 27 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: high, policies: 66 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: low, policies: 1 | Flame Graph | |
| resolve_policies_for_actor | user: system, selectivity: medium, policies: 29 | Flame Graph |
read_scaling_complete
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id;one_depth | 1 entities | Flame Graph | |
| entity_by_id;one_depth | 10 entities | Flame Graph | |
| entity_by_id;one_depth | 25 entities | Flame Graph | |
| entity_by_id;one_depth | 5 entities | Flame Graph | |
| entity_by_id;one_depth | 50 entities | Flame Graph | |
| entity_by_id;two_depth | 1 entities | Flame Graph | |
| entity_by_id;two_depth | 10 entities | Flame Graph | |
| entity_by_id;two_depth | 25 entities | Flame Graph | |
| entity_by_id;two_depth | 5 entities | Flame Graph | |
| entity_by_id;two_depth | 50 entities | Flame Graph | |
| entity_by_id;zero_depth | 1 entities | Flame Graph | |
| entity_by_id;zero_depth | 10 entities | Flame Graph | |
| entity_by_id;zero_depth | 25 entities | Flame Graph | |
| entity_by_id;zero_depth | 5 entities | Flame Graph | |
| entity_by_id;zero_depth | 50 entities | Flame Graph |
read_scaling_linkless
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | 1 entities | Flame Graph | |
| entity_by_id | 10 entities | Flame Graph | |
| entity_by_id | 100 entities | Flame Graph | |
| entity_by_id | 1000 entities | Flame Graph | |
| entity_by_id | 10000 entities | Flame Graph |
representative_read_entity
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1
|
Flame Graph | |
| entity_by_id | entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1
|
Flame Graph |
representative_read_entity_type
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| get_entity_type_by_id | Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba
|
Flame Graph |
representative_read_multiple_entities
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| entity_by_property | traversal_paths=0 | 0 | |
| entity_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| entity_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=0 | 0 | |
| link_by_source_by_property | traversal_paths=255 | 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true | |
| link_by_source_by_property | traversal_paths=2 | 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true |
scenarios
| Function | Value | Mean | Flame graphs |
|---|---|---|---|
| full_test | query-limited | Flame Graph | |
| full_test | query-unlimited | Flame Graph | |
| linked_queries | query-limited | Flame Graph | |
| linked_queries | query-unlimited | Flame Graph |


🌟 What is the purpose of this PR?
Allows intrinsics to be used in value position (passed as arguments, bound to variables) by generating synthetic MIR wrapper bodies during reification. Previously, intrinsics like
<=could only appear at call sites where specialization rewrites them. Now they can flow through higher-order functions:🔍 What does this change?
Synthetic body generation:
Syntheticsstruct inCrossCompileStatemanages creation and caching of wrapper bodiesSyntheticBuildergenerates MIR bodies with fat closure ABI (unit env as first parameter)T!macro for matching qualified paths againstConstantSymbolarrays:&T![::core::cmp::gt]in slice patterns,sym::path::core::cmp::gt::CONSTfor the namebinary!/unary!macros deriving path and name from the same segmentsThin-call specialization:
rvalue_call_thin_specializerecognizesCall(Thin, Qualified(intrinsic), [])(the administrative thunk-force from the thunking phase) and produces the closure aggregate directly, skipping thunk body generation() -> ClosureTypewrapper to extract the actual signatureIntrinsic classification:
gt,lt,gte,lte,eq,ne), boolean (and,or,not), arithmetic (add,sub), bitwise (and,or,not)!payload):mul,div,rem,mod,pow,xor,shl,shrsqrt,cbrt,rootgraph::head::entities,graph::body::filter,graph::tail::collectproduce a specific user-facing diagnosticDiagnostics:
intrinsic_not_first_class: user-facing error for graph intrinsics in value positionsynthetic_binary_arity_mismatch/synthetic_unary_arity_mismatch: ICE for monomorphized type invariant violationsPre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
"<="as the entire program) is not supported. The thunking phase does not wrap the root body when it is a variable, so it never reaches the synthetic machinery. Tracked with arun: skiptest.🐾 Next steps
🛡 What tests cover this?
Compiletests (8 new):
reify/synthetic-intrinsic-value: basic synthetic body generation and thin-call specializationreify/synthetic-intrinsic-reused: same intrinsic used twice shares one body (caching)reify/synthetic-intrinsic-multiple: different intrinsics get separate bodiesreify/synthetic-graph-not-first-class: graph intrinsic in value position produces errorreify/synthetic-intrinsic-bare: bare intrinsic at root (run: skip)interpret/synthetic-intrinsic-value: end-to-end correctness (evaluates totrue)post_inline/synthetic-intrinsic-hof: full optimization cascade with constants (collapses toreturn true)post_inline/synthetic-intrinsic-hof-dynamic: HOF with dynamic inputs collapses to barex <= y❓ How to test this?
cargo run --package hashql-compiletest -- run --filter "test(synthetic)"📹 Demo
The
post_inline/synthetic-intrinsic-hof-dynamictest shows the full cascade. A higher-order function receiving<=as a value with dynamic inputs:All indirection eliminated. Indistinguishable from writing
["<=", x, y]directly.