Skip to content

BE-615: HashQL: Introduce the concept of synthetic closures in the MIR#8894

Open
indietyp wants to merge 8 commits into
bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-modulefrom
bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure
Open

BE-615: HashQL: Introduce the concept of synthetic closures in the MIR#8894
indietyp wants to merge 8 commits into
bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-modulefrom
bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure

Conversation

@indietyp

Copy link
Copy Markdown
Member

🌟 What is the purpose of this PR?

Introduces Source::Synthetic for compiler-generated wrapper bodies and extends administrative reduction to handle single-operation function bodies (TrivialClosure).

This is groundwork for making intrinsics usable in value position (e.g. passing + as an argument to a higher-order function). The Synthetic source variant will identify wrapper bodies that bridge non-first-class operations into callable closures, and TrivialClosure ensures they are always inlined away.

🔍 What does this change?

  • Adds Source::Synthetic(Symbol) to the MIR Source enum for compiler-synthesized operation wrappers
  • Adds ReductionKind::TrivialClosure to administrative reduction, recognizing single-basic-block bodies where a trivial prelude leads to a single non-call operation (Binary, Unary, Aggregate, Input, Load) whose result is returned
  • Synthetic bodies get InlineDirective::Always (same as Ctor)
  • All downstream Source matches updated: execution pipeline, statement placement, eval context, pretty printer
  • Adds [synthetic sym::path] source to the body! test macro

The TrivialClosure classification is intentionally broader than just synthetic bodies. Any function matching the shape is reducible, which unlocks optimization cascades: single-operation closures get inlined, exposing constant-foldable expressions, enabling branch elimination downstream.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

🐾 Next steps

  • Implement qualified_path in reification to generate synthetic wrapper bodies for intrinsics in value position
  • Intrinsic classification (first-classable vs syntactic forms)
  • Call handler optimization to recognize administrative thunk-force patterns and skip throwaway thunk body generation

🛡 What tests cover this?

  • Classification unit tests: classify_trivial_closure_binary, classify_trivial_closure_unary, classify_non_reducible_non_trivial_prelude
  • Inlining snapshot tests: inline_trivial_closure_binary, inline_trivial_closure_transitive
  • Existing compiletest suite (e.g. nested-branch-elimination now demonstrates the full optimization cascade)

❓ How to test this?

cargo test --package hashql-mir --lib -- administrative_reduction::tests

📹 Demo

The nested-branch-elimination compiletest shows the cascade effect: closures containing single comparisons (> 100, < 0) are now inlined by AR, exposing constant-foldable comparisons (50 > 100, 50 < 0), which enables the entire branch structure to collapse to return "in range".

Copilot AI review requested due to automatic review settings June 22, 2026 13:47
@vercel

vercel Bot commented Jun 22, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview, Comment Jul 6, 2026 3:01pm
petrinaut Ready Ready Preview Jul 6, 2026 3:01pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview Jul 6, 2026 3:01pm

@vercel vercel Bot temporarily deployed to Preview – petrinaut June 22, 2026 13:47 Inactive
@cursor

cursor Bot commented Jun 22, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes MIR inlining/reduction and Postgres SQL for graph reads; risk is mitigated by broad snapshot and compiletest updates but affects query codegen paths.

Overview
Adds Source::Synthetic for compiler-generated operation wrappers (always inlined like ctors) and ReductionKind::TrivialClosure so administrative reduction inlines single-block bodies that end in one primitive op (Binary, Unary, etc.), not only forwarding Apply calls. Source::Intrinsic now carries an Intrinsic value (IntrinsicId + optimize flag) instead of a bare DefId; built-in DefId dict/list ops are removed from def.rs.

Synthetic and the new intrinsic shape are threaded through execution placement, eval live-out, inline heuristics (SyntheticInlineDirective::Always), and the body! macro ([synthetic sym::path]). Reification uses MixedBitSetPool with a custom allocator.

Stdlib: core::json is public with shared json_path / json_path_segment helpers; graph::entity::property is typed to return unknown (not Option) until BE-62. Postgres codegen joins entity_editions via CROSS JOIN LATERAL subselect instead of INNER JOIN. UI MIR fixtures mostly reorder locals; lowering tests snapshot degenerate CFGs (unreachable_only, goto_into_unreachable).

Reviewed by Cursor Bugbot for commit 761a195. Bugbot is set up for automated code reviews on this repo. Configure here.

indietyp commented Jun 22, 2026

Copy link
Copy Markdown
Member Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@codecov

codecov Bot commented Jun 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 76.92308% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 53.38%. Comparing base (638fbcb) to head (761a195).

Files with missing lines Patch % Lines
libs/@local/hashql/core/src/collections/pool.rs 44.44% 5 Missing ⚠️
...@local/hashql/core/src/module/std_lib/core/json.rs 93.75% 1 Missing ⚠️
Additional details and impacted files
@@                                          Coverage Diff                                          @@
##           bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-module    #8894   +/-   ##
=====================================================================================================
  Coverage                                                                 53.37%   53.38%           
=====================================================================================================
  Files                                                                       645      645           
  Lines                                                                     39575    39595   +20     
  Branches                                                                   3734     3734           
=====================================================================================================
+ Hits                                                                      21125    21139   +14     
- Misses                                                                    18198    18204    +6     
  Partials                                                                    252      252           
Flag Coverage Δ
apps.hash-ai-worker-ts 1.39% <ø> (ø)
apps.hash-api 6.39% <ø> (ø)
local.hash-backend-utils 2.81% <ø> (ø)
local.hash-graph-sdk 10.02% <ø> (ø)
local.hash-isomorphic-utils 0.18% <ø> (ø)
rust.hashql-core 79.28% <76.92%> (-0.01%) ⬇️
rust.hashql-syntax-jexpr 94.04% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot AI review requested due to automatic review settings July 3, 2026 14:28
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 410fa1b to ac8ed9e Compare July 3, 2026 14:28
@indietyp indietyp force-pushed the bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-module branch from 558d30c to d08cc92 Compare July 3, 2026 14:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 51 out of 52 changed files in this pull request and generated 2 comments.

Comment on lines 27 to 29
impl DefId {
/// Built-in dictionary insert operation (immutable).
///
/// This operation inserts a key-value pair into a dictionary,
/// returning a new dictionary with the added pair. The original
/// dictionary remains unchanged.
pub const DICT_INSERT: Self = Self::new(0xFFFF_FE00);
/// Built-in dictionary insert operation (mutable).
///
/// This operation inserts a key-value pair into a dictionary in-place,
/// modifying the original dictionary. Used for efficient dictionary
/// construction and updates.
pub const DICT_INSERT_MUT: Self = Self::new(0xFFFF_FE01);
/// Built-in dictionary remove operation (immutable).
///
/// This operation removes a key-value pair from a dictionary,
/// returning a new dictionary without the specified key. The
/// original dictionary remains unchanged.
pub const DICT_REMOVE: Self = Self::new(0xFFFF_FE02);
/// Built-in dictionary remove operation (mutable).
///
/// This operation removes a key-value pair from a dictionary in-place,
/// modifying the original dictionary and returning the removed value
/// if the key existed.
pub const DICT_REMOVE_MUT: Self = Self::new(0xFFFF_FE03);
/// Built-in list pop operation (immutable).
///
/// This operation removes the last element from a list, returning
/// both the element and a new list without the element. The original
/// list remains unchanged.
pub const LIST_POP: Self = Self::new(0xFFFF_FE04);
/// Built-in list pop operation (mutable).
///
/// This operation removes the last element from a list in-place,
/// returning the removed element while modifying the original list.
pub const LIST_POP_MUT: Self = Self::new(0xFFFF_FE05);
/// Built-in list push operation (immutable).
///
/// This operation appends an element to a list, returning a new list
/// without modifying the original. Used for functional-style list
/// manipulation where immutability is preferred.
pub const LIST_PUSH: Self = Self::new(0xFFFF_FE06);
/// Built-in list push operation (mutable).
///
/// This operation appends an element to a list in-place, modifying
/// the original list. Used for imperative-style list manipulation
/// where performance is critical.
pub const LIST_PUSH_MUT: Self = Self::new(0xFFFF_FE07);
pub const PLACEHOLDER: Self = Self::MAX;
}
Comment on lines 11 to 17
pub(in crate::module::std_lib) mod bits;
pub(in crate::module::std_lib) mod bool;
pub(in crate::module::std_lib) mod cmp;
pub(in crate::module::std_lib) mod json;
pub mod json;
pub(in crate::module::std_lib) mod math;
pub mod option;
pub(in crate::module::std_lib) mod result;
@indietyp indietyp force-pushed the bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-module branch from d08cc92 to 5014ae0 Compare July 3, 2026 14:36
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from ac8ed9e to 4aacfd8 Compare July 3, 2026 14:36

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4aacfd8. Configure here.

Copilot AI review requested due to automatic review settings July 3, 2026 15:21
@indietyp indietyp force-pushed the bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-module branch from 5014ae0 to 8c7419a Compare July 3, 2026 15:21
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 4aacfd8 to f2b45cb Compare July 3, 2026 15:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 51 out of 52 changed files in this pull request and generated 3 comments.

Comment on lines +140 to +164
// In this case it may either be a forwarding closure **or** a synthetic closure, this
// depends on the final statement, in either case the last statement must be an assignment.
let StatementKind::Assign(Assign { lhs, ref rhs }) = final_stmt.kind else {
return None;
};

// The kind depends on the final statement. Both cases are beta-reduction
// (substituting arguments into a function body at a call site):
//
// Apply: the body forwards to another call. The wrapper (lambda x. f(x)) is an
// eta-expansion of the inner function; inlining at the call site is beta-reduction
// that exposes the inner call directly.
//
// Binary, Unary, and other primitive operations: the body computes a single result
// from its arguments. Inlining substitutes the arguments and exposes the primitive
// operation. A subsequent constant folding pass may then perform delta-reduction
// (evaluating the primitive when operands are known).
let kind = match rhs {
RValue::Apply(_) => Self::ForwardingClosure,
RValue::Aggregate(_)
| RValue::Binary(_)
| RValue::Unary(_)
| RValue::Input(_)
| RValue::Load(_) => Self::TrivialClosure,
};
Comment on lines 167 to 171
if bb.terminator.kind
== TerminatorKind::Return(Return {
value: Operand::Place(lhs),
})
{
Comment on lines +66 to 73
// `property<T>(entity: Entity<T>, path: JsonPath) -> ?`
// TODO(BE-62): return `Option<?>` once pattern matching allows for option destructuring, to
// allow for proper comparison
let decl = decl!(context;
<T>(entity: context.ty.apply([(entity_ty.arguments[0].id, T)], entity_ty.id),
path: json_path_ty.id
) -> option(&context.ty, context.ty.unknown())
) -> context.ty.unknown()
);
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from f2b45cb to 489dd1a Compare July 3, 2026 16:24
@indietyp indietyp force-pushed the bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-module branch from 8c7419a to 4abefab Compare July 3, 2026 16:24
Copilot AI review requested due to automatic review settings July 6, 2026 07:48
@indietyp indietyp force-pushed the bm/be-607-hashql-remove-smallvec-from-the-standardlibrary-for-module branch from 4abefab to 9088654 Compare July 6, 2026 07:48
@indietyp indietyp force-pushed the bm/be-615-hashql-introduce-synthetic-closures-and-trivial-closure branch from 489dd1a to 5628792 Compare July 6, 2026 07:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 51 out of 52 changed files in this pull request and generated 4 comments.

Comment on lines 27 to 29
impl DefId {
/// Built-in dictionary insert operation (immutable).
///
/// This operation inserts a key-value pair into a dictionary,
/// returning a new dictionary with the added pair. The original
/// dictionary remains unchanged.
pub const DICT_INSERT: Self = Self::new(0xFFFF_FE00);
/// Built-in dictionary insert operation (mutable).
///
/// This operation inserts a key-value pair into a dictionary in-place,
/// modifying the original dictionary. Used for efficient dictionary
/// construction and updates.
pub const DICT_INSERT_MUT: Self = Self::new(0xFFFF_FE01);
/// Built-in dictionary remove operation (immutable).
///
/// This operation removes a key-value pair from a dictionary,
/// returning a new dictionary without the specified key. The
/// original dictionary remains unchanged.
pub const DICT_REMOVE: Self = Self::new(0xFFFF_FE02);
/// Built-in dictionary remove operation (mutable).
///
/// This operation removes a key-value pair from a dictionary in-place,
/// modifying the original dictionary and returning the removed value
/// if the key existed.
pub const DICT_REMOVE_MUT: Self = Self::new(0xFFFF_FE03);
/// Built-in list pop operation (immutable).
///
/// This operation removes the last element from a list, returning
/// both the element and a new list without the element. The original
/// list remains unchanged.
pub const LIST_POP: Self = Self::new(0xFFFF_FE04);
/// Built-in list pop operation (mutable).
///
/// This operation removes the last element from a list in-place,
/// returning the removed element while modifying the original list.
pub const LIST_POP_MUT: Self = Self::new(0xFFFF_FE05);
/// Built-in list push operation (immutable).
///
/// This operation appends an element to a list, returning a new list
/// without modifying the original. Used for functional-style list
/// manipulation where immutability is preferred.
pub const LIST_PUSH: Self = Self::new(0xFFFF_FE06);
/// Built-in list push operation (mutable).
///
/// This operation appends an element to a list in-place, modifying
/// the original list. Used for imperative-style list manipulation
/// where performance is critical.
pub const LIST_PUSH_MUT: Self = Self::new(0xFFFF_FE07);
pub const PLACEHOLDER: Self = Self::MAX;
}
Comment on lines 274 to 278
let inline = match body.source {
Source::Ctor(_) => InlineDirective::Always,
Source::Ctor(_) | Source::Synthetic(_) => InlineDirective::Always,
Source::Closure(_, _) | Source::Thunk(_, _) => InlineDirective::Heuristic,
Source::Intrinsic(_) | Source::GraphReadFilter(_) => InlineDirective::Never,
};
Comment on lines 11 to 17
pub(in crate::module::std_lib) mod bits;
pub(in crate::module::std_lib) mod bool;
pub(in crate::module::std_lib) mod cmp;
pub(in crate::module::std_lib) mod json;
pub mod json;
pub(in crate::module::std_lib) mod math;
pub mod option;
pub(in crate::module::std_lib) mod result;
Comment on lines +66 to 73
// `property<T>(entity: Entity<T>, path: JsonPath) -> ?`
// TODO(BE-62): return `Option<?>` once pattern matching allows for option destructuring, to
// allow for proper comparison
let decl = decl!(context;
<T>(entity: context.ty.apply([(entity_ty.arguments[0].id, T)], entity_ty.id),
path: json_path_ty.id
) -> option(&context.ty, context.ty.unknown())
) -> context.ty.unknown()
);

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 51 out of 52 changed files in this pull request and generated 2 comments.

Comment on lines +140 to +141
// In this case it may either be a forwarding closure **or** a synthetic closure, this
// depends on the final statement, in either case the last statement must be an assignment.
Comment on lines 384 to +386
(@source intrinsic) => {
$crate::body::Source::Intrinsic($crate::def::DefId::PLACEHOLDER)
$crate::body::Source::Intrinsic($crate::intrinsic::Intrinsic { id: $crate::intrinsic::IntrinsicId::EntityPropertyAccess, optimize: true })
};

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 51 out of 52 changed files in this pull request and generated 1 comment.

Comment on lines +140 to 144
// In this case it may either be a forwarding closure **or** a synthetic closure, this
// depends on the final statement, in either case the last statement must be an assignment.
let StatementKind::Assign(Assign { lhs, ref rhs }) = final_stmt.kind else {
return None;
};
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/libs Relates to first-party libraries/crates/packages (area) area/tests New or updated tests type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

2 participants