feat: allow extending hyperlight-js-runtime with custom native modules

[simongdavies]

Resolves #48

Overview

Adds an extension system that lets downstream crates add custom native modules, custom globals, and override built-in modules — without forking the runtime.

Architecture

Three macros, one init sequence:

JsRuntime::new()
  ├─ Step 1: globals::setup()         — install built-in globals (console, print, require)
  │                                      console + print are writable at this stage
  ├─ Step 2: custom_globals!(...)     — extender adds/extends globals (e.g. console.warn)
  ├─ Step 3: globals::freeze()        — Object.freeze(console), lock print
  └─ Module loader ready              — custom modules take priority over built-ins

native_modules! — Custom native modules

Register Rust-implemented modules that guest JS can import:

hyperlight_js_runtime::native_modules! {
    "math" => js_math,
    "io"   => js_custom_io,   // overrides built-in io
}

• Custom modules take priority over built-ins with the same name
• io, crypto, console can be overridden
• require cannot be overridden (panics — it's core infrastructure)
• Lazy initialization via #[no_mangle] + extern "Rust" linkage

custom_globals! — Custom global objects

Register global objects (constructors, polyfills, constants) without import:

fn setup_text_encoding(ctx: &rquickjs::Ctx<'_>) -> rquickjs::Result<()> {
    ctx.globals().set("TextEncoder", TextEncoder::constructor(ctx)?)?;
    Ok(())
}

hyperlight_js_runtime::custom_globals! {
    setup_text_encoding,
}

• Runs after built-in globals, before freeze
• Can extend console (add warn, error, info, debug)
• Both Rust classes (#[rquickjs::class]) and JS polyfills (ctx.eval()) supported

Globals freeze

After custom_globals! runs, built-in globals are locked down:

• console → Object.freeze() (no new properties, no modifications)
• print → non-writable, non-configurable
• require → already non-configurable from setup via Property::from()

Handler code cannot tamper with any of these.

Runtime as a library

hyperlight-js-runtime now exposes a [lib] target. Extender crates depend on it and provide a binary that links everything together. The HYPERLIGHT_JS_RUNTIME_PATH build-time env var tells hyperlight-js to embed the custom binary.

Guest infrastructure (entry point, host function dispatch, libc stubs) moved from src/main/ to src/guest/ and is provided by the lib — no copying needed.

Key changes

File	Change
src/hyperlight-js-runtime/src/modules/mod.rs	NativeModuleLoader, register_native_module, native_modules!, custom_globals!, setup_custom_globals()
src/hyperlight-js-runtime/src/lib.rs	3-step init: setup → custom_globals → freeze
src/hyperlight-js-runtime/src/globals/console.rs	Console as extensible Object (not frozen module namespace)
src/hyperlight-js-runtime/src/globals/print.rs	Print as writable global (frozen after custom_globals)
src/hyperlight-js-runtime/src/globals/freeze.rs	New — freezes console + print after custom_globals
src/hyperlight-js-runtime/src/globals/mod.rs	Added freeze() public function
src/hyperlight-js-runtime/src/main.rs	Empty native_modules! {} + custom_globals! {}
src/hyperlight-js-runtime/src/guest/	Moved from src/main/hyperlight.rs — lib-provided guest infra
src/hyperlight-js/build.rs	HYPERLIGHT_JS_RUNTIME_PATH support
docs/extending-runtime.md	Full guide: quick start, native CLI testing, API reference, custom globals

Tests

• 22 unit tests in tests/native_modules.rs: loader resolution, custom modules, built-in override, require protection, custom_globals! macro, console extensions, freeze behaviour
• 6 full-pipeline E2E tests: build fixture binary → run handler.js → verify output (custom modules, builtins + custom together, console.log, custom globals, globals + modules combined)
• 3 host-side integration tests in src/hyperlight-js/tests/native_modules.rs: HYPERLIGHT_JS_RUNTIME_PATH embedding
• Fixture crates: native_math (shared lib) + extended_runtime (binary with custom modules + globals)

[Copilot]

Pull request overview

Adds an extension mechanism to hyperlight-js-runtime so downstream crates can register custom native Rust modules and custom globals (and optionally override built-ins) without forking, and wires host-side embedding to support a custom runtime binary via HYPERLIGHT_JS_RUNTIME_PATH.

Changes:

• Introduces a custom native-module registry + native_modules! macro and a custom_globals! macro, and updates JsRuntime::new() init sequence to: built-in globals → custom globals → freeze.
• Adds host build-time support for embedding a custom runtime binary (HYPERLIGHT_JS_RUNTIME_PATH) and a just test-native-modules pipeline to exercise it.
• Adds fixtures, docs, and tests covering custom modules/globals, console extensibility, and post-init freezing behavior.

Reviewed changes

Copilot reviewed 19 out of 25 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/hyperlight-js/tests/native_modules.rs	Ignored VM integration tests validating custom modules/globals via embedded custom runtime.
src/hyperlight-js/build.rs	Adds HYPERLIGHT_JS_RUNTIME_PATH override for embedding a custom runtime binary.
src/hyperlight-js-runtime/tests/native_modules.rs	Unit/E2E tests for registry, macros, globals setup, freeze behavior, and fixture pipeline.
src/hyperlight-js-runtime/tests/fixtures/native_math/src/lib.rs	Fixture native module providing add/multiply.
src/hyperlight-js-runtime/tests/fixtures/native_math/Cargo.toml	Fixture crate manifest for native-math.
src/hyperlight-js-runtime/tests/fixtures/extended_runtime/src/main.rs	Fixture binary using native_modules! and custom_globals! to extend the runtime.
src/hyperlight-js-runtime/tests/fixtures/extended_runtime/Cargo.toml	Fixture binary manifest depending on the runtime lib + fixture module.
src/hyperlight-js-runtime/tests/fixtures/extended_runtime/Cargo.lock	Lockfile for the standalone fixture build.
src/hyperlight-js-runtime/src/modules/mod.rs	Implements custom module registry, lazy init symbol, unified loader, and both extension macros.
src/hyperlight-js-runtime/src/main.rs	Provides empty native_modules!{} / custom_globals!{} symbols for the upstream binary.
src/hyperlight-js-runtime/src/lib.rs	Exposes runtime as a library and adds 3-step init (setup → custom globals → freeze).
src/hyperlight-js-runtime/src/guest/stubs/srand.rs	Adds libc stub for srand.
src/hyperlight-js-runtime/src/guest/stubs/mod.rs	Registers the new/relocated libc stub modules.
src/hyperlight-js-runtime/src/guest/stubs/localtime.rs	Adds libc stub implementation for localtime_r.
src/hyperlight-js-runtime/src/guest/stubs/io.rs	Adds libc stubs for putchar/fflush.
src/hyperlight-js-runtime/src/guest/stubs/clock.rs	Adds libc stubs for clock_gettime/gettimeofday using host time.
src/hyperlight-js-runtime/src/guest/mod.rs	Moves guest entrypoint/plumbing into the library (cfg(hyperlight)).
src/hyperlight-js-runtime/src/globals/print.rs	Makes print writable during custom-globals phase (frozen later).
src/hyperlight-js-runtime/src/globals/mod.rs	Adds freeze() stage API and wires in new freeze module.
src/hyperlight-js-runtime/src/globals/freeze.rs	New freeze stage: freezes console object and locks down print.
src/hyperlight-js-runtime/src/globals/console.rs	Installs console as an extensible object (not module namespace) before freezing.
src/hyperlight-js-runtime/Cargo.toml	Adds [lib] target and makes the bin explicitly point to src/main.rs.
docs/extending-runtime.md	Adds documentation for extending runtime with custom modules/globals and embedding workflow.
Justfile	Adds test-native-modules pipeline and updates clean/test recipes accordingly.
Cargo.toml	Excludes the extended runtime fixture from the workspace.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 19 out of 25 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 19 out of 25 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The workspace exclude list only excludes the extended_runtime fixture, but tests/fixtures/native_math/Cargo.toml also declares its own [workspace]. If this is meant to avoid nested-workspace conflicts, it’s safer to exclude the whole fixtures directory (or at least both extended_runtime and native_math) to prevent Cargo from treating nested workspace roots as part of the parent workspace in some commands.

Suggested change

	exclude = ["src/hyperlight-js-runtime/tests/fixtures/extended_runtime"]
	exclude = ["src/hyperlight-js-runtime/tests/fixtures"]

[Copilot]

The hyperlight guest entrypoints (e.g. hyperlight_main, guest_dispatch_function) now live in the library crate, but this binary doesn’t reference them. Because an rlib is an archive, the linker may omit the guest module’s object/codegen units if nothing pulls them in, producing a guest binary missing the required exported symbols. Consider forcing the guest module to be linked (e.g., by referencing a #[cfg(hyperlight)] symbol from guest via a #[used] static, or otherwise ensuring the guest module is part of the final link).

Suggested change

	#[cfg(hyperlight)]
	extern "C" {
	/// Guest entrypoint provided by the library crate's `guest` module.
	///
	/// This declaration is only used to force the guest module to be linked
	/// into the final hyperlight binary; it is not invoked from this file.
	fn hyperlight_main();
	}
	#[cfg(hyperlight)]
	#[used]
	static _FORCE_GUEST_LINK: unsafe extern "C" fn() = hyperlight_main;

[Copilot]

This fixture binary relies on the runtime library to provide Hyperlight guest entrypoints, but it doesn’t reference any of those symbols. With static linking of rlibs, the guest module can be dropped if not pulled in by an undefined reference, resulting in a guest binary that builds but lacks hyperlight_main/dispatch symbols at runtime. Please ensure the fixture (and recommended downstream pattern) forces the guest module to be linked into the final artifact.

Suggested change

	// guest_dispatch_function, and all plumbing. Nothing else needed here.
	// guest_dispatch_function, and all plumbing. Nothing else needed here.
	// Force the guest module to be linked for hyperlight builds by creating
	// undefined references to the guest entrypoints provided by the runtime.
	#[cfg(hyperlight)]
	extern "C" {
	fn hyperlight_main();
	fn guest_dispatch_function();
	}
	/// Linker anchor to ensure Hyperlight guest entrypoints are retained.
	///
	/// This function is never called; it simply references the extern symbols
	/// so that the linker must resolve them from the runtime crate.
	#[cfg(hyperlight)]
	#[no_mangle]
	pub extern "C" fn _hyperlight_guest_link_anchor() {
	unsafe {
	let _ = hyperlight_main as unsafe extern "C" fn();
	let _ = guest_dispatch_function as unsafe extern "C" fn();
	}
	}