Major Feature Overhaul

[gselzer]

This PR is a major expansion of scenex, introducing multi-view layouts, a comprehensive event system, and new additions to the model hierarchy.

Multi-View Grid System

A flexible grid layout enables multiple synchronized views (e.g., orthoviewers) on a single canvas. Views can be added to any grid position, span rows/columns, and are automatically laid out. The Canvas now uses a Grid for structured multi-view support.

# Create a 2x2 grid of views
grid = Grid(row_sizes=(1.0, 1.0), col_sizes=(1.0, 1.0))
grid.add(view1, row=0, col=0)
grid.add(view2, row=0, col=1)
grid.add(view3, row=1, col=0)
grid.add(view4, row=1, col=1)
canvas = Canvas(grid=grid)

Architecture:

• Grid contains GridAssignment objects mapping views to positions
• Row/column sizes are relative weights (normalized to available space)
• Views can span multiple rows/columns
• Auto-expansion when adding views beyond current grid dimensions

Canvas Changes:

• Canvas.views (single list) replaced with Canvas.grid (structured layout)
• Layout computation happens automatically when grid is modified

Comprehensive Event System

A unified event API currently supporting mouse, and wheel events across Qt, wx, and Jupyter. Event and its many subclasses are derived from native widget events and are passed to the models. MouseEvents include a world_ray for interactive picking, enabling features like node intersection and custom event filtering.

Note that, to enable these events, we require significant amounts of app code. This code has been migrated from ndv, contained within the scenex.app module, such that it could be cleanly extracted later.

from scenex.app.events import MouseMoveEvent

def my_filter(event: MouseMoveEvent) -> bool:
    intersections = event.world_ray.intersections(view.scene)
    for node, distance in intersections:
        point = event.world_ray.point_at_distance(distance)
        # Do something with intersection
    return True

view.set_event_filter(my_filter)

Event Flow:

User Action → GUI Framework (Qt/wx/Jupyter)
           → Canvas (determines which view)
           → View.filter_event()
           → Camera.controller.handle_event()

Node Intersection:

• Nodes implement passes_through(ray) -> float | None
• Returns depth along ray if intersection occurs
• Invisible nodes (visible=False) don't intersect

New Geometry Node Types

This PR introduces (Poly)Lines, Meshes, and Text as new Node implementations. Example instances of each are shown below.

from cmap import Color
import numpy as np
import scenex as snx

# Line
line = snx.Line(
    vertices=np.array([[0, 0], [10, 5], [20, 0]]),
    color=snx.VertexColors(color=[Color("red"), Color("green"), Color("blue")]),
    width=2.0,
    antialias=1.0,
)

# Mesh
mesh = snx.Mesh(
    vertices=vertices,      # (N, 3) array
    faces=faces,           # (M, 3) triangle indices
    color=snx.UniformColor(color=Color("red")),
)

# Text
text = Text(
    text="Hello World",
    size=12,
    color=Color("white"),
)

ColorModel System

scenex needed structures determining how colors could map to geometry for various Node subclasses. The ColorModel base class has been added along with concrete subclasses: UniformColor, FaceColors, and VertexColors. This architecture enables all geometric nodes (Points, Line, Mesh) to declare which ColorModels they support (for example, Points will only support UniformColor and VertexColors).

# Uniform color for all elements
UniformColor(color=Color("red"))
# Per-face colors (for meshes)
FaceColors(color=[Color("red"), Color("blue"), ...])
# Per-vertex colors (for lines, meshes, points)
VertexColors(color=[Color("red"), Color("blue"), ...])

Camera Controllers & Projection

The Camera API has been overhauled to deduplicate state. Most notably, the Camera offers a projection: Transform which describes how 3D space is mapped to/from the 2D canvas. Common projection types can be generated from the utilities within scenex.util.projections.

For user interaction, Camera objects can also be assigned an InteractionStrategy which defines what happens to the Camera when Events occur. PanZoom and Orbit are two implementations of this model, and it is simple to add additional implementations later on.

Finally, the View object now can be assigned a ResizeStrategy that defines what happens when the View is resized either programmatically or through user events. Letterbox is one implementation of this model, and we could also add additional strategies later.

# PanZoom controller
Camera(
    controller=PanZoom(),
    interactive=True
)

# Orbit controller
Camera(
    controller=Orbit(center=(0, 0, 0)),
    interactive=True
)

# Explicit projection
from scenex.utils.projections import perspective, orthographic
Camera(projection=orthographic(width=10, height=10, depth=100))
# -- or -- #
Camera(projection=perspective(fov=60, aspect=1.5, near=0.1, far=1000))

# Camera utilities
camera.look_at((0, 0, 0), up=(0, 0, 1))

Testing & Documentation

All new features are covered by many new unit tests across all supported backends and platforms, and many examples have been added to showcase major additions. Documentation is overhauled with comprehensive NumPy-style docstrings and new examples, ensuring discoverability and ease of use.

Points of Discussion:

This is obviously a large change. I'd particularly like to draw the viewer's attention to the new models (ColorModel, ResizeStrategy, InteractionStrategy, Grid, etc.). I am very interested in critique of these structures. Are they well named? Are they declaritive? Are they missing things?

There are certainly more features to add as time goes on. One piece of low-hanging fruit is the implementation of FaceColors for meshes. But we can certainly start review before that gets solved.

The end goal of these changes, as alluded to by the branch name, is the ability to transform ndv by removing all of the pygfx/vispy-dependent code. If you'd like to see how this might look, check out the branch that depends on this changeset and feel free to consider and/or test that code.

One aspect of the changes that I removed fairly recently was the ability to add per-node event filters - I found this without real usage either here or on my ndv branch using it. I do think that it would be a nice feature to have, just don't want to add it (back) in without concrete need. Open to others' opinions on this, though.

Closes #16 as well as #38, #36, #34, #33, #32, #28, #27 (duplicate code)

[tlambert03]

I'll post observations as they come, rather than hold them for a big review...

The first is similar to something I said on zulip at one point: the resizing behavior has a significantly degraded experience relative to main. (showing pygfx below)

here's main. note how the aspect ratio never changes (only the letter-boxing), and the window size strictly follows the mouse without jittering.

main.mov

here's this PR. Note how the aspect ratio stretches and compresses, and only "fixes" itself after you release the mouse, and also how the window height rapidly flickers as you resize.

pr.mov

We need to recover the smoothness of main

edit
vispy does resize more smoothly ... but actually renders with poor aspect ratio until you resize it:

vispy.mov

[gselzer]

here's this PR. Note how the aspect ratio stretches and compresses, and only "fixes" itself after you release the mouse, and also how the window height rapidly flickers as you resize.

Ooh, thanks, yeah, I remember seeing this bug. It likely lies within the interconnection of pygfx+wx, since I don't see the flickering with either vispy+wx (which actually has a different bug where the aspect ratio is incorrect until you resize it) or pygfx+qt. Can you tell me whether resizing works well with another pair of backends?

EDIT: After investigating further, there are actually two separate issues here:

1. The flickering - I can fix this shortly.
2. The aspect ratio problem - This is a more complex upstream issue. Drawing during resize with Wx+RenderCanvas is disabled upstream, and the same behavior occurs on ndv's main branch.

I also noticed I can't run the main branch of scenex with pygfx+wx because the wx backend isn't available automatically. Can you confirm you were running pygfx+qt on the main branch and pygfx+wx on this branch?
If so, I think the aspect ratio issue is beyond the scope of this PR. The fixes I'm planning are:

1. Fix the flickering
2. (Optional) Should qt take precedence over wx when both are available? I suspect that's what's happening on your system, and the change should be straightforward. 🤞

[gselzer]

@tlambert03 can you confirm c6b8ccb fixes the flickering?

[tlambert03]

yes I think so! 🎉

side question: Do you know why, on both main and this PR, when I try vispy with SCENEX_CANVAS_BACKEND=vispy uv run examples/basic_scene.py, I get KeyError: "'PygfxAdaptorRegistry' has no adaptor for <class 'scenex.model.Canvas'> @ 10862ebc0, and create=False" ... probably something simple; I suspect you hit it know?

[gselzer]

side question: Do you know why, on both main and this PR, when I try vispy with SCENEX_CANVAS_BACKEND=vispy uv run examples/basic_scene.py, I get KeyError: "'PygfxAdaptorRegistry' has no adaptor for <class 'scenex.model.Canvas'> @ 10862ebc0, and create=False" ... probably something simple; I suspect you hit it know?

I actually didn't know, but but I can reproduce if the environment has both vispy and pygfx. Thus we have a bug where:

• add_imgui_controls can be imported because pygfx is in the environment
• you create a canvas that is vispy-backed (due to the environment variable plus the snx.show call which actually reaches into vispy.
• add_imgui_controls takes the view and looks for an existing canvas adaptor, backed by pygfx. It can't find one because the one we made is vispy-backed, and it fails.

So we just need to be a bit smarter about how we fail out of add_imgui_controls. I'll attempt a fix. Sorry about the bugginess of the imgui extensions - I admittedly did not use/test them until recently.

[gselzer]

@almarklein I hesitate to loop you into a monster PR, but I'm wondering if rendercanvas 2.5.0 is responsible for tests hanging on CI (I can't reproduce locally on my Windows machine). Would you have any idea why that might be? See these three sequential commits:

1. ebacc38, passes with rendercanvas==2.4.2
2. c6b8ccb, fails/hangs with rendercanvas==2.5.0 on wx and qt canvases
3. e118131, passes with rendercanvas==2.4.2

Note that in a failed job where I added import faulthandler; faulthandler.dump_traceback_later(10, repeat=True) to conftest.py, we get stack traces showing the hanging coming from the new loop.call_soon_threadsafe()...

[almarklein]

Hey @gselzer IIUC the hanging happened only with the Windows builds, right?

Edit: I can reproduce hanging tests with the rendercanvas test suite on Windows. Looking into it: pygfx/rendercanvas#160

[gselzer]

@almarklein it seems like the rendercanvas fixes have resolved the build! Thank you so much for being so speedy on the fix! 💯

[almarklein]

@gselzer awesome! And thanks for the heads-up!

Aside from the frustration etc., introducing breaking changes is actually a pretty effective way to get to know downstream developers! 😉

[gselzer]

@tlambert03 just wanted to bump this - are you still hoping to review this code, or is the only thing holding up merge here the fixes to add_imgui_controls (ala)?

So we just need to be a bit smarter about how we fail out of add_imgui_controls. I'll attempt a fix. Sorry about the bugginess of the imgui extensions - I admittedly did not use/test them until recently.

[tlambert03]

definitely still going to do another round of reviews (probably many more). However, if there's anything you know remains to be done... by all means go for it.