Skip to content

Nexus compliance #140

Description

@pecomyint

We are already doing the important first part correctly: the files are not just arbitrary HDF5. We explicitly assign NeXus classes such as NXentry, NXinstrument, NXdetector, NXsample, NXdata, NXsource, and NXtransformations. So structurally, we are already quite close.

What I think we should do next is consolidate the DAQ writer so that we have one canonical NeXus layout and make the semantics a little more complete.

Right now, the active DAQ save path uses save_caches_to_h5() / h5_save(), while there is also a save_scan_to_h5() implementation that already has a more complete NXdata definition, including the signal, scan axis, and axis indices. I think we should merge these ideas rather than maintain two slightly different NeXus-writing paths.

The structure I have in mind is roughly:

entry:NXentry
├── instrument:NXinstrument
│   ├── detector:NXdetector
│   │   └── data
│   ├── source:NXsource
│   ├── sensors:NXsensor
│   ├── monitors:NXmonitor
│   └── positioners / other instrument components
│
├── sample:NXsample
│   └── transformations:NXtransformations
│
└── data:NXdata
    ├── signal -> instrument/detector/data
    └── axes   -> scan/sample/motor axes

A few specific things I think we should address:

  • Make NXdata fully self-describing by setting signal, axes, and the corresponding *_indices, so a generic NeXus reader can determine what to plot/load without knowing our internal HDF5 paths.
  • Keep the detector data under NXdetector, with NXdata linking to it rather than duplicating it.
  • Move meaningful physical devices out of generic NXcollection where possible. For example, temperature/pressure sensors could be represented as NXsensor, beam monitors as NXmonitor, etc.
  • Make the sample/goniometer transformations more NeXus-complete by adding the appropriate transformation_type, vector, and depends_on metadata instead of only storing/linking the motor values.
  • Keep facility-specific metadata where we need it — NeXus allows extensions — but use NeXus base classes whenever there is already a standard semantic representation.
  • Add a small validation/unit test so that DAQ output is checked for the expected NeXus hierarchy and required attributes.

The goal is not to force every beamline to have exactly the same HDF5 tree. NeXus is intentionally extensible. The benefit is that common concepts — detector, source, sensor, sample, scan axis, primary signal, transformations — have standardized semantics, so downstream applications can discover and load the data without maintaining beamline-specific path mappings.

I think this would make the DAQ format considerably more robust for interoperability while requiring relatively modest changes to the writer we already have.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions