Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 52 additions & 1 deletion .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,16 @@ jobs:
- essreflectometry
- esssans
- essspectroscopy
# Packages whose tests write reduced data files via the --file-output
# pytest option. Their output is collected and published to the
# `reduced_data_nightly` release by the `publish` job below. To publish data
# from another package, add a `--file-output`/`output_folder` fixture to
# its conftest and list it here.
include:
- package: essdiffraction
file_output: true
- package: essreflectometry
file_output: true
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
Expand All @@ -34,7 +44,48 @@ jobs:
frozen: true
environments: ${{ matrix.package }}
- name: Run tests
run: pixi run -e ${{ matrix.package }} test ${{ matrix.package }}
run: pixi run -e ${{ matrix.package }} test ${{ matrix.package }} ${{ matrix.file_output && '-- --file-output=tests_outputs' || '' }}
- name: Upload reduced data files
if: matrix.file_output
uses: actions/upload-artifact@v7
with:
name: reduced-data-${{ matrix.package }}
path: tests_outputs/
if-no-files-found: warn

publish:
name: Publish nightly reduced data
needs: test
runs-on: ubuntu-24.04
permissions:
contents: write
steps:
- uses: actions/checkout@v6
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we even need to checkout here?

- name: Download reduced data files
uses: actions/download-artifact@v8
with:
path: ./artifacts
pattern: reduced-data-*
merge-multiple: true
- name: Publish to nightly release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
shopt -s nullglob
files=(./artifacts/*)
if [ ${#files[@]} -eq 0 ]; then
echo "No reduced data files were produced; nothing to publish."
exit 0
fi
echo "Publishing ${#files[@]} file(s):"
printf ' %s\n' "${files[@]}"
gh release delete reduced_data_nightly --yes --cleanup-tag 2>/dev/null || true
gh release create reduced_data_nightly \
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if we skip the delete step and just try to push new files to an existing release?
For example, when uploading the built docs as assets to a release, we use a different action that just add files to a release.
I wonder if you can replace files with that, and avoid having to create the release entirely (now that it has been created once)?

Copy link
Copy Markdown
Member

@nvaytet nvaytet May 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be an option for overwriting:
overwrite false If an asset with the same name already exists, overwrite it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another reason to maybe use the action to upload assets is that I seem to get a gh notification every time a release is made.

Just uploading would not add any noise to our inboxes.

--target "${{ github.sha }}" \
--title "Nightly reduced data files" \
--notes "Automated nightly build from $(date -u +%Y-%m-%dT%H:%MZ) (commit ${GITHUB_SHA::7})" \
--prerelease \
"${files[@]}"

lower-bound:
name: Lower bound ${{ matrix.package }}
Expand Down