-
Notifications
You must be signed in to change notification settings - Fork 0
[CI] Publish nightly reduced data files #596
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -23,6 +23,16 @@ jobs: | |
| - essreflectometry | ||
| - esssans | ||
| - essspectroscopy | ||
| # Packages whose tests write reduced data files via the --file-output | ||
| # pytest option. Their output is collected and published to the | ||
| # `reduced_data_nightly` release by the `publish` job below. To publish data | ||
| # from another package, add a `--file-output`/`output_folder` fixture to | ||
| # its conftest and list it here. | ||
| include: | ||
| - package: essdiffraction | ||
| file_output: true | ||
| - package: essreflectometry | ||
| file_output: true | ||
| runs-on: ubuntu-24.04 | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
|
|
@@ -34,7 +44,48 @@ jobs: | |
| frozen: true | ||
| environments: ${{ matrix.package }} | ||
| - name: Run tests | ||
| run: pixi run -e ${{ matrix.package }} test ${{ matrix.package }} | ||
| run: pixi run -e ${{ matrix.package }} test ${{ matrix.package }} ${{ matrix.file_output && '-- --file-output=tests_outputs' || '' }} | ||
| - name: Upload reduced data files | ||
| if: matrix.file_output | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: reduced-data-${{ matrix.package }} | ||
| path: tests_outputs/ | ||
| if-no-files-found: warn | ||
|
|
||
| publish: | ||
| name: Publish nightly reduced data | ||
| needs: test | ||
| runs-on: ubuntu-24.04 | ||
| permissions: | ||
| contents: write | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - name: Download reduced data files | ||
| uses: actions/download-artifact@v8 | ||
| with: | ||
| path: ./artifacts | ||
| pattern: reduced-data-* | ||
| merge-multiple: true | ||
| - name: Publish to nightly release | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: | | ||
| shopt -s nullglob | ||
| files=(./artifacts/*) | ||
| if [ ${#files[@]} -eq 0 ]; then | ||
| echo "No reduced data files were produced; nothing to publish." | ||
| exit 0 | ||
| fi | ||
| echo "Publishing ${#files[@]} file(s):" | ||
| printf ' %s\n' "${files[@]}" | ||
| gh release delete reduced_data_nightly --yes --cleanup-tag 2>/dev/null || true | ||
| gh release create reduced_data_nightly \ | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What happens if we skip the delete step and just try to push new files to an existing release?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There seems to be an option for overwriting:
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Another reason to maybe use the action to upload assets is that I seem to get a gh notification every time a release is made. Just uploading would not add any noise to our inboxes. |
||
| --target "${{ github.sha }}" \ | ||
| --title "Nightly reduced data files" \ | ||
| --notes "Automated nightly build from $(date -u +%Y-%m-%dT%H:%MZ) (commit ${GITHUB_SHA::7})" \ | ||
| --prerelease \ | ||
| "${files[@]}" | ||
|
|
||
| lower-bound: | ||
| name: Lower bound ${{ matrix.package }} | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we even need to checkout here?