Skip to content

Fix two GPU kernels that share a scalar every thread writes - #1823

Open
sbryngelson wants to merge 2 commits into
MFlowCode:masterfrom
sbryngelson:fix/kernel-private-drift
Open

Fix two GPU kernels that share a scalar every thread writes#1823
sbryngelson wants to merge 2 commits into
MFlowCode:masterfrom
sbryngelson:fix/kernel-private-drift

Conversation

@sbryngelson

Copy link
Copy Markdown
Member

Two GPU kernels write a scalar that is not in their private clause, so every thread shares one copy. Under OpenMP offload that is a data race; under OpenACC it is hidden, because the spec predetermines scalars in a compute region as private. The symptom is a run that gives a different answer every time on Cray OpenMP while every other backend looks fine.

The two sites

m_riemann_solver_hllc.fpp emits its kernel twice through #:for HYPO in [True, False], once specialised for hypoelasticity and once for pure fluids, so that the pure-fluid copy is not pinned at the register ceiling by variables it never uses. Each copy carried its own hand-written list of about eighty private names, and the two drifted: c_sum_Yi_Phi, written by the chemistry block that is common to both, is private in the hypoelastic list and missing from the other.

Measured on Frontier, three runs of 1D -> Chemistry -> Inert Shocktube -> Reacting Roe Average -> HLLC on one binary:

0.07593517165078
0.07593517172084
0.07593517170526

m_riemann_solver_lf.fpp has the same defect independently: Gamm_L and Gamm_R are assigned inside the kernel under chem_params%gamma_method == 2 and are absent from its private list.

The fix

The Lax-Friedrichs list gains the two names. The HLLC lists are no longer written twice: one list holds the shared names, another the elastic-only ones, and the two emissions are composed from them, so adding a variable to the shared body cannot leave one copy behind.

Verified against a pristine build of the same file: three of the four generated kernels are byte-identical, and the pure-fluid one gains exactly c_sum_Yi_Phi plus the collapse indices and an unused iterator that were already private in the sibling list.

How to find this class

Reading the source shows one loop body, so a missing name looks handled. It is only visible in the generated Fortran, where there are two directives and one of them is short. The check is to take each !$omp target directive and confirm every scalar assigned in its body appears in the clause.

Testing

Chemistry set passes, 15 of 15. Format and precheck clean. No golden files move: on a serial CPU build a private clause is a no-op, which is also why this never showed up outside GPU offload.

Copilot AI lite review requested due to automatic review settings September 4, 2026 21:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Fixes GPU offload data races caused by missing private scalars in two Riemann-solver kernels, making OpenMP offload behavior consistent with other backends.

Changes:

  • Add missing Gamm_L/Gamm_R to the Lax-Friedrichs kernel private list.
  • Refactor HLLC private-variable lists into shared vs hypoelastic-only fragments to prevent drift between the two generated kernels.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
src/simulation/m_riemann_solver_lf.fpp Adds missing chemistry-only scalars to the OpenMP private clause to avoid per-team sharing/races.
src/simulation/m_riemann_solver_hllc.fpp Deduplicates the HLLC private list definition by composing shared + elastic-only fragments to prevent future omissions.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/simulation/m_riemann_solver_hllc.fpp Outdated
Comment thread src/simulation/m_riemann_solver_hllc.fpp Outdated
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown

Lines of Code

File Lines Diff
src/simulation/m_riemann_solver_hllc.fpp 1291 +23
Directory Lines Diff
simulation 27691 +23
total 45461 +23

@codecov

codecov Bot commented Sep 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.26%. Comparing base (8010057) to head (fc8c570).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1823   +/-   ##
=======================================
  Coverage   62.26%   62.26%           
=======================================
  Files          84       84           
  Lines       21558    21558           
  Branches     3188     3195    +7     
=======================================
  Hits        13423    13423           
  Misses       5937     5937           
  Partials     2198     2198           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants