Non-deterministic META-INF/services/ entry inside shaded archive

[mthmulders]

Affected version

3.6.2

Bug description

The position of an META-INF/services/ directory entry in a shaded JAR depends on which iteration order project.getArtifacts() returned the first dependency with a services file inside.

The Shade plugin writes a META-INF/services/ directory entry at the current output position as soon as it encounters the first file META-INF/services/. It then delegates writing the services file itself to ServicesResourceTransformer (which writes the file at the end). Since the order in which project.getArtifacts() returns artifacts is unpredictable, the directory entry position is also non-deterministic dependency, potentially causing an non-reproducible build.

seen when rebuilding https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/content/it/mulders/mcs/README.md (and quite a few others like OpenFastrace https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/content/org/itsallcode/openfasttrace/openfasttrace-4.1.0.diffoscope#L13 / https://github.com/jvm-repo-rebuild/reproducible-central/blob/master/content/org/itsallcode/openfasttrace/README.md)

[hboutemy]

is project.getArtifacts() really non-deterministic?
surprised
but clearly, there is an order confusion happening in a few very specific cases

[mthmulders]

is project.getArtifacts() really non-deterministic?

Good question. I took three samples (with Maven 3.9.16 on Java 25) but the outcome of that method call is identical in all three cases. It's not a complete proof of course but at least it's suspicious.

I gave the implementation of project.getArtifacts() to Copilot and asked it, if the order of artifacts in the resulting Set is deterministic or non-deterministic. It claims the order is "iteration-order deterministic only relative to the input order" - put differently, it is deterministic only if the ordering inside resolvedArtifacts is deterministic.

[hboutemy]

getArtifacts() is used to create classpath: if it was still non-deterministic, it would have been detected for a long time (I fixed classpath issues in 2.0.0-beta-5 20 years ago, it was one of my first contributions to Maven...)

I rebuild mcs release with -X, the only META-INF/services content comes from jackson-core-2.22.0.jar

mcs is a really great project to dive into this issue, as the build is really small

but for now, I don't get what is happenning...

[mthmulders]

getArtifacts() is used to create classpath: if it was still non-deterministic, it would have been detected for a long time

That makes sense! It would propagate into many reproducibility issues. I think we should for now cross that off as "highly unlikely".

I rebuild mcs release with -X, the only META-INF/services content comes from jackson-core-2.22.0.jar

Yep, that's something I also noted. I'll have a closer look at how that particular JAR gets handled, and if/why/when/where the META-INF/services folder comes into play. I already debugged a bit, didn't see anything obvious so far, but I'll have another look.

[cstamas]

If you switch between Maven 3.9 and 3.10/4.0/4.1 you may sense different ordering as Resolver 2.x "flattens" level-order, while "classic" Maven 3.9 and olders are pre-order. Not sure this is the case, but may be related.

See apache/maven#12327

[mthmulders]

If you switch between Maven 3.9 and 3.10/4.0/4.1

That's an interesting remark! Local builds are always with Maven Wrapper, so the version of Maven is stable. In theory, that is. Because:

1. Specifically, version 0.10.1 may have used Maven 4.0, or 3.9 - I'm not sure. That's why I've recently introduced Maven Wrapper afterwards.
2. GitHub actions builds are with a global Maven version. Those only produce the native artifacts, but still... I've pushed an update that should make those use Maven Wrapper as well.