Recommendation for testing releases and producing artifacts before tagging

[mx-psi]

Hi here from the OpenTelemetry Collector community!

We use GoReleaser Pro and are trying to improve our release process (see open-telemetry/opentelemetry-collector-releases/issues/773). Ideally, we would like to get to a state where we can fix any release issues before pushing any particular tag to the repository, but we have not been able to find a way to do this with goreleaser. Our ask is similar to #1454 (comment) and #3174 (comment).

We have tried to use goreleaser release --prepare for this, since it seems like a good fit, but we are finding the following issues:

1. It is difficult for us to understand what goreleaser release --prepare does exactly and how it is supposed to work. It would be helpful to have a clear example of usage of it.
2. goreleaser prepare --release requires us to push a tag for it to prepare a release. We would like to avoid that, since part of the point is to not have to push a tag until we have verified that the artifacts (i) can be produced by the release pipeline and (ii) pass any validation we may want to do.

I am filing this discussion so we can get some clarity/recommendations regarding how to use goreleaser for our use case and get feedback on the two issues above from the goreleaser maintainers. Thank you for maintaining goreleaser, it is serving us well!

[caarlos0]

Hi!

You have a couple of options, depending on how complete of a test you need.

The first and easier one: goreleaser release --snapshot - will create all artifacts, but will not publish anything, and doesn't need a new tag

The next option is if you want to test the whole pipeline. Unfortunately, there's no way of doing it that is reversible, so my recommendation would be to run goreleaser release against a fork of the repository, or in a test repository, especially if you want to see what exactly will happen when you do the full process.

goreleaser release --prepare is intended to be used if you want to create the artifacts, run some check on them, and then publish them. It's a full release, made in steps.

Hope that helps - let me know if you have any more questions 🙏🏻

[caarlos0]

Yeah, so, there's no way to test out the whole flow without actually running it, unfortunately.

Best advice I can give is to play with it in a throwaway repo for a while, and then set it up for real.

Once you get it working, it hardly breaks, though.

Main thing to pay attention to is the permissions of GitHub tokens and other integrations, everything else is usually smooth sailing.

[mowies]

That is hard to do, both from a token perspective and from a contributor experience perspective in a big open source project with many different developers. We started working on nightly builds instead to at least run the release pipelines more regularly than on actual releases.

[caarlos0]

yes, nightlies are a good call as well.

[fredbi]

@caarlos0 trying to achieve the same, using a fork repo as a dress rehearsal environment (with appropriate associated testing namespaces for artifact registries). This should allow for a complete end-to-end rehearsal of the release process.

unfortunately, a few steps remain impossible to variabilize, to wit the release publishing part.

hopefully this could be addressed by allowing a few overrides in the release schema.

would you like a PR for that?

fred

[caarlos0]

@fredbi overrides of what specifically?