ci: use project bucket for assets COMPASS-10105#7682
Conversation
| - command: s3.put | ||
| params: | ||
| <<: *save-artifact-params-public | ||
| <<: *save-artifact-params-private |
There was a problem hiding this comment.
Do these needs to be -private instead of -public? With -private, we can't pass URLs to these builds to people anymore (esp. customers outside of the org), and it makes bulk downloads quite a bit more annoying (which I've had to do for SSDLC artifact compilation from time to time)
There was a problem hiding this comment.
I am checking this with the team if these can be public.
There was a problem hiding this comment.
So, backstage created s3 buckets with public access disabled and that's why I was not able to upload assets with public access which forced me to use private. DEVPROD will change this and I'll revert this bit.
There was a problem hiding this comment.
Okay, so DEVPROD wants us to keep the assets with private access. Regarding ssdlc report, would it help if we prioritize COMPASS-8030?
| ) | ||
|
|
||
| URL="https://mciuploads.s3.amazonaws.com/${EVERGREEN_PROJECT}/compass/dev/$1" | ||
| URL="https://s3.amazonaws.com/downloads.10gen.com/compass/dev/$1" |
There was a problem hiding this comment.
TODO: Update update-server with this url
There was a problem hiding this comment.
|
In order to avoid having aws credentials for update server, compass (gha), DEVPROD-26085 helped us with |
There was a problem hiding this comment.
Pull request overview
This PR migrates artifact storage from the generic mciuploads S3 bucket to a project-specific cdn-origin-compass-dev bucket, aligning with MongoDB's infrastructure modernization efforts.
Changes:
- Updated S3 bucket references from
mciuploadstocdn-origin-compass-devwith role-based authentication - Removed
EVERGREEN_BUCKET_NAMEenvironment variable dependency across the codebase - Introduced build attestations module to systematically manage artifact metadata and upload paths
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
.evergreen/functions.yml |
Updated artifact storage parameters to use project-specific bucket with role ARN authentication |
packages/hadron-build/lib/download-center.js |
Modified download URL to use new CDN path structure |
packages/hadron-build/lib/build-attestations.js |
New module for managing build attestation file paths and metadata |
packages/hadron-build/commands/upload.js |
Integrated attestations into the upload workflow |
packages/hadron-build/commands/download.js |
Added attestation download support |
packages/compass-smoke-tests/src/test-subject.ts |
Updated download URL to new CDN location |
packages/compass-smoke-tests/src/dispatch.ts |
Removed bucket name parameter from workflow dispatch |
packages/compass-smoke-tests/src/context.ts |
Removed bucket name from context type |
packages/compass-smoke-tests/src/cli.ts |
Removed bucket name CLI option and validation |
.github/workflows/test-installers.yml |
Removed bucket name workflow input |
.evergreen/print-debug-info.sh |
Removed bucket name from debug output |
.evergreen/buildvariants-and-tasks.yml |
Updated first-party dependency file path structure |
.evergreen/buildvariants-and-tasks.in.yml |
Updated first-party dependency filename template |
.evergreen/build-dev-release-info.sh |
Updated release info URL to use new CDN location |
packages/compass-smoke-tests/package.json |
Reordered dependencies alphabetically |
packages/hadron-build/test/build-attestations.spec.js |
Added comprehensive test coverage for new attestations module |
addaleax
left a comment
There was a problem hiding this comment.
Looks great! I'd suggest holding off on merging this until next week (after the AI assistant release + any potential hotfixes afterwards), if that's possible
…m/mongodb-js/compass into COMPASS-10105-use-project-buckets
…PASS-10105-use-project-buckets
We are moving away from storing artifacts in
mciuploadsto project specific S3 buckets. DEVPROD-25311 documents how to create these buckets.Description
Checklist
Motivation and Context
Open Questions
Dependents
Types of changes