fix(cp): apply --destination-region in shouldOverride for --no-clobber (#839)#862
Open
gustcol wants to merge 1 commit intopeak:masterfrom
Open
fix(cp): apply --destination-region in shouldOverride for --no-clobber (#839)#862gustcol wants to merge 1 commit intopeak:masterfrom
gustcol wants to merge 1 commit intopeak:masterfrom
Conversation
Fixes peak#839 shouldOverride performs a HEAD request on the destination object to decide whether to skip the copy. doUpload and doCopy both called c.storageOpts.SetRegion(c.dstRegion) before creating the destination client, but shouldOverride did not. When --destination-region is set and the bucket lives in a non-default region, the HEAD was signed for the wrong region, producing HTTP 400 AuthorizationHeaderMalformed. Fix: derive a local dstOpts copy inside shouldOverride and set the region on it before constructing the destination client, matching the pattern already used in doUpload and doCopy. Reproducer: s5cmd cp --no-clobber --destination-region eu-central-1 \ local.txt s3://eu-bucket/object.txt # Fails: AWS Error: AuthorizationHeaderMalformed (400) After this change: HEAD request is signed for eu-central-1; --no-clobber works correctly.
Author
Test resultsUnit tests ( Specific tests: Wasabi smoke test: Before this fix, the HEAD request used the wrong region signing, causing HTTP 400. Now |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Fixes #839
Root cause
shouldOverrideperforms a HEAD request on the destination to determinewhether a copy should be skipped (
--no-clobber,--if-size-differ,--if-source-newer). It created a destinationstorage.Clientusingc.storageOptsdirectly — butc.storageOptsdoes not yet have thedestination region applied at that point.
doUploadanddoCopyboth callc.storageOpts.SetRegion(c.dstRegion)just before creating their destination clients, but
shouldOverridehadno equivalent. When
--destination-regionpoints to a non-default region,the HEAD request was signed for the wrong region, causing:
Fix
Derive a local
dstOptscopy insideshouldOverrideand applydstOpts.SetRegion(c.dstRegion)before constructing the destinationclient — matching the pattern already used in
doUploadanddoCopy.Files changed:
command/cp.goTests
TestShouldOverride_NoFlagsIsNoop— verifies the fast-path (no override flags) still returnsnil, and acts as a compile-time check thatdstRegionis correctly wired into theCopystruct andshouldOverride.Manual verification