Skip to content

feat(storage): support delete_source_objects in compose API (Address PR feedback)#17057

Draft
nidhiii-27 wants to merge 2 commits into
mainfrom
feat/storage-compose-delete-source-objects-5220279067164674388
Draft

feat(storage): support delete_source_objects in compose API (Address PR feedback)#17057
nidhiii-27 wants to merge 2 commits into
mainfrom
feat/storage-compose-delete-source-objects-5220279067164674388

Conversation

@nidhiii-27
Copy link
Copy Markdown
Contributor

This update addresses the PR comments by:

  • Improving the system test cleanup logic and removing unnecessary fixture dependencies for the specific test case.
  • Enhancing the unit test to verify that destination properties (like contentType) are correctly passed in the request body during a compose operation.
  • Re-verifying all unit and system tests pass.

PR created automatically by Jules for task 5220279067164674388 started by @nidhiii-27

…PR feedback)

- Removed unnecessary fixture usage in system test.
- Ensured destination metadata is not empty in unit test by setting content_type.
- verified end-to-end functionality in system test.

Reference PR: googleapis/google-cloud-java#12873

Co-authored-by: nidhiii-27 <224584462+nidhiii-27@users.noreply.github.com>
@google-labs-jules
Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a delete_source_objects parameter to the Blob.compose method, allowing source blobs to be automatically deleted after a successful composition. The changes include updates to the method signature, docstrings, and request logic, along with new unit and system tests. Feedback was provided to improve the robustness of the system test by using the blobs_to_delete fixture for resource cleanup instead of a manual try...finally block, which helps prevent resource leaks and avoids masking potential errors.

Comment on lines +856 to +873
def test_blob_compose_delete_source_objects(shared_bucket):
payload_1 = b"AAA\n"
source_1 = shared_bucket.blob("source-1-delete")
source_1.upload_from_string(payload_1)

payload_2 = b"BBB\n"
source_2 = shared_bucket.blob("source-2-delete")
source_2.upload_from_string(payload_2)

destination = shared_bucket.blob("destination-delete")
destination.compose([source_1, source_2], delete_source_objects=True)

try:
assert destination.download_as_bytes() == payload_1 + payload_2
assert not source_1.exists()
assert not source_2.exists()
finally:
destination.delete()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The manual try...finally block for cleanup is less robust than using the blobs_to_delete fixture. If destination.compose fails before the object is created, destination.delete() in the finally block will raise a NotFound exception, which can mask the original error. Furthermore, if the test fails during the upload or composition steps, the source blobs will be leaked. Using the standard blobs_to_delete fixture ensures all resources are cleaned up safely even on failure and simplifies the test logic.

Suggested change
def test_blob_compose_delete_source_objects(shared_bucket):
payload_1 = b"AAA\n"
source_1 = shared_bucket.blob("source-1-delete")
source_1.upload_from_string(payload_1)
payload_2 = b"BBB\n"
source_2 = shared_bucket.blob("source-2-delete")
source_2.upload_from_string(payload_2)
destination = shared_bucket.blob("destination-delete")
destination.compose([source_1, source_2], delete_source_objects=True)
try:
assert destination.download_as_bytes() == payload_1 + payload_2
assert not source_1.exists()
assert not source_2.exists()
finally:
destination.delete()
def test_blob_compose_delete_source_objects(shared_bucket, blobs_to_delete):
payload_1 = b"AAA\n"
source_1 = shared_bucket.blob("source-1-delete")
source_1.upload_from_string(payload_1)
blobs_to_delete.append(source_1)
payload_2 = b"BBB\n"
source_2 = shared_bucket.blob("source-2-delete")
source_2.upload_from_string(payload_2)
blobs_to_delete.append(source_2)
destination = shared_bucket.blob("destination-delete")
blobs_to_delete.append(destination)
destination.compose([source_1, source_2], delete_source_objects=True)
assert destination.download_as_bytes() == payload_1 + payload_2
assert not source_1.exists()
assert not source_2.exists()

…PR feedback)

- Re-added blobs_to_delete fixture in system test.
- Implemented conditional cleanup for source objects in system test.
- Ensured destination metadata is populated in unit test.
- verified end-to-end functionality.

Reference PR: googleapis/google-cloud-java#12873

Co-authored-by: nidhiii-27 <224584462+nidhiii-27@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant