Skip to content

fix: optimize resumable uploads to allow sending bytes during finalization#2146

Merged
BenWhitehead merged 1 commit intomainfrom
unbufferedWriteAndClose
Jul 31, 2023
Merged

fix: optimize resumable uploads to allow sending bytes during finalization#2146
BenWhitehead merged 1 commit intomainfrom
unbufferedWriteAndClose

Conversation

@BenWhitehead
Copy link
Collaborator

Add new methods to UnbufferedWritableByteChannel to allow writing and closing in a single call

  • writeAndClose(ByteBuffer)
  • writeAndClose(ByteBuffer[])
  • writeAndClose(ByteBuffer[], int, int)

Update grpc and json implementation to leverage new methods and to write and finalize in the same call.

DefaultBufferedWritableByteChannel will use the new methods as appropriate.

@BenWhitehead BenWhitehead added the owlbot:ignore instruct owl-bot to ignore a PR label Jul 28, 2023
@BenWhitehead BenWhitehead requested a review from a team as a code owner July 28, 2023 03:25
@product-auto-label product-auto-label bot added size: l Pull request size is large. api: storage Issues related to the googleapis/java-storage API. labels Jul 28, 2023
if (exception instanceof IllegalArgumentException) {
IllegalArgumentException illegalArgumentException = (IllegalArgumentException) exception;
if (illegalArgumentException.getMessage().equals("no JSON input found")) {
if ("no JSON input found".equals(illegalArgumentException.getMessage())) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move the constant to the left side to avoid NullPointerException if there is ever an IllegalArgumentException which doesn't have a message.

@BenWhitehead BenWhitehead force-pushed the unbufferedWriteAndClose branch from 4e299bd to c710b7b Compare July 28, 2023 03:27
@BenWhitehead BenWhitehead requested a review from a team July 28, 2023 03:27
@BenWhitehead BenWhitehead changed the title chore: optimize resumable uploads to allow sending bytes during finalization fix: optimize resumable uploads to allow sending bytes during finalization Jul 28, 2023
…ization

Add new methods to UnbufferedWritableByteChannel to allow writing and closing in a single call
* writeAndClose(ByteBuffer)
* writeAndClose(ByteBuffer[])
* writeAndClose(ByteBuffer[], int, int)

Update grpc and json implementation to leverage new methods and to write and finalize in the same call.

DefaultBufferedWritableByteChannel will use the new methods as appropriate.
@BenWhitehead BenWhitehead force-pushed the unbufferedWriteAndClose branch from c710b7b to 78f8ca9 Compare July 31, 2023 16:44
@Override
default int write(ByteBuffer src) throws IOException {
return Math.toIntExact(write(new ByteBuffer[] {src}));
return Math.toIntExact(write(new ByteBuffer[] {src}, 0, 1));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does 0, 1 represent there's only 1 buffer right now?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. These values represent the index to start from in the provided array and how many indices to consume.

This is reducing a stack entry where before it would do write(ByteBuffer) -> write(ByteBuffer[]) -> write(ByteBuffer[], int, int). While logically equivalent (and the JVM can probably inline the method to reduce impact anyway) doing this makes sure we don't add another stack frame.

@BenWhitehead BenWhitehead merged commit b318075 into main Jul 31, 2023
@BenWhitehead BenWhitehead deleted the unbufferedWriteAndClose branch July 31, 2023 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: storage Issues related to the googleapis/java-storage API. owlbot:ignore instruct owl-bot to ignore a PR size: l Pull request size is large.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants