Skip to content

NPE if s3AsyncClient.putObject is called with body containing empty content and unknown length #6464

@PeterWittmann

Description

@PeterWittmann

Describe the bug

Excecuting main (see reproductions steps) results in ExecutionException with a NPE as cause.

main works if either

  • content contain at least one byte or
  • content length inAsyncRequestBody.fromInputStream() is set to 0 (instead of unknown: null)

main does not work with content=""; and contentlength=null;

I used google storage, but I don't think that the issue is google specific

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

Main should put empty object without Exception

Current Behavior

Exception is thrown:
Exception in thread "main" java.util.concurrent.ExecutionException: software.amazon.awssdk.core.exception.SdkClientException: Cannot invoke "software.amazon.awssdk.core.async.AsyncRequestBody.contentType()" because "this.asyncRequestBody" is null
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)
at com.opentext.ecm.ac.migration.server.ContentNPE.main(ContentNPE.java:58)
Caused by: software.amazon.awssdk.core.exception.SdkClientException: Cannot invoke "software.amazon.awssdk.core.async.AsyncRequestBody.contentType()" because "this.asyncRequestBody" is null
at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:130)
at software.amazon.awssdk.core.internal.util.ThrowableUtils.asSdkException(ThrowableUtils.java:98)
at software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.doExecute(BaseAsyncClientHandler.java:248)
at software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.lambda$execute$1(BaseAsyncClientHandler.java:80)
at software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.measureApiCallSuccess(BaseAsyncClientHandler.java:294)
at software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.execute(BaseAsyncClientHandler.java:73)
at software.amazon.awssdk.awscore.client.handler.AwsAsyncClientHandler.execute(AwsAsyncClientHandler.java:49)
at software.amazon.awssdk.services.s3.DefaultS3AsyncClient.putObject(DefaultS3AsyncClient.java:12982)
at software.amazon.awssdk.services.s3.DelegatingS3AsyncClient.lambda$putObject$89(DelegatingS3AsyncClient.java:10008)
at software.amazon.awssdk.services.s3.internal.multipart.MultipartS3AsyncClient$1.invokeOperation(MultipartS3AsyncClient.java:108)
at software.amazon.awssdk.services.s3.DelegatingS3AsyncClient.putObject(DelegatingS3AsyncClient.java:10008)
at software.amazon.awssdk.services.s3.internal.multipart.MultipartUploadHelper.uploadInOneChunk(MultipartUploadHelper.java:155)
at software.amazon.awssdk.services.s3.internal.multipart.UploadWithUnknownContentLengthHelper$UnknownContentLengthAsyncRequestBodySubscriber.onComplete(UploadWithUnknownContentLengthHelper.java:282)
at software.amazon.awssdk.utils.async.SimplePublisher.doProcessQueue(SimplePublisher.java:275)
at software.amazon.awssdk.utils.async.SimplePublisher.processEventQueue(SimplePublisher.java:224)
at software.amazon.awssdk.utils.async.SimplePublisher.complete(SimplePublisher.java:157)
at software.amazon.awssdk.core.internal.async.SplittingPublisher$SplittingSubscriber.onComplete(SplittingPublisher.java:247)
at software.amazon.awssdk.utils.async.SimplePublisher.doProcessQueue(SimplePublisher.java:275)
at software.amazon.awssdk.utils.async.SimplePublisher.processEventQueue(SimplePublisher.java:224)
at software.amazon.awssdk.utils.async.SimplePublisher.complete(SimplePublisher.java:157)
at software.amazon.awssdk.utils.async.InputStreamConsumingPublisher.doBlockingWrite(InputStreamConsumingPublisher.java:62)
at software.amazon.awssdk.core.async.BlockingInputStreamAsyncRequestBody.writeInputStream(BlockingInputStreamAsyncRequestBody.java:95)
at software.amazon.awssdk.core.internal.async.InputStreamWithExecutorAsyncRequestBody.doBlockingWrite(InputStreamWithExecutorAsyncRequestBody.java:112)
at software.amazon.awssdk.core.internal.async.InputStreamWithExecutorAsyncRequestBody.lambda$subscribe$0(InputStreamWithExecutorAsyncRequestBody.java:80)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.lang.NullPointerException: Cannot invoke "software.amazon.awssdk.core.async.AsyncRequestBody.contentType()" because "this.asyncRequestBody" is null
at software.amazon.awssdk.core.runtime.transform.AsyncStreamingRequestMarshaller.marshall(AsyncStreamingRequestMarshaller.java:50)
at software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$finalizeSdkHttpFullRequest$0(BaseClientHandler.java:73)
at software.amazon.awssdk.core.internal.util.MetricUtils.measureDuration(MetricUtils.java:64)
at software.amazon.awssdk.core.internal.handler.BaseClientHandler.finalizeSdkHttpFullRequest(BaseClientHandler.java:72)
at software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.doExecute(BaseAsyncClientHandler.java:199)
... 26 more

Reproduction Steps

public static void main(String[] args) throws ExecutionException, InterruptedException {

    String bucket                   = "<your bucket>";
    URI endpointOverride            = URI.create( "https://storage.googleapis.com/"); // same result for AWS
    String region                   = "<your region>";
    AwsBasicCredentials credentials = AwsBasicCredentials.create( "<your access key>"
                                                                ,"<your secret>" );

    // "" throws ExecutionException with NPE. Works if not empty e.g. "a"
    String content="";
    InputStream inStream = new ByteArrayInputStream(content.getBytes(StandardCharsets.UTF_8));

    S3AsyncClient s3AsyncClient = S3AsyncClient.builder()
            .credentialsProvider( StaticCredentialsProvider.create( credentials ) )
            .endpointOverride( endpointOverride )
            .forcePathStyle( Boolean.TRUE )
            .multipartEnabled(true)
            .httpClientBuilder( NettyNioAsyncHttpClient.builder() )
            .region( Region.of( region ) )
            .requestChecksumCalculation(RequestChecksumCalculation.WHEN_REQUIRED)     // needed for google
            .responseChecksumValidation(ResponseChecksumValidation.WHEN_REQUIRED)     // needed for google
            .build();

    ExecutorService executor = Executors.newSingleThreadExecutor();
    try {
        AsyncRequestBody body = AsyncRequestBody.fromInputStream(inStream, (Long) null /* contentLength unknown */, executor);
        CompletableFuture<PutObjectResponse> poFuture = s3AsyncClient.putObject(PutObjectRequest.builder()
                        .bucket(bucket)
                        .key( "testContent/empty.txt")
                        .build(),
                body);

        PutObjectResponse poRes = poFuture.get(); // may throw InterruptedException or ExecutionException
        if (poRes == null || !poRes.sdkHttpResponse().isSuccessful()) {
            throw new ExecutionException(new IOException("Object upload failed"));
        }
        System.out.println("Success for content=\"" + content + "\"");
    } finally {
        executor.shutdownNow();
    }
}

Possible Solution

No response

Additional Information/Context

Workaround:
I use a BufferedInputStream to detect, if inStream is empty or not. If empty I set contentlength=0 in AsyncRequestBody.fromInputStream(). If not empty, I usecontentlength=null

AWS Java SDK version used

2.34.5

JDK version used

openjdk17.0.15

Operating System and version

windows 11

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugThis issue is a bug.needs-triageThis issue or PR still needs to be triaged.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions