[leo_gateway] Cannot get an object with range

[yosukehara]

I've received this issue as below:

I found another related problem. I use aws-java-sdk-s3 access Leofs V1.2.10. When getting object with range, the client sdk throws an AmazonClientException.

S3Object s3Object = s3Client.getObject(new GetObjectRequest(ConfigProps.S3_BUKET, objectKey)
                .withRange(0, 3));

Exception in thread "main" com.amazonaws.AmazonClientException: More data read than expected: dataLength=4; expectedLength=0; includeSkipped=true; in.getClass()=class com.amazonaws.services.s3.AmazonS3Client$2; markedSupported=false; marked=0; resetSinceLastMarked=false; markCount=0; resetCount=0
    at com.amazonaws.util.LengthCheckInputStream.checkLength(LengthCheckInputStream.java:155)
    at com.amazonaws.util.LengthCheckInputStream.read(LengthCheckInputStream.java:110)
    at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:73)
    at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
    at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
    at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
    at java.io.InputStreamReader.read(InputStreamReader.java:184)
    at java.io.BufferedReader.fill(BufferedReader.java:154)
    at java.io.BufferedReader.readLine(BufferedReader.java:317)
    at java.io.BufferedReader.readLine(BufferedReader.java:382)
    at com.lenovo.leoss.client.Utils.displayTextInputStream(Utils.java:43)
    at com.lenovo.leoss.client.Main.main(Main.java:44)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

It seems the aws-java-sdk-s3 need a content-length header in response, but leofs used chunked transfer encoding.

• related issue: [S3] Get object with range header returns empty(wrong) content. #349

[mocchira]

I remembered that S3 doesn't respond http responses with transfer-encoding:chunked.

#153 (comment)

We always need to respond with content-length when handling GET requests.

[yosukehara]

@mocchira Thank you for sharing the info. I'll check that.

[windkit]

Looking into the issue, hope I can propose a pull request this weekend

[yosukehara]

@windkit I'm waiting for your request 👍

[yosukehara]

We've fixed this issue:

• Fixed codes: Handle get_object with range leo_gateway#21
• Test: https://github.com/leo-project/leofs_client_tests/blob/develop/aws-sdk-java/LeoFSSample.java#L213-L243