S3 .get(Range=...) doesn't return ETag

Describe the bug

When specifying a Range parameter for an S3 .get(), the response doesn’t include metadata like ETag. (It works without the Range parameter.)

Steps to reproduce

import boto3
from moto import mock_s3


with mock_s3():

    bucket = 'fake-bucket'
    key = 'my-file'

    boto3.client('s3').create_bucket(
        Bucket=bucket,
        CreateBucketConfiguration={
            'LocationConstraint': 'ap-southeast-2'
        },
    )

    obj = boto3.resource('s3').Object(bucket, key)

    obj.put(Body=b"1234")

    boto3.set_stream_logger('')

    resp = obj.get()
    assert resp['ETag'] == obj.e_tag

    resp = obj.get(Range="bytes=0-1")
    assert resp['ETag'] == obj.e_tag

Expected behavior

Response includes the following top-level keys:

• ResponseMetadata
• AcceptRanges
• LastModified
• ContentLength
• ETag
• ContentRange
• ContentType
• ServerSideEncryption
• Metadata
• Body

actual behavior

Response includes the following top level keys:

• ResponseMetadata
• ContentLength
• ContentRange
• Metadata
• Body

Debug logs Full stack trace by adding boto3.set_stream_logger('') to your code.

These are pretty long, so I only added it after the setup calls.

/usr/lib/python3/dist-packages/requests/__init__.py:89: RequestsDependencyWarning: urllib3 (1.26.6) or chardet (4.0.0) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "
2021-08-20 21:11:25,305 boto3.resources.action [DEBUG] Calling s3:get_object with {'Bucket': 'fake-bucket', 'Key': 'my-file', 'Range': 'bytes=0-1'}
2021-08-20 21:11:25,305 botocore.hooks [DEBUG] Event before-parameter-build.s3.GetObject: calling handler <function sse_md5 at 0x7f85cc8b4ca0>
2021-08-20 21:11:25,305 botocore.hooks [DEBUG] Event before-parameter-build.s3.GetObject: calling handler <function validate_bucket_name at 0x7f85cc8b4c10>
2021-08-20 21:11:25,305 botocore.hooks [DEBUG] Event before-parameter-build.s3.GetObject: calling handler <bound method S3RegionRedirector.redirect_from_cache of <botocore.utils.S3RegionRedirector object at 0x7f85c9f32ca0>>
2021-08-20 21:11:25,305 botocore.hooks [DEBUG] Event before-parameter-build.s3.GetObject: calling handler <bound method S3ArnParamHandler.handle_arn of <botocore.utils.S3ArnParamHandler object at 0x7f85c9f32d60>>
2021-08-20 21:11:25,305 botocore.hooks [DEBUG] Event before-parameter-build.s3.GetObject: calling handler <function generate_idempotent_uuid at 0x7f85cc8b4a60>
2021-08-20 21:11:25,306 botocore.hooks [DEBUG] Event before-call.s3.GetObject: calling handler <function add_expect_header at 0x7f85cc8b4f70>
2021-08-20 21:11:25,306 botocore.hooks [DEBUG] Event before-call.s3.GetObject: calling handler <bound method S3RegionRedirector.set_request_url of <botocore.utils.S3RegionRedirector object at 0x7f85c9f32ca0>>
2021-08-20 21:11:25,306 botocore.hooks [DEBUG] Event before-call.s3.GetObject: calling handler <function inject_api_version_header_if_needed at 0x7f85cc8bb310>
2021-08-20 21:11:25,306 botocore.endpoint [DEBUG] Making request for OperationModel(name=GetObject) with params: {'url_path': '/fake-bucket/my-file', 'query_string': {}, 'method': 'GET', 'headers': {'Range': 'bytes=0-1', 'User-Agent': 'Boto3/1.18.25 Python/3.8.10 Linux/5.4.0-81-generic Botocore/1.21.25 Resource'}, 'body': b'', 'url': 'https://s3.ap-southeast-2.amazonaws.com/fake-bucket/my-file', 'context': {'client_region': 'ap-southeast-2', 'client_config': <botocore.config.Config object at 0x7f85c9f321f0>, 'has_streaming_input': False, 'auth_type': None, 'signing': {'bucket': 'fake-bucket'}}}
2021-08-20 21:11:25,306 botocore.hooks [DEBUG] Event request-created.s3.GetObject: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7f85c9f32160>>
2021-08-20 21:11:25,306 botocore.hooks [DEBUG] Event choose-signer.s3.GetObject: calling handler <bound method ClientCreator._default_s3_presign_to_sigv2 of <botocore.client.ClientCreator object at 0x7f85ca1ed4f0>>
2021-08-20 21:11:25,306 botocore.hooks [DEBUG] Event choose-signer.s3.GetObject: calling handler <function set_operation_specific_signer at 0x7f85cc8b4940>
2021-08-20 21:11:25,306 botocore.hooks [DEBUG] Event before-sign.s3.GetObject: calling handler <bound method S3EndpointSetter.set_endpoint of <botocore.utils.S3EndpointSetter object at 0x7f85c9f32df0>>
2021-08-20 21:11:25,307 botocore.utils [DEBUG] Checking for DNS compatible bucket for: https://s3.ap-southeast-2.amazonaws.com/fake-bucket/my-file
2021-08-20 21:11:25,307 botocore.utils [DEBUG] URI updated to: https://fake-bucket.s3.ap-southeast-2.amazonaws.com/my-file
2021-08-20 21:11:25,307 botocore.auth [DEBUG] Calculating signature using v4 auth.
2021-08-20 21:11:25,307 botocore.auth [DEBUG] CanonicalRequest:
GET
/my-file

host:fake-bucket.s3.ap-southeast-2.amazonaws.com
range:bytes=0-1
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20210820T111125Z

host;range;x-amz-content-sha256;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
2021-08-20 21:11:25,307 botocore.auth [DEBUG] StringToSign:
AWS4-HMAC-SHA256
20210820T111125Z
20210820/ap-southeast-2/s3/aws4_request
bf91fce9fa54974a782ae9089a298156beb3a9f9d05ce8db4a6a2ff9590e7f55
2021-08-20 21:11:25,307 botocore.auth [DEBUG] Signature:
4a8d7a50dd47910b266396a81e1adcb6531c6688f8f55a12e2b20eb7d1259222
2021-08-20 21:11:25,307 botocore.endpoint [DEBUG] Sending http request: <AWSPreparedRequest stream_output=True, method=GET, url=https://fake-bucket.s3.ap-southeast-2.amazonaws.com/my-file, headers={'Range': b'bytes=0-1', 'User-Agent': b'Boto3/1.18.25 Python/3.8.10 Linux/5.4.0-81-generic Botocore/1.21.25 Resource', 'X-Amz-Date': b'20210820T111125Z', 'X-Amz-Content-SHA256': b'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': b'AWS4-HMAC-SHA256 Credential=foobar_key/20210820/ap-southeast-2/s3/aws4_request, SignedHeaders=host;range;x-amz-content-sha256;x-amz-date, Signature=4a8d7a50dd47910b266396a81e1adcb6531c6688f8f55a12e2b20eb7d1259222'}>
2021-08-20 21:11:25,307 botocore.hooks [DEBUG] Event before-send.s3.GetObject: calling handler <moto.core.models.BotocoreStubber object at 0x7f85cc5ae2e0>
2021-08-20 21:11:25,308 botocore.parsers [DEBUG] Response headers: {'content-range': 'bytes 0-1/4', 'content-length': 2}
2021-08-20 21:11:25,308 botocore.parsers [DEBUG] Response body:
<botocore.response.StreamingBody object at 0x7f85c9fab5e0>
2021-08-20 21:11:25,308 botocore.hooks [DEBUG] Event needs-retry.s3.GetObject: calling handler <botocore.retryhandler.RetryHandler object at 0x7f85c9f32c40>
2021-08-20 21:11:25,308 botocore.retryhandler [DEBUG] No retry needed.
2021-08-20 21:11:25,308 botocore.hooks [DEBUG] Event needs-retry.s3.GetObject: calling handler <bound method S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7f85c9f32ca0>>
2021-08-20 21:11:25,308 boto3.resources.action [DEBUG] Response: {'ResponseMetadata': {'HTTPStatusCode': 206, 'HTTPHeaders': {'content-range': 'bytes 0-1/4', 'content-length': 2}, 'RetryAttempts': 0}, 'ContentLength': 2, 'ContentRange': 'bytes 0-1/4', 'Metadata': {}, 'Body': <botocore.response.StreamingBody object at 0x7f85c9fab5e0>}
Traceback (most recent call last):
  File "etag.py", line 24, in <module>
    assert resp['ETag'] == obj.e_tag
KeyError: 'ETag'

[mdavis-xyz]

Ah, woops. My apologies. I meant to raise this issue on the moto repo. It’s an issue with moto, and I just got my browser tabs mixed up.

[mdavis-xyz]

Raised on the moto repo here