question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Firehose client `generate_presigned_url` method throws exception

See original GitHub issue

When trying to generate a presigned URL for a firehose put_record operation, the following exception is thrown:

Traceback (most recent call last):
  File "foo.py", line 17, in <module>
    'Data': b'FOOBARBAZQUX'
  File "[...]/boto3/venv/lib/python3.4/site-packages/botocore/signers.py", line 517, in generate_presigned_url
    operation_name=operation_name)
  File "[...]/boto3/venv/lib/python3.4/site-packages/botocore/signers.py", line 253, in generate_presigned_url
    'presign-url', expires_in)
  File "[...]/boto3/venv/lib/python3.4/site-packages/botocore/signers.py", line 147, in sign
    auth.add_auth(request)
  File "[...]/boto3/venv/lib/python3.4/site-packages/botocore/auth.py", line 319, in add_auth
    self._modify_request_before_signing(request)
  File "[...]/boto3/venv/lib/python3.4/site-packages/botocore/auth.py", line 459, in _modify_request_before_signing
    query_dict.update(request.data)
TypeError: cannot convert dictionary update sequence element #0 to a sequence

The following code demonstrates the issue:

import boto3

kinesis = boto3.client(
    'firehose',
    region_name='us-east-1',
    aws_access_key_id ='KEY',
    aws_secret_access_key='SECRET'
)

kinesis.generate_presigned_url(
    'put_record',
    Params={
        'DeliveryStreamName':'STREAM',
        'Record': {
            'Data': b'FOOBARBAZQUX'
        }
    }
)

It appears as if the request body is being converted to a bytestring via serialize_to_request (https://github.com/boto/botocore/blob/develop/botocore/serialize.py#L307). However, when the presigning happens later, the request body is assumed to be a dictionary request_signer.generate_presigned_url (https://github.com/boto/botocore/blob/develop/botocore/auth.py#L459)

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:7 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
swetashrecommented, May 1, 2020

@ebenpack - Following up on this issue. I am not getting runtime error now but it looks like service does not support this. I am marking this as documentation as we may want to remove it from documentation.

0reactions
swetashrecommented, Nov 5, 2020

Let’s track this issue under the issue https://github.com/boto/boto3/issues/1354

I am closing this one as duplicate.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Class Aws\Firehose\Exception\FirehoseException
Kinesis Data Firehose throws this exception when an attempt to put records or to start or stop delivery stream encryption fails. This happens...
Read more >
Aws::Firehose::FirehoseClient Class Reference - Amazon AWS
If the PutRecord operation throws a ServiceUnavailableException , back off and retry. If the exception persists, it is possible that the throughput limits...
Read more >
AmazonS3Client.generatePresignedUrl(...) - Tabnine
@Override public URL generatePresignedUrl(String bucketName, String key, Date expiration, HttpMethod method) throws SdkClientException ...
Read more >
AmazonS3 (AWS SDK for Java - 1.11.56)
Note: Do not directly implement this interface, new methods are added to it ... Creates a new Amazon S3 bucket in the region...
Read more >
Getting errors in java program to write to Kinesis Firehose ...
Also getting an error on the second line above: "The method putRecord(com.amazonaws.services.kinesisfirehose.model.PutRecordRequest) in the ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found