question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

SQS returns uncapitalized dataType, stringValue, but Boto3 requires capitalization

See original GitHub issue

When using SQS as a Lambda trigger, the MessageAttributes return values with “dataType” and “stringValue”, starting with lower case letters. When trying to send to an SQS queue, Boto3 requires capitalized “DataType” and “StringValue” to be part of the messageAttributes.

This is confusing and I think should be altered, as it looks odd to capitalize differently depending on whether you are sending or receiving a message.

Example: Sent the following to SQS queue

'messageAttributes': {
        'startTime': {
          'StringValue': '20190929T000000',
          'DataType': 'String'
        },
        'endTime': {
          'StringValue': '20191006T000000',
          'DataType': 'String'
        },

Received the following in the Lambda event

'messageAttributes': {
        'startTime': {
          'stringValue': '20190929T000000',
          'stringListValues': [],
          'binaryListValues': [],
          'dataType': 'String'
        },
        'endTime': {
          'stringValue': '20191006T000000',
          'stringListValues': [],
          'binaryListValues': [],
          'dataType': 'String'
        },

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:2
  • Comments:9 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
swetashrecommented, Jan 22, 2020

@Tylermarques - Thank you for providing me the steps to reproduce the issue. Now i am able to reproduce the issue. This is because lambda process the event in different way than the service SQS. I have internally escalated this issue to the service team. I will post here when i get any update.

1reaction
vfilimonovcommented, Jan 13, 2020

Hello @swetashre

The example is actually trivial. Put it in a newly created lambda function, add SQS permissions and connect to some SQS feed.

import json

def lambda_handler(event, context):
    print(json.dumps(event, indent=2).replace('\n', '\t'))
    return dict(event)

Upon adding the message in the SQS - check the CloudWatch logs. You’ll see the event as it is received by lambda - non-capitalized:

{
    "Records": [
        {
            "messageId": "...",
            "receiptHandle": "...",
            "body": "hello there",
            "attributes": {
                "ApproximateReceiveCount": "1",
                "SentTimestamp": "1578907121082",
                "SequenceNumber": "...",
                "MessageGroupId": "lambdas",
                "SenderId": "...",
                "MessageDeduplicationId": "...",
                "ApproximateFirstReceiveTimestamp": "1578907121082"
            },
            "messageAttributes": {
                "Name": {
                    "stringValue": "SQS-test",
                    "stringListValues": [],
                    "binaryListValues": [],
                    "dataType": "String"
                }
            },
            "md5OfMessageAttributes": "",
            "md5OfBody": "...",
            "eventSource": "aws:sqs",
            "eventSourceARN": "...",
            "awsRegion": ""
        }
    ]
}

Note:

  1. Records -> messageAttributes -> Name -> stringValue (and as you’ve seen, Boto3 function “receive_message” would return StringValue)
  2. Records -> messageId (and Boto3 would return MessageId)
  3. The structure is called Records (and Boto3 would return Messages)

All of these inconsitencies make it quite problematic to develop code which works with both boto3-responses and the direct responses from AWS.

Read more comments on GitHub >

github_iconTop Results From Across the Web

class SQS. Client - Boto3 Docs 1.26.31 documentation - AWS
If the queue name, attribute names, or attribute values don't match an existing queue, CreateQueue returns an error.
Read more >
MessageAttributeValue - Amazon Simple Queue Service
The user-specified message attribute value. For string data types, the Value attribute has the same restrictions on the content as the message body....
Read more >
Unable to send the MessageAttributes to AWS SQS using ...
This should fix your code but right way is to have a look at this import boto3 sqs = boto3.client('sqs') response = sqs.queue.send_message( ......
Read more >
Working with Amazon SQS in Python using Boto3
This article covers managing SQS queues, concepts of working with SQS messages, ... In general, here's what you need to have installed:.
Read more >
Ultimate Guide to Python SQS: 13+ Operations With Easy ...
You'll learn to create Python SQS Queues using the Boto3 library, set up Queue permissions, attributes, and tags, configure Queue URLs, ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found