Unable to stream models to ElasticSearch

What is the current behavior? When I create a mutation, the lambda function that streams dynamodb models to elasticsearch fails and logs Unable to import module 'python_streaming_function': cannot import name 'BotocoreHTTPSession'. I haven’t edited any of the code as it is automatically generated by the GraphQL Transform

What is the expected behavior? I should be able to search my models in AWS elasticsearch

Which versions and which environment (browser, react-native, nodejs) / OS are affected by this issue? Did this work in previous versions?

    "aws-amplify": "^1.1.40",
    "aws-amplify-react-native": "^2.1.19",
    "aws-appsync": "^2.0.0",
    "aws-appsync-react": "^2.0.0",
    "react": "^16.9.0",
    "react-apollo": "2.5.8",
    "react-native": "^0.60.5",

     aws-cli/1.16.243 Python/3.6.0 Windows/10 botocore/1.12.233

[sollipse]

+1 We have started seeing this issue in an extremely similar timeframe.

Has the version of Botocore made available to AWS Lambda changed? Alternatively, have the amplify team made any changes to the underlying zipfile housing this Streaming code?

Please treat this as a high severity issue. As Amplify does not offer a way to “refresh” entries in ES, the only remedy to missing ES records at them moment is a full table rewrite.

[kaustavghosh06]

@usmansbk We dug into the issue and after talking to the Lambda team, we understood the reason for the change and errors. We’re working on a fix in the CLI that will enable you to re-deploy your function as a part of the ampliy push command to avoid seeing the Lambda errors. For the time being, here’s an explanation and the workaround provided by the Lambda team to prevent your app from seeing errors:

AWS Lambda is updating the default boto3 and botocore SDKs for the Python 2.7, Python 3.6 and Python 3.7 runtimes. This update changes the version of boto3 from 1.7.74 to 1.9.221 for the Python 2.7 and Python 3.6 runtimes and from 1.9.42 to 1.9.221 for the Python 3.7 runtime. The update contains SDK enhancements and updated APIs. One notable example of an internal class that underwent change is the BotocoreHTTPSessions class which is no longer supported starting with botocore version 1.11.0.

Since the CLI used the BotocoreHTTPSessions class, that’s the reason why you’re seeing these errors.

As a workaround, you can update your DdbToEsFn-xxxxxxfunction in the AWS Console to use a Lambda layer that contains the previous version of the SDK. For your convenience, the AWS Lambda team already published these layers for you to use. You can find additional information on Lambda layers out here - https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html

Here are the list of lambda layer ARN’s which you can use based on the region you are in:_

ap-northeast-1: arn:aws:lambda:ap-northeast-1:249908578461:layer:AWSLambda-Python-AWS-SDK:1
us-east-1: arn:aws:lambda:us-east-1:668099181075:layer:AWSLambda-Python-AWS-SDK:1
ap-southeast-1: arn:aws:lambda:ap-southeast-1:468957933125:layer:AWSLambda-Python-AWS-SDK:1
eu-west-1: arn:aws:lambda:eu-west-1:399891621064:layer:AWSLambda-Python-AWS-SDK:1
us-west-1: arn:aws:lambda:us-west-1:325793726646:layer:AWSLambda-Python-AWS-SDK:1
ap-east-1: arn:aws:lambda:ap-east-1:118857876118:layer:AWSLambda-Python-AWS-SDK:1
ap-northeast-2: arn:aws:lambda:ap-northeast-2:296580773974:layer:AWSLambda-Python-AWS-SDK:1
ap-northeast-3: arn:aws:lambda:ap-northeast-3:961244031340:layer:AWSLambda-Python-AWS-SDK:1
ap-south-1:631267018583: arn:aws:lambda:ap-south-1:631267018583:layer:AWSLambda-Python-AWS-SDK:1
ap-southeast-2: arn:aws:lambda:ap-southeast-2:817496625479:layer:AWSLambda-Python-AWS-SDK:1
ca-central-1: arn:aws:lambda:ca-central-1:778625758767:layer:AWSLambda-Python-AWS-SDK:1
eu-central-1: arn:aws:lambda:eu-central-1:292169987271:layer:AWSLambda-Python-AWS-SDK:1
eu-north-1: arn:aws:lambda:eu-north-1:642425348156:layer:AWSLambda-Python-AWS-SDK:1
eu-west-2: arn:aws:lambda:eu-west-2:142628438157:layer:AWSLambda-Python-AWS-SDK:1
eu-west-3: arn:aws:lambda:eu-west-3:959311844005:layer:AWSLambda-Python-AWS-SDK:1
sa-east-1: arn:aws:lambda:sa-east-1:640010853179:layer:AWSLambda-Python-AWS-SDK:1us-us-east-2: arn:aws:lambda:us-east-2:259788987135:layer:AWSLambda-Python-AWS-SDK:1
us-west-2: arn:aws:lambda:us-west-2:420165488524:layer:AWSLambda-Python-AWS-SDK:1
cn-north-1: arn:aws-cn:lambda:cn-north-1:683298794825:layer:AWSLambda-Python-AWS-SDK:1
cn-northwest-1: arn:aws-cn:lambda:cn-northwest-1:382066503313:layer:AWSLambda-Python-AWS-SDK:1
us-gov-west-1:: arn:aws-us-gov:lambda:us-gov-west-1:556739011827:layer:AWSLambda-Python-AWS-SDK:1
us-gov-east-1: arn:aws-us-gov:lambda:us-gov-east-1:138526772879:layer:AWSLambda-Python-AWS-SDK:1