question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Bug: Subsegment ssm discarded due to Lambda worker still initializing

See original GitHub issue

Expected Behaviour

The expected behavior would be to not receive this warning message

Current Behaviour

  • Receiving an intermittent warning, stating “Subsegment ssm discarded due to Lambda worker still initializing”, followed by another warning stating “No subsegment to end.” These appear to be raised by the powertools Parameters utility.
  • I am doing the parameter retrieval before the lambda handler (global scope) since this will not change very often, regardless of execution environment, as demonstrated below.

Code snippet

from aws_lambda_powertools.utilities import parameters
import boto3

config = Config(
   retries = {
      'max_attempts': 15,
      'mode': 'standard'
   }
)
client=boto3.client('rekognition', config=config)
model_arn=parameters.get_parameter('/RekCL/MODEL_ARN', max_age=300)

def lambda_handler(event, context):
  try:
        response = client.detect_custom_labels(
            ProjectVersionArn=model_arn,
            Image={'S3Object':{'Bucket':bucket,'Name':photo}})

 # etc...

Possible Solution

Please see this: https://github.com/aws/aws-xray-sdk-python/issues/331

I believe there is a check here, but I don’t see anything in the powertools docs that says I need to have x-ray enabled.

Steps to Reproduce

retrieve a parameter in a lambda function, with global scope (before lambda handler)

AWS Lambda Powertools for Python version

latest

AWS Lambda function runtime

3.9

Packaging format used

Lambda Layers

Debugging logs

No response

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:7 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
94Sipcommented, Nov 2, 2022

Hi @94Sip thanka for flagging this - IIRC this is an issue in X-Ray SDK which hasn’t been resolved yet (I’ll use this as a basis for a repro).

In the meantime, you can safely add the get_parameter inside the handler, because only the first call will be made to SSM.

We keep that in-memory and it’ll only fetch another call if Lambda spins up another container (concurrency), which is also true for a new cold start to happen.

Thank you for this quick reply - moving the call to the handler appears to have resolved this warning being thrown.

1reaction
94Sipcommented, Nov 2, 2022

A few more updates:

This lambda is being orchestrated by a Step Function that has x-ray tracing enabled. I have enabled x-ray on the lambda function AND added the appropriate policy, as described here: https://docs.aws.amazon.com/lambda/latest/dg/services-xray.html

Still receiving the intermittent warning.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Subsegment discarded due to Lambda worker still initializing ...
My current workaround is to create an instance of a "MyOneTime" class at the start of every lambda_handler(). That class checks its own ......
Read more >
Ask Question - Stack Overflow
Unable to write to /tmp/.aws-xray/initialized. Failed to signal SDK initialization. Subsegment put_item discarded due to Lambda worker still ...
Read more >
Using AWS Lambda with AWS X-Ray - AWS Documentation
The Initialization subsegment represents the init phase of the Lambda execution environment lifecycle. During this phase, Lambda creates or unfreezes an ...
Read more >
Debugging AWS Lambda Functions with AWS X-Ray
This error occurs because we have built our code to execute in the AWS Lambda environment, and X-Ray creates the root segment for...
Read more >
aws x-ray python example
If you encounter a bug with the AWS X-Ray SDK for Python, we want to hear about it. ... subsegment ssm discarded due...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found