question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

SSM command times out before given executionTimeout seconds have elapsed

See original GitHub issue

When I run the following code to run an SSM command on another instance, the command stops after 1 hour rather than 2505600 seconds = 29 days (although no InvalidParameter errors are thrown when I start the command):

ssm_client.send_command(
  InstanceIds=[instance_id],
  DocumentName="RunShellScript-LongRunning",
  Parameters={
      "commands": [command],
      "executionTimeout": ["2505600"],
  }
)

where RunShellScript-LongRunning is the name of my created SSM document (of the same form as AWS-RunShellScript except with anything up to 2505600 allowed for executionTimeout instead of 172800 being the max; "allowedPattern": "([1-9][0-9]{0,5})|(1[0-9]{6})|(2[0-4][0-9]{5})|(250[0-4][0-9]{3})|(2505[0-5][0-9]{2})|(2505600)"):

{
  "schemaVersion": "1.2",
  "description": "Run a shell script or specify the commands to run.",
  "parameters": {
    "commands": {
      "type": "StringList",
      "description": "(Required) Specify a shell script or a command to run.",
      "minItems": 1,
      "displayType": "textarea"
    },
    "workingDirectory": {
      "type": "String",
      "default": "",
      "description": "(Optional) The path to the working directory on your instance.",
      "maxChars": 4096
    },
    "executionTimeout": {
      "type": "String",
      "default": "3600",
      "description": "(Optional) The time in seconds for a command to complete before it is considered to have failed. Default is 3600 (1 hour). Maximum is 2505600 (29 days).",
      "allowedPattern": "([1-9][0-9]{0,5})|(1[0-9]{6})|(2[0-4][0-9]{5})|(250[0-4][0-9]{3})|(2505[0-5][0-9]{2})|(2505600)"
    }
  },
  "runtimeConfig": {
    "aws:runShellScript": {
      "properties": [
        {
          "id": "0.aws:runShellScript",
          "runCommand": "{{ commands }}",
          "workingDirectory": "{{ workingDirectory }}",
          "timeoutSeconds": "{{ executionTimeout }}"
        }
      ]
    }
  }
}

When I pass an executionTimeout parameter of “172800” to send_command, it does not stop after 1 hour and continues running normally. When I use the same document but pass an executionTimeout parameter of “172801” to send_command, it does stop after 1 hour. It seems that there is some bug in the SSM that implicitly rejects an executionTimeout of above 172800 seconds = 48 hours by instead timing out execution after the default of 1 hour, even if it is not rejected explicitly by the allowedPattern in the SSM document (which would throw an InvalidParameter error).

Do you have any idea about how to fix this?

In this original guidance thread, I was advised to use SSM documents to increase the max executionTimeout of commands initiated by send_command, but this seems to be an actual bug.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:16 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
kdailycommented, Apr 1, 2021

Hello @markjayne, I’m looking into this.

0reactions
github-actions[bot]commented, Nov 27, 2022

Greetings! It looks like this issue hasn’t been active in longer than five days. We encourage you to check if this is still an issue in the latest release. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or upvote with a reaction on the initial post to prevent automatic closure. If the issue is already closed, please feel free to open a new one.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Understanding command statuses - AWS Systems Manager
When SSM Agent determines that a command has timed out, it sends executionTimeout to the service. The default for Timeout (seconds) is 3600...
Read more >
SSM Automation Run Command is not longer than default ...
I have changed the execution timeout here in the parameters but does not work. enter image description here. also, I added a timeoutSeconds...
Read more >
Understanding command statuses - Amazon Systems Manager
When SSM Agent determines that a command has timed out, it sends executionTimeout to the service. The default for Timeout (seconds) is 3600...
Read more >
Troubleshoot query time-out errors - SQL Server
The timeout period elapsed while attempting to consume the pre-login handshake acknowledgment. This could be because the pre-login handshake ...
Read more >
SSM Automation Run Command longer than default 3600 ...
I had issues with AWS Data Pipeline RunShellScript send commands timing out after an hour (default 3600 seconds) when I upgraded the SSM...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found