question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Clock Drift/Skew code does not handle all skew error scenarios

See original GitHub issue

Describe the bug See my comments https://github.com/aws-amplify/amplify-js/issues/2014#issuecomment-668914147 onwards. DateUtils’ isClockSkewError only checks for one kind of error type related to bad clock, but there’s actually many possible error types. See https://github.com/aws/aws-sdk-js-v3/blob/47770fa493c3405f193069cd18319882529ff484/packages/service-error-classification/src/constants.ts#L8

To Reproduce Steps to reproduce the behavior:

  1. Setup API Gateway via amplify cli
  2. Set your computer time 10 minutes early or more
  3. Have your code sign into cognito, then make a REST API call to API Gateway
  4. You’ll get error back with ‘x-amzn-errortype’ of “InvalidSignatureException” and now retry will occur with proper clock offset.

Expected behavior Retry would occur with correct clock offset. For some reason your code expects “BadRequestException” error to be returned but in my case “InvalidSignatureException” is returned.

Screenshots See https://github.com/aws-amplify/amplify-js/issues/2014#issuecomment-668914147 for example error. In this case error came from cloudfront in an edge-optimized API Gateway, but same error comes back from regional API Gateway too.

89360718-8bf0d780-d686-11ea-8ca0-01ef564f5aae

What is Configured? If applicable, please provide what is configured for Amplify CLI:

  • Which resources do you have configured?
    • If applicable, please provide your aws-exports file:
     "aws_project_region": "us-east-1",
     "aws_cloud_logic_custom": [
         {
             "name": "EssRestAPI",
             "endpoint": "https://....execute-api.us-east-1.amazonaws.com/...",
             "region": "us-east-1"
         }
     ],
     "aws_appsync_graphqlEndpoint": "https://....appsync-api.us-east-1.amazonaws.com/graphql",
     "aws_appsync_region": "us-east-1",
     "aws_appsync_authenticationType": "AMAZON_COGNITO_USER_POOLS",
     "aws_cognito_identity_pool_id": "us-east-1:8...",
     "aws_cognito_region": "us-east-1",
     "aws_user_pools_id": "us-east-1_g0...",
     "aws_user_pools_web_client_id": "1e1rhg332...",
     "oauth": {}
    

};```

  • If applicable, please provide your manual configuration example:
Auth.configure({
authenticationFlowType: 'USER_PASSWORD_AUTH'
});```




System: OS: macOS 10.15.6 CPU: (12) x64 Intel® Core™ i7-8850H CPU @ 2.60GHz Memory: 104.32 MB / 32.00 GB Shell: 5.7.1 - /bin/zsh Binaries: Node: 13.14.0 - ~/.nvm/versions/node/v13.14.0/bin/node Yarn: 1.22.0 - /usr/local/bin/yarn npm: 6.14.4 - ~/.nvm/versions/node/v13.14.0/bin/npm Watchman: 4.9.0 - /usr/local/bin/watchman Browsers: Chrome: 84.0.4147.105 Firefox: 78.0.2 Safari: 13.1.2 npmPackages: @angular-builders/custom-webpack: ^10.0.0 => 10.0.0 @angular-devkit/build-angular: ^0.1000.4 => 0.1000.4 @angular/animations: ~10.0.5 => 10.0.5 @angular/cdk: ^10.1.0 => 10.1.0 @angular/cli: ~10.0.4 => 10.0.4 @angular/common: ~10.0.5 => 10.0.5 @angular/compiler: ~10.0.5 => 10.0.5 @angular/compiler-cli: ^10.0.5 => 10.0.5 @angular/core: ~10.0.5 => 10.0.5 @angular/elements: ~10.0.5 => 10.0.5 @angular/forms: ~10.0.5 => 10.0.5 @angular/material: ^10.1.0 => 10.1.0 @angular/platform-browser: ~10.0.5 => 10.0.5 @angular/platform-browser-dynamic: ~10.0.5 => 10.0.5 @angular/router: ~10.0.5 => 10.0.5 @aws-amplify/api: ^3.1.20 => 3.1.20 @aws-amplify/api-rest: ^1.1.3 => 1.1.3 @aws-amplify/auth: ^3.3.2 => 3.3.2 @ng-select/ng-option-highlight: 0.0.5 => 0.0.5 @ng-select/ng-select: ^4.0.0 => 4.0.4 @ngneat/until-destroy: ^7.1.6 => 7.3.2 @stripe/stripe-js: ^1.8.0 => 1.8.0 @types/jasmine: ~3.5.0 => 3.5.11 @types/jasminewd2: ~2.0.3 => 2.0.8 @types/node: 14.0.4 => 14.0.4 @types/node-zendesk: ^1.4.0 => 1.4.0 array-flat-polyfill: ^1.0.1 => 1.0.1 axios: ^0.19.2 => 0.19.2 body-parser: ^1.19.0 => 1.19.0 codelyzer: ^6.0.0 => 6.0.0 document-register-element: ^1.7.2 => 1.14.3 folder-hash: ^3.3.0 => 3.3.2 fs-extra: ^9.0.0 => 9.0.1 jasmine-core: ~3.5.0 => 3.5.0 jasmine-spec-reporter: ~5.0.0 => 5.0.2 karma: ~5.0.0 => 5.0.9 karma-chrome-launcher: ~3.1.0 => 3.1.0 karma-coverage-istanbul-reporter: ~3.0.2 => 3.0.3 karma-jasmine: ~3.3.0 => 3.3.1 karma-jasmine-html-reporter: ^1.5.0 => 1.5.4 libphonenumber-js: ^1.7.51 => 1.7.54 lodash-es: ^4.17.15 => 4.17.15 lodash.merge: ^4.6.2 => 4.6.2 lodash.snakecase: ^4.1.1 => 4.1.1 ng2-pdf-viewer: ^6.3.2 => 6.3.2 ngx-build-plus: ^10.1.1 => 10.1.1 ngx-stripe: ^10.1.1 => 10.1.1 node-sass: ^4.13.1 => 4.14.1 object.fromentries: ^2.0.2 => 2.0.2 postcode-validator: ^3.1.0 => 3.1.0 prettier: ^1.19.1 => 1.19.1 prettier-tslint: ^0.4.2 => 0.4.2 prompt-sync: ^4.2.0 => 4.2.0 protractor: ~7.0.0 => 7.0.0 rxjs: ~6.6.0 => 6.6.0 stripe: ^8.25.0 => 8.72.0 tmp: ^0.1.0 => 0.1.0 ts-node: ~8.3.0 => 8.3.0 tslib: ^1.13.0 => 1.13.0 tslint: ~6.1.0 => 6.1.2 typescript: ~3.9.6 => 3.9.6 webpack-bundle-analyzer: ^3.8.0 => 3.8.0 which: ^2.0.2 => 2.0.2 zone.js: ~0.10.3 => 0.10.3 npmGlobalPackages: @angular/cli: 10.0.3 @aws-amplify/cli: 4.26.0 local-web-server: 4.2.1 node-gyp: 7.0.0 npm: 6.14.4```

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:2
  • Comments:28 (10 by maintainers)

github_iconTop GitHub Comments

15reactions
jamesaucodecommented, Nov 30, 2021

Hi all, I spent a good chunk of today trying to debug this issue. So, It looks like this is not a library issue but is instead an API Gateway / CLI issue.

The way we correct any clock skew error, as pointed out by @hisham is as follow

  1. Make a request
  2. Gets an error response from the server
  3. Check the type of error to see if it’s related to a clock skew
  4. If yes, check the Date header from the error response and call DateUtils.setClockOffset by calculating the difference between the request time & the server response time

see: https://github.com/aws-amplify/amplify-js/blob/8448b4fe1f995f0d6056bcb46dd0fb2023fb197d/packages/api-rest/src/RestClient.ts#L194-L225

This part actually works as intended, the problem is, if we look at the screenshot provided, the CORS headers are missing (which is also the problem I encountered when I repro-ed the problem). Javascript won’t be able to access the error response, therefore the library couldn’t identify it as a clock skew error. Therefore the solution is to actually make API gateway respond with the CORS header for 4XX errors.

You can follow the walkthrough below for a temporary workaround:

  1. Go to your AWS console -> look for the API gateway page -> select the API you want to configure Screen Shot 2021-11-29 at 5 39 58 PM
  2. Go to ‘Gateway responses’ Screen Shot 2021-11-29 at 7 14 50 PM
  3. Click ‘Invalid Signature’ (This is the error I’m getting for 403, if yours is different then select the error type you are getting instead of this) Configure the response headers to allow CORS, and expose the Date and x-amzn-errortype headers Here’s an example, (change ‘my-domain.com’ to a value that fits your use case) Screen Shot 2021-11-29 at 6 20 05 PM
  4. Deploy the API
  5. Now your API should work despite a clock time mismatch!

Demo:

https://user-images.githubusercontent.com/40404256/143974566-429a7c05-d989-485e-8ad1-8a2e41e1981f.mov You can see the first request fails, then it was able to automatically retry and succeed on the second call.

I will look to get a PR on the CLI side to add this setting by default.

4reactions
jamesaucodecommented, Dec 1, 2021

https://github.com/aws-amplify/amplify-cli/pull/9158 I have submitted a PR on the CLI side to add this behavior as a default 4xx / 5xx gateway response

Read more comments on GitHub >

github_iconTop Results From Across the Web

System Clock Skewed? Read this Post, Especially if You Don't ...
First, NTP is a beautiful thing. Make sure you have a consistent NTP implementation across all your servers. Second, periodically verify the ...
Read more >
Clock errors in land and ocean bottom seismograms
In land stations, nonlinear clock drifts and clock jumps of up to 6 min are detected and successfully corrected. For 52 out of...
Read more >
Clock skew - Wikipedia
Clock skew (sometimes called timing skew) is a phenomenon in synchronous digital circuit systems (such as computer systems) in which the same sourced...
Read more >
TensorFlow Data Validation | TFX
One of the key causes for distribution skew is using different code or different data sources to generate the training dataset. Another reason...
Read more >
Unanswered 'skew' Questions - Stack Overflow
I am trying to write R code to generate a nonnormal distribution (i.e., a normal distribution with specific skewness and kurtosis). I would...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found