Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

"Invalid signature" seen for most calls to Para

See original GitHub issue

Scoold version: 1.43.3 (though observed the same on 1.42.1) Para Version: 1.38.4

Issue observed: Standing up a new Docker-based deployment and running into issues where a user can self-register (email verification is off), but can’t log in. When they do, the UI shows “Authentication was failed or cancelled” and the logs will show something like: 2021-07-14 20:11:04 [ERROR] 403 - Invalid signature for request GET /v1/_id/{userId} coming from app devspark-api-dev

Troubleshooting performed:

Oddly, I can look in the DB and see that the user object is indeed in the DB, so there’s communication happening properly at some level.

I’ve also verified that para.access_key and para.secret_key are set properly. I can use those same values in the Para Admin UI and view users, etc in there.

We have the same images running in another Kubernetes platform without any issues.

I also tried running Scoold locally and pointing at the Para in question, but I get the same behavior.

More details:

Based on #242 and others implying that there might be an issue with headers being dropped or URIs manipulated, I gathered a tcpdump on the Para host and can provide more detail, if needed.

Here’s an example interaction:

Call from Scoold to Para to find a given user (IPs and domain redacted):

19:11:47.877806 IP (tos 0x0, ttl 64, id 28282, offset 0, flags [DF], proto TCP (6), length 471)
    SCOOLD_IP.41736 > PARA_IP.80: Flags [P.], cksum 0x5d40 (incorrect -> 0x1b23), seq 3906450685:3906451116, ack 1058202953, win 272, length 431: HTTP, length: 431
	GET /v1/_id/ HTTP/1.1
	Accept: application/json
	Authorization: AWS4-HMAC-SHA256 Credential=app:devspark-api-dev/20210714/us-east-1/para/aws4_request, SignedHeaders=host;x-amz-date, Signature=dcacc597168670930f7137cd9ce4dcc1057ba5fac159f143a09fdc45eb7debd2
	X-Amz-Date: 20210714T191147Z
	User-Agent: Para client 1.39.1 app:devspark-api-dev (Java 11.0.11+9)
	Host: para-dev
	Connection: keep-alive

Response from Para:

19:11:47.878462 IP (tos 0x0, ttl 62, id 42571, offset 0, flags [DF], proto TCP (6), length 726)
    PARA_IP.80 > SCOOLD_IP.41736: Flags [P.], cksum 0xe08a (correct), seq 1058202953:1058203639, ack 3906451116, win 269, length 686: HTTP, length: 686
	HTTP/1.1 403 Forbidden
	Date: Wed, 14 Jul 2021 19:11:47 GMT
	Set-Cookie: para-csrf-token-anonid=VeUwPjbamPN/WWIOMNOE3g==; Path=/; Expires=Thu, 15-Jul-2021 19:11:47 GMT; Max-Age=86400
	Expires: Thu, 01 Jan 1970 00:00:00 GMT
	Set-Cookie: para-csrf-token=TeCvJLpta4No9vK6z71Y9Bhkxq3LnVquhqb+RODNSCY=; Path=/; Expires=Thu, 15-Jul-2021 19:11:47 GMT; Max-Age=86400
	WWW-Authenticate: Bearer
	Content-Type: application/json;charset=utf-8
	X-Content-Type-Options: nosniff
	X-XSS-Protection: 1; mode=block
	X-Frame-Options: DENY
	Content-Length: 142

	  "code" : 403,
	  "message" : "Invalid signature for request GET /v1/_id/ coming from app devspark-api-dev"

Let me know if there’s any other details I can provide.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:9 (5 by maintainers)

github_iconTop GitHub Comments

cjchandcommented, Jul 16, 2021

OK, we finally found the root cause. Thanks for feeding us the info to get us there, @albogdano!

For posterity, here’s what we found: The reason the headers, etc are so important is that they are a key part of how the requests inbound to Para are signed. @albogdano can keep me honest, but I believe this is the key param in question:

Anyway, here’s the rundown of our config:

Scoold was pointed to the K8s service for Para (para-dev:80), which - from a K8s standpoint - was correct. We could communicate between the two.

But here’s the key: Para’s para.app_name was indeed para-dev, but it was listening on the default port of 8080. That means when Para went to validate the signature, it failed b/c Scoold signed it using http://para-dev:80 for the param linked above.

We changed the K8s service to listen on 8080 and updated Scoold config to para.endpoint = "http://para-dev:8080". That did the trick.

tl;dr: Ensure that para.endpoint in the Scoold config exactly matches Para’s para.app_name, para.port (defaults to 8080 if not set), and protocol (http:// or https://).

Thanks again, @albogdano!

albogdanocommented, Jul 16, 2021

Good job! I’m still confused as to why the other POST requests didn’t get rejected because they are still using the same host and Host header. But yeah, the signatures are very susceptible to changes in URLs, query parameters and headers. Sometimes that’s more of a hassle and a source of frustration for end users, which is why I’m considering switching to a simpler authentication mechanism for API requests (in a future v2 of Para). And just to be clear, para.app_name is used for changing the name of your application from “Scoold” to, say, “Your app” just for branding purposes. So it’s not essential that it matches the value of para.endpoint.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Auth/v2/auth giving a 40103 but auth/v2/preauth working fine
EXPLANATION: The signature is formatted incorrectly. RESOLUTION: Verify that the signature is encoded in hexadecimal ASCII; is using the correct ...
Read more >
Reverse proxy causes "Invalid signature for server ..." resulting ...
I am unable to invite people on some servers into a chat. In Riot I see no difference (but no one ever accepts...
Read more >
Need To Resolve Yelp API Response Error: “Signature is ...
Error in your API call url,yelp using oauth 1.0a,Which one url parameters should be in alphabet order(this one is telling clearly in your...
Read more >
Security: Invalid Signature | Revit 2022
When you try to load an executable file or add-in, a dialog indicates that there may be a security issue. Issue: Executable files...
Read more >
Check the webhook signatures | Stripe Documentation
To protect against timing attacks, use a constant-time string comparison to compare the expected signature to each of the received signatures. See also....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found