"Invalid signature" seen for most calls to Para

Scoold version: 1.43.3 (though observed the same on 1.42.1) Para Version: 1.38.4

Issue observed: Standing up a new Docker-based deployment and running into issues where a user can self-register (email verification is off), but can’t log in. When they do, the UI shows “Authentication was failed or cancelled” and the logs will show something like: 2021-07-14 20:11:04 [ERROR] 403 - Invalid signature for request GET /v1/_id/{userId} coming from app devspark-api-dev

Troubleshooting performed:

Oddly, I can look in the DB and see that the user object is indeed in the DB, so there’s communication happening properly at some level.

I’ve also verified that para.access_key and para.secret_key are set properly. I can use those same values in the Para Admin UI and view users, etc in there.

We have the same images running in another Kubernetes platform without any issues.

I also tried running Scoold locally and pointing at the Para in question, but I get the same behavior.

More details:

Based on #242 and others implying that there might be an issue with headers being dropped or URIs manipulated, I gathered a tcpdump on the Para host and can provide more detail, if needed.

Here’s an example interaction:

Call from Scoold to Para to find a given user (IPs and domain redacted):

19:11:47.877806 IP (tos 0x0, ttl 64, id 28282, offset 0, flags [DF], proto TCP (6), length 471)
    SCOOLD_IP.41736 > PARA_IP.80: Flags [P.], cksum 0x5d40 (incorrect -> 0x1b23), seq 3906450685:3906451116, ack 1058202953, win 272, length 431: HTTP, length: 431
	GET /v1/_id/yet.another.user%40REDACTED.com HTTP/1.1
	Accept: application/json
	Authorization: AWS4-HMAC-SHA256 Credential=app:devspark-api-dev/20210714/us-east-1/para/aws4_request, SignedHeaders=host;x-amz-date, Signature=dcacc597168670930f7137cd9ce4dcc1057ba5fac159f143a09fdc45eb7debd2
	X-Amz-Date: 20210714T191147Z
	User-Agent: Para client 1.39.1 app:devspark-api-dev (Java 11.0.11+9)
	Host: para-dev
	Connection: keep-alive

Response from Para:

19:11:47.878462 IP (tos 0x0, ttl 62, id 42571, offset 0, flags [DF], proto TCP (6), length 726)
    PARA_IP.80 > SCOOLD_IP.41736: Flags [P.], cksum 0xe08a (correct), seq 1058202953:1058203639, ack 3906451116, win 269, length 686: HTTP, length: 686
	HTTP/1.1 403 Forbidden
	Date: Wed, 14 Jul 2021 19:11:47 GMT
	Set-Cookie: para-csrf-token-anonid=VeUwPjbamPN/WWIOMNOE3g==; Path=/; Expires=Thu, 15-Jul-2021 19:11:47 GMT; Max-Age=86400
	Expires: Thu, 01 Jan 1970 00:00:00 GMT
	Set-Cookie: para-csrf-token=TeCvJLpta4No9vK6z71Y9Bhkxq3LnVquhqb+RODNSCY=; Path=/; Expires=Thu, 15-Jul-2021 19:11:47 GMT; Max-Age=86400
	WWW-Authenticate: Bearer
	Content-Type: application/json;charset=utf-8
	X-Content-Type-Options: nosniff
	X-XSS-Protection: 1; mode=block
	X-Frame-Options: DENY
	Content-Length: 142

	{
	  "code" : 403,
	  "message" : "Invalid signature for request GET /v1/_id/yet.another.user%40REDACTED.com coming from app devspark-api-dev"
	}

Let me know if there’s any other details I can provide.

[cjchand]

OK, we finally found the root cause. Thanks for feeding us the info to get us there, @albogdano!

For posterity, here’s what we found: The reason the headers, etc are so important is that they are a key part of how the requests inbound to Para are signed. @albogdano can keep me honest, but I believe this is the key param in question: https://github.com/Erudika/para/blob/8e050f4b11aabb47d77d2ed3c1ceeb0cd4a351bf/para-core/src/main/java/com/erudika/para/rest/Signer.java#L191

Anyway, here’s the rundown of our config:

Scoold was pointed to the K8s service for Para (para-dev:80), which - from a K8s standpoint - was correct. We could communicate between the two.

But here’s the key: Para’s para.app_name was indeed para-dev, but it was listening on the default port of 8080. That means when Para went to validate the signature, it failed b/c Scoold signed it using http://para-dev:80 for the param linked above.

We changed the K8s service to listen on 8080 and updated Scoold config to para.endpoint = "http://para-dev:8080". That did the trick.

tl;dr: Ensure that para.endpoint in the Scoold config exactly matches Para’s para.app_name, para.port (defaults to 8080 if not set), and protocol (http:// or https://).

Thanks again, @albogdano!

[albogdano]

Good job! I’m still confused as to why the other POST requests didn’t get rejected because they are still using the same host and Host header. But yeah, the signatures are very susceptible to changes in URLs, query parameters and headers. Sometimes that’s more of a hassle and a source of frustration for end users, which is why I’m considering switching to a simpler authentication mechanism for API requests (in a future v2 of Para). And just to be clear, para.app_name is used for changing the name of your application from “Scoold” to, say, “Your app” just for branding purposes. So it’s not essential that it matches the value of para.endpoint.