question-mark

Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Help with Authentication when proxying through CDN

See original GitHub issue

Issue Description

Overview

I recently wrote an article on using Airtable (Airtable.js + lambda) as a content backend. https://medium.com/@gtalarico/using-airtable-as-a-content-backend-e373cd0d9974

A few people mentioned the 5 req/sec rate limit could be an issue, so I am trying to add additional details about how one could add a CDN (eg. Cloudfront) to cache the requests and avoid the rate limit.

I created airtable as the origin on cloudfront, and set it up to pass through Authorization headers. I am able to successful use the CDN to proxy/cache the calls

Request works as expected through curl:

$ curl -L "https://d2j5hchu5g3uxq.cloudfront.net/v0/appNtnZ99fkL1cByn/entries?maxRecords=100&view=all" \
    -H "authorization: Bearer keyOmitted"

STATUS: 200

{"records":[{"id":"recW0FpRZMaQv5Ble","fields":{"title":"3D 
[... truncated ]

Requests fail when using airtable.js

I then make the exact same request When I make the same call through airtable.js, but I get AUTHENTICATION_REQUIRED.

I logged the options variable right before airtable.js runs runAction and the payload looks good and matches my curl, so I really can’t understand why it would fail:

// run_action.js
# line 52
console.log(JSON.stringify(options))
request(options, function(error, resp, body) {
[ ... ]

Outputs:

// `options`
{
   "method":"GET",
   "url":"https://d2j5hchu5g3uxq.cloudfront.net/v0/appNtnZ99fkL1cByn/entries?maxRecords=100&view=all",
   "json":true,
   "timeout":300000,
   "headers":{
      "authorization":"Bearer keyOMITTED",
      "x-api-version":"0.1.0",
      "x-airtable-application-id":"appNtnZ99fkL1cByn",
      "User-Agent":"Airtable.js/0.1.0"
   },
   "agentOptions":{
      "rejectUnauthorized":false
   }
}

Response with status 500 in 686 ms.
Error during invocation:  s {
  error: 'AUTHENTICATION_REQUIRED',
  message: 'You should provide valid api key to perform this operation',
  statusCode: 401
}

I apologize as this may not be an issue with the lib but I am out of ideas here. Let me know if you have any insights, otherwise feel free to close it.

Here is section I am adding to the article - I would be happy to add a wiki page or doc so other users can use the same approach to cache the rate limit:

Article Update:

image image

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:10 (8 by maintainers)

github_iconTop GitHub Comments

3reactions
gtalaricocommented, Jul 4, 2019

After some more tests and tweaking CDN settings I was able to get it to work. I am not 100% what the change was that fixed it, but I will post the setting here as a reference.

Thank you @EvanHahn and @syrnick for you help troubleshooting this.

Distribution Config

image

Behaviour Config

image

Origin Config

image

Netlify Lambda Code

const Airtable = require('airtable')

Airtable.configure({
  // AIRTABLE_API_URL="https://d2j5hchu5g3uxq.cloudfront.net"
  endpointUrl: process.env.AIRTABLE_API_URL,
  apiKey: process.env.AIRTABLE_API_KEY
})
const base = Airtable.base('appNtnZ99fkL1cByn')

exports.handler = function(event, context, callback) {
  const allRecords = []
  base('entries')
    .select({
      maxRecords: 100,
      view: 'all'
    })
    .eachPage(
      function page(records, fetchNextPage) {
        records.forEach(function(record) {
          allRecords.push(record)
        })
        fetchNextPage()
      },
      function done(err) {
        if (err) {
          callback(err)
        } else {
          const body = JSON.stringify({ records: allRecords })
          const response = {
            statusCode: 200,
            body: body,
            headers: {
              'content-type': 'application/json',
              'cache-control': 'Cache-Control: max-age=300, public'
            }
          }
          callback(null, response)
        }
      }
    )
}

1reaction
ricricucitcommented, May 23, 2020

ah! magic for me, as well. It started working after some time, and after invalidating all objects.

Not sure, but I think it was only cloudfront needing some time to go up, properly. 🤔

Read more comments on GitHub >

github_iconTop Results From Across the Web

Deploying a Cloud CDN origin authentication proxy
The authentication proxy performs AWS Signature Version 4 signing, using credentials stored in Secret Manager. The authentication proxy then forwards signed ...
Read more >
Proxy-Authenticate | Fastly Developer Hub
The Proxy-Authenticate header field consists of at least one challenge that indicates the authentication scheme(s) and parameters applicable to the proxy for ...
Read more >
What is a reverse proxy? | Proxy servers explained | Cloudflare
A reverse proxy protects web servers from attacks and can provide performance and reliability benefits. Learn more about forward and reverse proxies.
Read more >
Configure Azure CDN as Reverse Proxy - Auth0
To set up Azure CDN as a reverse proxy, an Azure CDN Premium plan is required. Configure Custom Domains with Self-Managed Certificates if...
Read more >
Using Authentication and Proxies - TechDocs - Broadcom Inc.
The appliance performs authentication to obtain proof of user identity and then make decisions based on the identity.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found