Gatsby Plugin: Asset Manifest (for Server-Side Authentication in Front of Built Assets without client-side routes)

Summary

Gatsby static sites are very fast and optimized. But some content may not be suitable to deliver to all audiences. There are often use cases for including an authentication layer.

Currently, Gatsby promotes using client-only routes for authentication, which negate many of the benefits of the static site generation.

It is possible to set up a Node.js server to achieve server-side authentication (example with Auth0 here: https://github.com/karlhorky/auth0-node-heroku). This example can be extended to serve the Gatsby static assets via express.static or similar - if the user is authenticated, they get the static content back; if not, they receive a 403 Forbidden.

This almost achieves what we want! But it is an all-or-nothing solution - there is no way to restrict access to specific Gatsby assets (for example, based on pages), without multiple crazy, error-prone regexes like this:

const allowedUrlsUserBob = /^(\/|\/(webpack-runtime|app|styles|commons|component---src-pages(-|-courses-1(-|-modules-001-|-modules-002-))index-mdx)-[a-z0-9]+\.js|\/page-data(\/|\/index\/|\/courses\/1\/|\/courses\/1\/modules\/(001|002)\/)(page|app)-data.json|\/courses\/1\/|\/courses\/1\/modules\/(001|002)\/|\/(static|icons)\/.+\.png(\?v=[a-z0-9]+)?)(\?[^/]+)?$/;

Spoiler: I’m using this crazy regular expression option right now 😅

Proposals

I propose offering and documenting one or more tools to support a server-side authentication flow for completely static sites (without using client-only routes), similar to @pieh’s comment on https://github.com/gatsbyjs/gatsby/issues/1100#issuecomment-477978790:

If gatsby would generate asset manifest on builds detailing what assets are used for given urls/pathnames - would that help?

1. An Asset Manifest would be a great start!

This would allow for simpler configuration of user-level and page-level access-control:

import assets from 'manifest.json'

// assets['/courses/1/modules/001'] === [
//   '/webpack-runtime-ef3a9f9842cf40a03163.js',
//   '/commons-cdc988b7a0635a52ddb8.js',
//   '/app-cafc7b4b1730f061489d.js',
//   '/styles-6c9411e5cef0c7a2398a.js',
//   '/component---src-pages-courses-1-modules-001-index-mdx-19fb7194fc6837729ecd.js',
//   '/page-data/courses/prep-l-webfs-gen-0/page-data.json',
//   '/page-data/app-data.json',
// ]

const accessControl = {
  // Key: User ID
  // Value: Array of unique pages they are allowed to view
  2: [...new Set([
    ...assets['/courses/1/modules/001'],
    ...assets['/courses/1/modules/002'],
  ])],
}

Of course, I’m not fixed on the API for the manifest. I’d be open to having helpers to extend this too!

2. Configurable Pre-fetching

One thing that these solutions cause is a lot of failed pre-fetching requests for users without full access:

Maybe there could be a way to configure pre-fetching client-side? So that different resources could be pre-fetched per user?

Basic example

Examples in Proposals section above.

Motivation

Gatsby users will commonly want authentication flows in their apps, and they should also want performant applications, which can be achieved with static site generation.

Alternatives Considered

Existing boilerplates, articles and blog posts, such as those below:

1. https://github.com/auth0-blog/gatsby-auth0

2. From @rwieruch in https://github.com/gatsbyjs/gatsby/issues/1100#issuecomment-351585069:

I implemented a quick MVP this morning to checkout a whole Firebase authentication flow in Gatsby. Turns out it works.

This doesn’t really protect the static content (@sarneeh in https://github.com/gatsbyjs/gatsby/issues/1100#issuecomment-368295841):

@rwieruch Alright, but I think this is not how you should block website content. In your case you just have client-side authentication logic on your page - anyone with a link to your authorised content will still be able to get it (because it’s just a static file somewhere on your host). To make it reliable you still need to authorise the user on the server which serves the content.

Ref (“Authentication support”): https://github.com/gatsbyjs/gatsby/issues/1100

cc @simoneb @pieh @samjulien

[karlhorky]

Ok great! So I just need to import the webpack.stats.json file on the start of the Express server and use the information within it, I guess.

I’ll see if I can make something work in the repo: https://github.com/karlhorky/gatsby-serverside-auth0

---

Edit: Done:

Updated the proof of concept repo:

• https://github.com/karlhorky/gatsby-serverside-auth0/commit/72d41bfde2c6eca27bee9947a1420eecf95070b8
• https://github.com/karlhorky/gatsby-serverside-auth0/commit/da65392a75cf22aa89a9a1dcc7188dfa30c1570a

Here’s the difference:

Old Solution

// Regular expression to match allowed assets related
// to src/pages/index.mdx in the Gatsby website.
//
// Trying to navigate to assets related to src/pages/page-2.mdx
// will return an "Access denied."
const allowedGatsbyWebsiteUrls = /^(\/|\/(webpack-runtime|app|styles|commons|component---src-pages-index-mdx)-[a-z0-9]+\.js|\/page-data\/(index\/)?(page|app)-data.json|\/(static|icons)\/.+\.png(\?v=[a-z0-9]+)?)(\?[^/]+)?$/;

New Solution

// Require the Gatsby asset manifest from the build
// to get paths to all assets that are required by
// each "named chunk group" (each named chunk group
// corresponds to a page).
//
// Ref: https://github.com/gatsbyjs/gatsby/issues/20745#issuecomment-577685950
const {
  namedChunkGroups,
} = require('../gatsby-website/public/webpack.stats.json');

function pageToWebpackFormat(page) {
  // Replace slashes and periods with hyphens
  return page.replace(/(\/|\.)/g, '-');
}

function pageToGatsbyPageDataPath(page) {
  // Strip the /index.mdx at the end of the page
  // If it's the index, just strip the .mdx
  return page.replace(/(\/index)?\.mdx$/, '');
}

function pageToWebPaths(page) {
  // Strip the index.mdx at the end of the page
  let pageWithoutIndex = page.replace(/((\/)?index)?\.mdx$/, '');
  // Add a slash, but only for non-root paths
  if (pageWithoutIndex !== '') pageWithoutIndex += '/';
  return [pageWithoutIndex, pageWithoutIndex + 'index.html'];
}

function getPathsForPages(pages) {
  return (
    pages
      .map(page => {
        return [
          // All asset paths from the webpack manifest
          ...namedChunkGroups[
            `component---src-pages-${pageToWebpackFormat(page)}`
          ].assets,
          // All of the Gatsby page-data.json files
          `page-data/${pageToGatsbyPageDataPath(page)}/page-data.json`,
          ...pageToWebPaths(page),
        ];
      })
      // Flatten out the extra level of array nesting
      .flat()
      .concat(
        // Everything general for the app
        ...namedChunkGroups.app.assets,
        'page-data/app-data.json',
      )

      .filter(
        assetPath =>
          // Root
          assetPath === '' ||
          // Only paths ending with js, json, html and slashes
          assetPath.match(/(\.(html|js|json)|\/)$/),
      )
      // Add a leading slash to make a root-relative path
      // (to match Express' req.url)
      .map(assetPath => '/' + assetPath)
  );
}

const allowedWebpackAssetPaths = getPathsForPages([
  'index.mdx',
]);

function isAllowedPath(path) {
  const pathWithoutQuery = path.replace(/^([^?]+).*$/, '$1');

  // Allow access to the manifest
  if (pathWithoutQuery === '/manifest.webmanifest') return true;

  // Allow access to images within static and icons
  if (pathWithoutQuery.endsWith('png')) {
    if (
      pathWithoutQuery.startsWith('/static/') ||
      pathWithoutQuery.startsWith('/icons/')
    ) {
      return true;
    }
  }

  return allowedWebpackAssetPaths.includes(pathWithoutQuery);
}

[simoneb]

Thanks for taking the time to track this request @karlhorky 👍