Requests for page-data.json return 404 in client side routes

Description

Hey team! I’ve noticed an issue with client side routes recently.

When navigating to client side routes, Gatsby requests the associated page-data.json which returns a 404. For example, navigating to /app/profile loads /page-data/app/profile/page-data.json, which returns a 404 if the /app page has a matchPath of /app/*. This is reproducible in the simple-auth example which has a gatsby-node.js like this:

exports.onCreatePage = async ({ page, actions }) => {
  const { createPage } = actions

  // page.matchPath is a special key that's used for matching pages
  // only on the client.
  if (page.path.match(/^\/app/)) {
    page.matchPath = `/app/*`

    // Update the page.
    createPage(page)
  }
}

Note this doesn’t seem to affect the rendering of the page. It looks like it’s still getting the correct page data for the matchPath page even though Gatsby is making a request for different page data.

Steps to reproduce

You can reproduce this in the the simple-auth example.

1. Clone the simple-auth example
2. Install the latest gatsby
3. gatsby build and gatsby serve
4. Navigate to localhost:9000/app/login

You get some 404s for page-data

Expected result

Gatsby shouldn’t request page-data.json for client side routes.

Actual result

Gatsby requests page-data.json for client side routes and receives a 404.

Environment

System: OS: macOS 10.14.5 CPU: (8) x64 Intel® Core™ i7-7820HQ CPU @ 2.90GHz Shell: 3.0.2 - /usr/local/bin/fish Binaries: Node: 12.3.1 - ~/.asdf/shims/node Yarn: 1.16.0 - ~/.asdf/shims/yarn npm: 6.9.0 - ~/.asdf/shims/npm Languages: Python: 2.7.15 - /Users/thanx/.asdf/shims/python Browsers: Chrome: 75.0.3770.142 Safari: 12.1.1 npmPackages: gatsby: ^2.13.41 => 2.13.41 gatsby-plugin-react-helmet: ^3.0.0 => 3.0.12

[wardpeet]

Sorry for the late notice, I have been on vacation for the past week and a half. I can bring you the good news that we will be fixing this natively in gatsby. We’ll put urls inside match-path.json that match a dynamic route.

I’ll be creating a PR to fix this issue. I’ll let you know when it’s up and running to get some feedback.

[eknowles]

@kirbydcool I think this issue should be reopened, the 404 is an issue. For applications with large amounts of links per page (https://github.com/gatsbyjs/gatsby/issues/16097#issuecomment-515488361) it can cause 404s to go through the roof, if we know that the route will result in a 404 the application should not be making it in the first place, instead it should route to a 200.

My temp solution to this issue is to add 301 redirects for page-data.json

page-data/vehicle/([0-9]+)/page-data.json
--->
%{scheme}://%{host}/page-data/vehicle/*/page-data.json

This does result in the correct page-data for the dynamic route to be displayed. However a 301 still results in a network request for a new document each time.

e.g. page-data/vehicle/123456789/page-data.json

Gatsby should identify this route as being dynamic and the loader should fetch the */page-data.json.

The result would be:-

1. Browsers would intelligently cache the page-data.json.
2. Bursty requests would not fire for pages with n Link items.
3. CDN logs would not be polluted with 100,000s of 404s

Is there someone on the gatsby core team who can prioritise this issue?