JSON data disappears from graphQL on second and subsequent runs of the development server

Description

I have a website that loads data from several JSON files and uses them to generate pages in in gatsby-node.js using createPage().

It works fine the first time I run npm start. But if I stop and restart the server, around half of the results from one of the JSON files are no longer returned from graphQL, and so many of the pages do not get generated. There’s no visible failure either to tell me anything’s gone wrong.

From my experiments there seems to be an interaction between two of the json files (sets.json and press.json). I suspect it might be to do with the same IDs appearing in both files, as I can solve it by deleting one file, or prefixing all the IDs in one with a string. Also, the one record that doesn’t disappear is the one ID that isn’t also present in the other file.

Steps to reproduce

I’ve made a small reproduction of the problem in this repository: https://github.com/jamiemill/gatsby-json-bug

Steps:

1. npm run clean
2. npm start
3. Observe output, correctly finding and generating 3 press pages:

GENERATING
Press records found: 3
Press: createPage #182
Press: createPage #183
Press: createPage #184
Press pages created
GENERATION FINISHED

1. ctrl-c to terminate dev server
2. npm start
3. Observe output changed to only find 1 press page:

GENERATING
Press records found: 1
Press: createPage #184
Press pages created
GENERATION FINISHED

Expected result

I expect the number of press records to stay stable at 3 on each run of the dev server.

Actual result

On the second and subsequent runs the number of press records drops to 1.

Environment

System: OS: macOS 10.15.7 CPU: (4) x64 Intel® Core™ i5-4258U CPU @ 2.40GHz Shell: 5.7.1 - /bin/zsh Binaries: Node: 12.18.1 - ~/.nvm/versions/node/v12.18.1/bin/node npm: 6.14.5 - ~/.nvm/versions/node/v12.18.1/bin/npm Languages: Python: 2.7.16 - /usr/bin/python Browsers: Chrome: 87.0.4280.88 Firefox: 80.0.1 Safari: 14.0.1 npmPackages: gatsby: ^2.26.1 => 2.29.1 gatsby-source-filesystem: ^2.8.0 => 2.8.0 gatsby-transformer-json: ^2.8.0 => 2.8.0

[LekoArts]

Can I assume there’s no harm in omitting the id field from my JSON now? My mental model was close to the react.js key concept: that it’s somehow helpful for Gatsby if each of my json records has a stable identifier for diffing purposes. But from what you’re saying it doesn’t seem necessary.

Yeah, you can remove the id now.

I’ll close this as answered and the bug is now tracked here: https://github.com/gatsbyjs/gatsby/issues/28846

[KyleAMathews]

This is also another fix too @jamiemill which is to just change your id fields to be something else.