Add an example how to add robots.txt or sitemap.xml

Edit by Maintainers

Root level files are supported in Next.js 9.1 and newer.

Create the robots.txt file in the public/ directory.

For example:

pages/index.js
public/robots.txt

Note the public folder is on the same level as the pages folder, not inside of it!

Read more in the docs: https://nextjs.org/docs/basic-features/static-file-serving

---

There’s a closed issue related to this without a clear answer: https://github.com/zeit/next.js/issues/226

It’s kind of closed in favor of the Programmatic API so I guess I should implement that with custom server?)

[spleshka]

Another way of doing this: put robots.txt into /static folder, then add this to your server config:

const options = {
  root: __dirname + '/static/',
  headers: {
    'Content-Type': 'text/plain;charset=UTF-8',
  }
};
server.get('/robots.txt', (req, res) => (
  res.status(200).sendFile('robots.txt', options)
));

[Timer]

Next.js now has a public/ folder that mounts files at the root automatically! Create public/robots.txt and you’re done!