How to Ignore GET requests to /robots.txt file?
See original GitHub issueI have a Node.js project. This project use local-auth with passport.js
, express-session
and connect-mongo
package to store sessions in MongoDB sessions
collection.
Each time when I update page or moving to another page, to sessions
collection inserted 6 new items. Thus, across some time sessions
collection becomes very large.
Code snippet for use express-session
and connect-mongo
:
var session = require('express-session')
var MongoStore = require('connect-mongo')(session)
app.use(session({
resave: true,
saveUninitialized: true,
secret: secret.secretKey,
store: new MongoStore({
url: secret.database,
autoReconnect: true
})
}))
Morgan output in console (this happens for each page and page update):
C:\Users\User\Desktop\Project>node server.js
Node.js listening on port 3000
GET / 200 37.946 ms - 2670
GET /robots.txt 400 23.490 ms - 2328
GET /robots.txt 400 185.459 ms - 2328
GET /robots.txt 400 4.794 ms - 2328
GET /robots.txt 400 6.684 ms - 2328
GET /robots.txt 400 3.966 ms - 2328
GET /favicon.ico 400 7.314 ms - 2328
GET /robots.txt 400 3.713 ms - 2328
So, I think this happens by reason of many requests to non-existent robots.txt
URL.
How can I make this file ignored in express-session
?
Also I found this; http://codingteam.net/project/tapage/browse/node_modules/express/node_modules/connect/lib/middleware/session.js (lines 84 - 91):
* Ignore Paths:
*
* By default `/favicon.ico` is the only ignored path, all others
* will utilize sessions, to manipulate the paths ignored, use
* `connect.session.ignore.push('/my/path')`. This works for _full_
* pathnames only, not segments nor substrings.
*
* connect.session.ignore.push('/robots.txt');
But that’s does not work in new versions of express
and express-session
.
Issue Analytics
- State:
- Created 6 years ago
- Comments:5 (4 by maintainers)
Top GitHub Comments
Looking how that link implements ignore, it’s identical to https://www.npmjs.com/package/ignore-paths so if that’s ehat you need, wrap this middleware with that module to ignore the paths you need.
Idk, that was never in this project.