No longer resolving file paths correctly - Error: ENOENT: no such file or directory, scandir

Environment

Knex version: 0.20.9 Database + version: Postgres 10.6 OS: debian

Bug

1. Since the latest version of Knex (0.20.9), once knex has switched to the working directory. Knex doesn’t resolve the full path correctly. From the logs, I can see that Knex is trying to resolve the files from, for example, /app/migrations when in fact our folder structure is:

/app/config/knexfile.js
/app/config/migrations/*.js
/app/config/seed/*.js

event-service_1    | DB_MIGRATE set to 'true' (default is true)
event-service_1    | Running db migrations...
event-service_1    | 
event-service_1    | > event-service@1.0.0 db:migrate /app
event-service_1    | > knex --knexfile ./config/knexfile.js migrate:latest
event-service_1    | 
event-service_1    | Working directory changed to /app/config
event-service_1    | Error: ENOENT: no such file or directory, scandir '/app/migrations'
event-service_1    | npm ERR! code ELIFECYCLE
event-service_1    | npm ERR! errno 1
event-service_1    | npm ERR! event-service@1.0.0 db:migrate: `knex --knexfile ./config/knexfile.js migrate:latest`
event-service_1    | npm ERR! Exit status 1
event-service_1    | npm ERR! 
event-service_1    | npm ERR! Failed at the event-service@1.0.0 db:migrate script.
event-service_1    | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
event-service_1    | 
event-service_1    | npm ERR! A complete log of this run can be found in:
event-service_1    | npm ERR!     /home/node/.npm/_logs/2020-02-11T19_29_33_262Z-debug.log
event-service_1    | Completed db migrations
event-service_1    | DB_SEED set to 'true' (default is false)
event-service_1    | Running db seeding...
event-service_1    | 
event-service_1    | > event-service@1.0.0 db:seed /app
event-service_1    | > knex --knexfile ./config/knexfile.js seed:run
event-service_1    | 
event-service_1    | Working directory changed to /app/config
event-service_1    | Error: ENOENT: no such file or directory, scandir '/app/seeds'

Looks to be related to changes here: https://github.com/knex/knex/commit/947273e24a73545a269fbe90ef30d55889d5db72

Thanks for the help! Richard

[rlee89]

@rlee89 : Okay, I think I see the issue now. The sequence of operations within the Knex CLI is:

1. Open the knexfile.js
2. Change the process’s current working directory
3. Attempt to run the migrations / etc.

Your knexfile.js is resolving some paths as soon as it’s opened. The problem: they are being resolved before Knex has changed the process’s current working directory. This is causing the paths to be calculated relative to /app instead of /app/config.

So, I’ll poke around a bit and see if I can fix the order of operations on the Knex side. As for a short-term solution: try removing the calls to path.resolve(..) from your knexfile.js. This will cause the paths to remain relative until the moment that Knex attempts to run the migrations / etc . By then, the process’s current working directory will already be set to the correct location.

Yup, we look good! Removed the path.resolve(...) and that worked. We’ll look at seeing if can’t suggest cleaning that up for all our services too… 😄

event-service_1    | DB_MIGRATE set to 'true' (default is true)
event-service_1    | wait-for-it.sh: postgres:5432 is available after 4 seconds
event-service_1    | Running db migrations...
event-service_1    | 
event-service_1    | > event-service@1.0.0 db:migrate /app
event-service_1    | > knex --knexfile ./config/knexfile.js migrate:latest
event-service_1    | 
event-service_1    | Batch 1 run: 5 migrations
event-service_1    | Completed db migrations
event-service_1    | DB_SEED set to 'true' (default is false)
event-service_1    | Running db seeding...
event-service_1    | 
event-service_1    | > event-service@1.0.0 db:seed /app
event-service_1    | > knex --knexfile ./config/knexfile.js seed:run
event-service_1    | 
event-service_1    | Ran 1 seed files
event-service_1    | Completed db seeding
event-service_1    | 
event-service_1    | > event-service@1.0.0 start /app
event-service_1    | > node app.js
event-service_1    | 

[briandamaged]

@rlee89 : Actually, this sorta relates to the earlier conversation between @kibertoad and I.

Generally speaking, it’s risky for modules to make assumptions about the current value of process.cwd(). It can lead to all sorts of surprises when another piece of code changes the process’s current working directory.

If possible, you should try to resolve paths relative to a fixed value, such as __dirname. For example:

// __dirname   will always be the folder that contains  knexfile.js
const MIGRATION_DIR = path.resolve(__dirname, 'migrations');