Thoughts

This is a very intriguing approach to database migrations! I like the idea of watching a single file during development.

I have a few questions:

1. Why Javascript?

For a fast CLI tool it would appear Go or an otherwise compiled language would be a great choice. i.e. having a single binary to deal with, working with bytes of the files directly rather than converting to strings, etc… I ask not to push one choice over the other but because I am curious if this is because of some some specific plans, i.e. “this module will eventually become a part of postgraphile”.

1. What is the shadow database doing? Because current.sql is idempotent, it is not exactly clear how the shadow database is helping.

2. How would branching work out? I can imagine a lot of merge conflicts on current.sql and even within the migrations folder.

I realize this tool is brand new but FYI, it is not possible to commit a second migration:

Error: Previous migration with hash 'sha1:f2dbea7bf3b31f50e57be68cb3e6fb4c9a6b5924' doesn't match '000002.sql''s expected previous hash 000001.sql

[benjie]

a. Where does the pg_dump come in?
b. Is the idea to commit this to git as well?

After ‘graphile-worker commit’ a dump will be made and stored to disk. This should be committed. You can then review the diff to ensure that no breaking changes have take place/you’ve not accidentally dropped things you didn’t mean to.

c. Does it always hang one commit behind the development database?

You can run graphile-migrate dump [--shadow] manually to dump your development (or shadow) database; you can then git diff to ensure you’re changing what you think you’re changing. It’s also useful for cleaning up changes from other branches you may have been working on.

But under normal operation, what would cause the shadow database to drift?

Most likely this will be caused by PG version upgrades.

This is the first time current.sql is being executed against the shadow rather than the development DB correct?

current.sql is ran against the shadow database during commit. I can’t remember if it gets put into committed first or not, that’s an implementation detail really.

Shadow DB receives migrations on commit and Dev DB receives migrations on the start of watch.

Correct; you can also run various commands manually to apply migrations to dev or shadow.

only the commit command reads or writes to the shadow DB.

Whilst this may be true currently, it likely won’t be true going forward. The shadow database is something we can reset on a whim. (The reset --shadow command also resets the shadow db.)

[benjie]

afterReset (and related hooks) may or may not already do what you need (or be able to do it without much work). In case you’re not familiar with my work, I’m a big fan of hooks and plugins… 😁