[BUG] migration failed to run during server deployment

Describe the bug The migration seems to be failing during the deployment of the server. I checked twice already on multiple days and still face this issue.

To Reproduce Steps to reproduce the behavior:

1. Clone the repository
2. rename the config
3. docker-compose up
4. See error

Expected behavior The server should have started without any error.

Os - Gnu/Linux (Ubuntu) x86_64 Docker version - 20.10.12 Docker compose version - 20.10.12

Build Log

3:30:50 AM - Starting compilation in watch mode...
web      | [CO] 
web      | [FE] Webpack: Rebuilding...
web      | [CO] 
web      | [CO] 3:30:53 AM - Found 0 errors. Watching for file changes.
web      | [BE] 
web      | [BE] 3:31:04 AM - Found 0 errors. Watching for file changes.
web      | [BE] $ cd .. && node server/js/main.js
web      | [BE] Could not load config.json, using defaults (error message: ENOENT: no such file or directory, open './config.json')
web      | [BE] No Auth0 configuration found
web      | [BE] APPLICATION -- starting
web      | [BE] Cache is redis
db       | 2022-08-10T03:31:06.187592Z 2 [Warning] Following users were specified in CREATE USER IF NOT EXISTS but they already exist. Corresponding entry in binary log used default authentication plugin 'mysql_native_password' to rewrite authentication information(if any) for them: 'voicecommons'@'%'
db       | 
web      | [BE] APPLICATION -- performing Maintenance
web      | [BE] Running migrations
web      | [BE] APPLICATION -- listening at http://localhost:9000
web      | [BE] unhandled promise rejection Error: ER_BAD_FIELD_ERROR: Unknown column 'undefined' in 'field list'
web      | [BE]     at Query.Sequence._packetToError (/code/node_modules/mysql/lib/protocol/sequences/Sequence.js:47:14)
web      | [BE]     at Query.ErrorPacket (/code/node_modules/mysql/lib/protocol/sequences/Query.js:79:18)
web      | [BE]     at Protocol._parsePacket (/code/node_modules/mysql/lib/protocol/Protocol.js:291:23)
web      | [BE]     at Parser._parsePacket (/code/node_modules/mysql/lib/protocol/Parser.js:433:10)
web      | [BE]     at Parser.write (/code/node_modules/mysql/lib/protocol/Parser.js:43:10)
web      | [BE]     at Protocol.write (/code/node_modules/mysql/lib/protocol/Protocol.js:38:16)
web      | [BE]     at Socket.<anonymous> (/code/node_modules/mysql/lib/Connection.js:88:28)
web      | [BE]     at Socket.<anonymous> (/code/node_modules/mysql/lib/Connection.js:526:10)
web      | [BE]     at Socket.emit (events.js:314:20)
web      | [BE]     at Socket.EventEmitter.emit (domain.js:483:12)
web      | [BE]     at addChunk (_stream_readable.js:297:12)
web      | [BE]     at readableAddChunk (_stream_readable.js:272:9)
web      | [BE]     at Socket.Readable.push (_stream_readable.js:213:10)
web      | [BE]     at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
web      | [BE]     --------------------
web      | [BE]     at Protocol._enqueue (/code/node_modules/mysql/lib/protocol/Protocol.js:144:48)
web      | [BE]     at Connection.query (/code/node_modules/mysql/lib/Connection.js:198:25)
web      | [BE]     at /code/node_modules/db-migrate-mysql/index.js:371:29
web      | [BE]     at Promise._execute (/code/node_modules/bluebird/js/release/debuggability.js:384:9)
web      | [BE]     at Promise._resolveFromExecutor (/code/node_modules/bluebird/js/release/promise.js:518:18)
web      | [BE]     at new Promise (/code/node_modules/bluebird/js/release/promise.js:103:10)
web      | [BE]     at Object.runSql (/code/node_modules/db-migrate-mysql/index.js:366:12)
web      | [BE]     at /code/server/src/lib/model/db/migrations/20220222145730-import-variant-data.ts:115:10
web      | [BE]     at Array.map (<anonymous>)
web      | [BE]     at Class.exports.up (/code/server/src/lib/model/db/migrations/20220222145730-import-variant-data.ts:114:14) {
web      | [BE]   code: 'ER_BAD_FIELD_ERROR',
web      | [BE]   errno: 1054,
web      | [BE]   sqlMessage: "Unknown column 'undefined' in 'field list'",
web      | [BE]   sqlState: '42S22',
web      | [BE]   index: 0,
web      | [BE]   sql: 'INSERT IGNORE INTO variants (locale_id, variant_token, variant_name) VALUES ( undefined,"cy-northwes","North-Western Welsh")'
web      | [BE] }
web      | [BE] [ERROR] Error: ER_BAD_FIELD_ERROR: Unknown column 'undefined' in 'field list'
web      | [BE]     at Query.Sequence._packetToError (/code/node_modules/mysql/lib/protocol/sequences/Sequence.js:47:14)
web      | [BE]     at Query.ErrorPacket (/code/node_modules/mysql/lib/protocol/sequences/Query.js:79:18)
web      | [BE]     at Protocol._parsePacket (/code/node_modules/mysql/lib/protocol/Protocol.js:291:23)
web      | [BE]     at Parser._parsePacket (/code/node_modules/mysql/lib/protocol/Parser.js:433:10)
web      | [BE]     at Parser.write (/code/node_modules/mysql/lib/protocol/Parser.js:43:10)
web      | [BE]     at Protocol.write (/code/node_modules/mysql/lib/protocol/Protocol.js:38:16)
web      | [BE]     at Socket.<anonymous> (/code/node_modules/mysql/lib/Connection.js:88:28)
web      | [BE]     at Socket.<anonymous> (/code/node_modules/mysql/lib/Connection.js:526:10)
web      | [BE]     at Socket.emit (events.js:314:20)
web      | [BE]     at Socket.EventEmitter.emit (domain.js:483:12)
web      | [BE]     at addChunk (_stream_readable.js:297:12)
web      | [BE]     at readableAddChunk (_stream_readable.js:272:9)
web      | [BE]     at Socket.Readable.push (_stream_readable.js:213:10)
web      | [BE]     at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
web      | [BE]     --------------------
web      | [BE]     at Protocol._enqueue (/code/node_modules/mysql/lib/protocol/Protocol.js:144:48)
web      | [BE]     at Connection.query (/code/node_modules/mysql/lib/Connection.js:198:25)
web      | [BE]     at /code/node_modules/db-migrate-mysql/index.js:371:29
web      | [BE]     at Promise._execute (/code/node_modules/bluebird/js/release/debuggability.js:384:9)
web      | [BE]     at Promise._resolveFromExecutor (/code/node_modules/bluebird/js/release/promise.js:518:18)
web      | [BE]     at new Promise (/code/node_modules/bluebird/js/release/promise.js:103:10)
web      | [BE]     at Object.runSql (/code/node_modules/db-migrate-mysql/index.js:366:12)
web      | [BE]     at /code/server/src/lib/model/db/migrations/20220222145730-import-variant-data.ts:115:10
web      | [BE]     at Array.map (<anonymous>)
web      | [BE]     at Class.exports.up (/code/server/src/lib/model/db/migrations/20220222145730-import-variant-data.ts:114:14)
db       | 2022-08-10T03:31:06.313848Z 4 [Note] Aborted connection 4 to db: 'voiceweb' user: 'root' host: '172.18.0.5' (Got an error reading communication packets)
db       | 2022-08-10T03:31:06.313862Z 3 [Note] Aborted connection 3 to db: 'voiceweb' user: 'voicecommons' host: '172.18.0.5' (Got an error reading communication packets)
db       | 2022-08-10T03:31:06.313911Z 2 [Note] Aborted connection 2 to db: 'voiceweb' user: 'root' host: '172.18.0.5' (Got an error reading communication packets)
web      | [BE] error Command failed with exit code 1.
web      | [BE] info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
web      | [FE] Webpack: Built!

[MichaelKohler]

Instead of using Pontoon as source of truth to get language data, it may make sense to populate the database via CLDR on start up, or create a migration that autopopulates the db with a preset languages.

Generally not against that idea, but let’s keep in mind that Sentence Collector takes the list from Pontoon. If the CV website shows other languages that are not contributeable on SC, it might be confusing. Happy to align on this when it comes to it 😃

[moz-rotimib]

@ftyers and @moz-rotimib can I get either/both of you to test the PR #3795 to see it fixes the above issues? Just pull it down locally, run on a fresh (or existing) setup and it should work. Confirm that the locales table is populated as expected (all columns are filled in, name column should have the ISO code of the language). And that variants table is populated with cy and sw variant information.

I’m testing locally now