Firestore emulator fails to retrieve refresh_token in CI, errors out

[REQUIRED] Environment info

firebase-tools: 7.12.1

Platform: Linux

[REQUIRED] Test case

Starting the simulator in Google Cloud Build fails randomly and is causing our tests to be flaky

[REQUIRED] Steps to reproduce

Right now I’m running lerna with jest. The command is run on Node 10.15.1 on Cloud Build. Here’s the yarn command I’m running: firebase emulators:exec --only firestore 'yarn jest'".

The yarn jest command looks like: "jest": "FIRESTORE_EMULATOR_HOST=localhost:9090 jest --runInBand --coverage"

When run with the --debug flag, I get some more information and a stack trace of where things blew up:

Already have image: build-image
yarn run v1.13.0
$ lerna run test --stream
lerna notice cli v3.20.2
lerna info versioning independent
lerna info Executing command in 2 packages: "yarn run test"
siteline-server: $ firebase emulators:exec --only firestore --debug 'yarn jest'
siteline-server: [2020-02-10T21:44:12.408Z] ----------------------------------------------------------------------
siteline-server: [2020-02-10T21:44:12.411Z] Command:       /usr/local/bin/node /workspace/node_modules/.bin/firebase emulators:exec --only firestore --debug yarn jest
siteline-server: [2020-02-10T21:44:12.412Z] CLI Version:   7.12.1
siteline-server: [2020-02-10T21:44:12.412Z] Platform:      linux
siteline-server: [2020-02-10T21:44:12.412Z] Node Version:  v10.15.1
siteline-server: [2020-02-10T21:44:12.412Z] Time:          Mon Feb 10 2020 21:44:12 GMT+0000 (Coordinated Universal Time)
siteline-server: [2020-02-10T21:44:12.414Z] ----------------------------------------------------------------------
siteline-server: [2020-02-10T21:44:12.414Z] 
siteline-server: [2020-02-10T21:44:12.425Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
siteline-server: [2020-02-10T21:44:12.426Z] > attempting to authenticate via app default credentials
siteline-server: [2020-02-10T21:44:12.551Z] TypeError: Cannot create property 'refresh_token' on string 'Not Found
siteline-server: '
siteline-server:     at /workspace/node_modules/google-auto-auth/node_modules/google-auth-library/lib/auth/oauth2client.js:208:28
siteline-server:     at /workspace/node_modules/google-auto-auth/node_modules/google-auth-library/lib/auth/computeclient.js:85:7
siteline-server:     at Request._callback (/workspace/node_modules/google-auto-auth/node_modules/google-auth-library/lib/transporters.js:106:7)
siteline-server:     at Request.self.callback (/workspace/node_modules/request/request.js:185:22)
siteline-server:     at Request.emit (events.js:189:13)
siteline-server:     at Request.EventEmitter.emit (domain.js:441:20)
siteline-server:     at Request.<anonymous> (/workspace/node_modules/request/request.js:1161:10)
siteline-server:     at Request.emit (events.js:189:13)
siteline-server:     at Request.EventEmitter.emit (domain.js:441:20)
siteline-server:     at IncomingMessage.<anonymous> (/workspace/node_modules/request/request.js:1083:12)
siteline-server:     at Object.onceWrapper (events.js:277:13)
siteline-server:     at IncomingMessage.emit (events.js:194:15)
siteline-server:     at IncomingMessage.EventEmitter.emit (domain.js:441:20)
siteline-server:     at endReadableNT (_stream_readable.js:1103:12)
siteline-server:     at process._tickCallback (internal/process/next_tick.js:63:19)
siteline-server: Error: An unexpected error has occurred.
siteline-server: error Command failed with exit code 2.
siteline-server: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
lerna ERR! yarn run test exited 1 in 'siteline-server'
lerna WARN complete Waiting for 1 child process to exit. CTRL-C to exit immediately.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[REQUIRED] Expected behavior

Does not crash

[REQUIRED] Actual behavior

Crashes

[joelpoloney]

I followed these instructions to get a variable setup: https://cloud.google.com/cloud-build/docs/securing-builds/use-encrypted-secrets-credentials#example_build_request_using_an_encrypted_variable.

That combined with your cloudbuild.yaml suggestions, I was able to get it working (on first try!). For those curious, I simplified this to just decode to the FIREBASE_TOKEN env variable directly with:

secrets:
- kmsKeyName: projects/[PROJECT-ID]/locations/global/keyRings/[RING-ID]/cryptoKeys/FIREBASE_TOKEN
  secretEnv:
    FIREBASE_TOKEN: <base64 encoded value>
steps:
- name: 'build-image'
  entrypoint: yarn
  args: ['test']
  secretEnv:
    - FIREBASE_TOKEN

[joelpoloney]

Ah ok great. This is probably the issue – having a permanent token looks like it fixes it. I’ll try the route with KMS and report back how it goes. Thank you for the super detailed info!