Emulator: `invalid_grant` and erratic error handling

Steps to reproduce

I can’t get PubSub working at all locally with the emulator (all just going by the docs) - emulator runs fine with no errors, but doesn’t look like its receiving anything.

src/subscriptions.ts defines a map of subscription names to consumer funcs, and after our express server has started running we asynchronously attempt to start listening to all subscriptions (including creating subscriptions/topics if they don’t exist). We’re initialising the SubscriberClient for Synchronous Lease Management and PubSub for standard async consumption.

// Init v1 PubSub SubscriberClient and ensure it is initialised
const client = new v1.SubscriberClient()
await client.initialize().catch(e => {
  console.error(e)
  // sentry error handling goes here
  return
})

// Init v2 PubSub client and find/create topic
const pubsub = new PubSub()
let topic = pubsub.topic(consumer.topic)
let findTopicRsp = await new Promise(resolve => {
  topic.get((err, topic, apiResponse) => {
    console.log(JSON.stringify(apiResponse))
    if (err != null) throw err
    return resolve(topic)
  })
}).catch(e => {
  console.error(e)
  // sentry error handling goes here
  return
})
...

This function suddenly dies with no error or stacktrace at topic.get().

Using Promise.resolve() is a bit messy with findTopicRsp, but I’ve found that every fews runs (randomly) I’ll actually get an error back - Failed to retrieve auth metadata with error: invalid_grant. I originally had it written as a typical await call but could never get any kind of error response (regardless of wrappers / try/catch / .catch() / etc).

To me, an invalid_grant error doesn’t makes much sense connecting to a local emulator. I decided to log the options which new PubSub() ends up collecting:

➜  backend git:(staging) ✗ $(gcloud beta emulators pubsub env-init)
➜  backend git:(staging) ✗ yarn start
$ some other build steps go here
$ npm run build
$ npx ts-node -r dotenv/config -r tsconfig-paths/register src/index.ts
process.env.PUBSUB_EMULATOR_HOST: ::1:8267
process.env.PUBSUB_PROJECT_ID: undefined
pubsub.isEmulator: true
pubsub.options: {"libName":"gccl","libVersion":"2.10.0","scopes":["https://www.googleapis.com/auth/cloud-platform","https://www.googleapis.com/auth/pubsub"],"servicePath":"","sslCreds":{"callCredentials":{}}}

Even after onerously course-correcting the servicePath by initialising like this:

let port =
    process.env.PUBSUB_EMULATOR_HOST != null
      ? process.env.PUBSUB_EMULATOR_HOST.substring(
          process.env.PUBSUB_EMULATOR_HOST.length - 4,
          process.env.PUBSUB_EMULATOR_HOST.length,
        )
      : 8085
const pubsub = new PubSub({
    port,
    apiEndpoint: `https://localhost:${port}`,
})

I now get a response with the correct port/endpoint:

pubsub.options: {"libName":"gccl","libVersion":"2.10.0","scopes":["https://www.googleapis.com/auth/cloud-platform","https://www.googleapis.com/auth/pubsub"],"port":8267,"apiEndpoint":"https://localhost:8267","servicePath":"localhost","sslCreds":{"callCredentials":{}}}

But the code still suddenly stops executing at topic.get(), and I’ve not had a single error caught by our local Sentry instance or in console or in the emulator. When I use vscode’s debugger, I manage to pry out the error: Error: 16 UNAUTHENTICATED: Failed to retrieve auth metadata with error: invalid_grant error.

I can’t find any resources on invalid_grant, so pointers here would be greatly appreciated. Also not sure what’s happening with error handling - might be a stupid mistake I’ve made up the chain somewhere but I can’t see anything atm. Thanks

Environment details

• OS: macOS 11.2
• Node.js version: v14.15.5
• npm version: 6.14.11
• @google-cloud/pubsub version: 2.10.0

[feywind]

I’d like to get that in soon… there is some other weirdness going on with the build process though, might be next week.

[lucasjohnston]

Have submitted a PR for one of these docs changed btw - https://github.com/googleapis/nodejs-pubsub/pull/1245