question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

user witnessing timeouts on @grpc/grpc-js@0.6.9

See original GitHub issue

@ericpearson, I went ahead and opened this issue in reference to the discussion in #790.

I’d like start delving in a bit deeper, so that we can hopefully figure out a pattern in the timeouts you’re seeing.

Environment details

  • OS/platform: What are you running @google-cloud/pubsub on, Google App Engine, Kubernetes Engine, other infrastructure?
  • Node.js version: ?
  • npm version: ?
  • @google-cloud/pubsub version: ?? (Is PubSub a good place for us to start with debugging, you’d mentioned you’d seen the issue with some other grpc libs, but PubSub tends to be people’s highest workload).

Getting us debug information

If you run one of your clients with the following environment variables set:

export GRPC_TRACE=all
export GRPC_VERBOSITY=DEBUG

or in your application run the following (before your pubsub import):

process.env.GRPC_TRACE = 'all';
process.env.GRPC_VERBOSITY = 'DEBUG';
const {PubSub} = require('@google-cloud/pubsub');

☝️ we will start seeing detailed grpc logs, and should be able to identify exactly what happened immediately before a message timing out.

To get the logs over to me, you can both open an issue:

https://issuetracker.google.com/savedsearches/559741, and send an email my way so that I can make sure the grpc team gets the logs immediately (I’m ben [at] google.com).

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:16 (9 by maintainers)

github_iconTop GitHub Comments

1reaction
npomfretcommented, Nov 11, 2019

The “Deadline exceeded” bug is back for me using:

  "dependencies": {
    "@google-cloud/firestore": "^2.6.0",
    "@google-cloud/logging": "5.5.5",
    "@google-cloud/logging-bunyan": "^2.0.0",
    "@google-cloud/pubsub": "1.1.5",

I thought it was fixed? This is the error:

{"ackIds":["KlgRTgQhIT4wPkVTRFAGFixdRkhRNxkIaFEOT14jPzUgKEUaAiBuFD1CWhoOB1xcdQdRDB8neDB2a1xHUwFPU3RfcysvV1RYcgBWCRtye2d0bFsVAyrBuOnH3ZuSWUZgTc3pmMdHXbXW3-giZh49WxJLLD5-MzVFQV5AEkw4AERJUytDCw"],"code":4,"details":"Deadline exceeded","metadata":{"internalRepr":{},"options":{}}}

Is there any more information I can provide?

1reaction
bcoecommented, Nov 4, 2019

@oande I’ve created a tracking issue to add tuning documentation to our README.md:

https://github.com/googleapis/nodejs-pubsub/issues/810


I’m closing this issue for the time being, due to lack of information about the original underlying bug. @ericpearson, please don’t hesitate to re-open if you’re continuing to bump into issue,s and can provide some specific debugging information about the timeouts you’re seeing.

Read more comments on GitHub >

github_iconTop Results From Across the Web

No results found

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found