Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

AggregationSpout does not release IsInQuery boolean sometimes

See original GitHub issue

First remark: I haven’t identified the cause of this issue, but I think it could still be good for others to be aware of it, therefore I’m opening this issue.

Sometimes my AggregationSpout gets stuck doing nothing because the isInQuery boolean is never reset to false. I suppose this can happen when somehow the onSuccess() and onFailure() methods are not called. Currently I’m considering adding in functionality to reset the isInQuery boolean after a certain amount of time.

Maybe others have encountered this issue as well and have a nicer solution, that would be great to know. I will update this ticket in case I find out more.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:1
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

thijswesterveldcommented, May 27, 2022

Hi @jnioche,

We may have found the root cause of this. The issue seems to be with the onFailure() methods in Hybridspout and AggregationSpout. The onFailure() method in AggregationSpout calls markQueryReceivedNow(), but the one in HybridSpout doesn’t .

We are using a variant of Hybridspout. Failures are always handled by the onFailure() method in that child class, even though the query or response that is causing the failure is from the parent AggregationSpout class.

onResponse() decides between parent/child handling by checking the response properties. Ideally, we’d do something similar in onFailure(), but I don’t think we have the information there to make that decision.

In our variants of the Hybridspout we have solved it by removing the onFailure() override our variant of HybridSpout to make sure we always release isInQuery. This of course introduces the reverse issue of releasing too quickly if the failure comes from a single queue .

jniochecommented, Jun 6, 2022

hi @thijswesterveld, I have just pushed a commit which should solve the problem. We will now use a separate listener to handle the results for specific queues and will let the super class deal with onFailure(), which I think is a lot cleaner. Would be great if you could give it a try. Feel free to reopen this issue if necessary. Thanks again for your reporting it in the first place and the detective work to get to the bottom of the problem.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Releases · DigitalPebble/storm-crawler - GitHub
In a nutshell. various dependency upgrades (JSoup, CrawlerCommons, Tika, Elasticsearch); Java 11; bugfix AggregationSpout does not release IsInQuery boolean ...
Read more >
Expressions - Drift
Expressions are used in all kinds of situations. For instance, where expects an expression that returns a boolean. In most cases, you're writing...
Read more >
Boolean short circuiting is not guaranteed - Morning Coffee
Last week I was debugging a weird bug in one of our services. In the logs we saw an SQL error “repetition-operator operand...
Read more >
where Search Operator - Sumo Logic Docs
The where operator allows you to filter results based on a boolean expression. For example, using where with the boolean operator isValidIP: Filters...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found