Elasticsearch IndexerBolt not being acked correctly causing failures in spout

We found with @jcruzmartini that elasticsearch Indexer is bolt acking before emit tuples in afterBulk method is causing ack failures in spout after timeout set in topology.

Proposed solution is change order of emit / ack in com.digitalpebble.stormcrawler.elasticsearch.bolt.IndexerBolt :

if (!failed) {
                    acked++;
                    _collector.emit(StatusStreamName, t, new Values(u,
                            metadata, Status.FETCHED));
                    _collector.ack(t);
                } else {
...
...

After migrate from 1.13 to 1.16 we noticed bad performance in our crawler, and also a lot of failures in the spout, after add IndexerBolt class in our project with that modification it started working correctly with great performance

@jnioche we can create a pull request with a simple change in that class if you want

Thanks! Matias

[jnioche]

changed the title a bit as it is not about the anchoring as such

@jcruzmartini it wasn’t an easy one to spot, but you and @matiascrespof have great detective skills 😉

Thanks again for reporting it and submitting a PR. I’ll go through all the acks to see if this happens anywhere else

[jnioche]

Fixed by #801