question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Asking for help,BulkRequest operation , which caused many threads to be blocked

See original GitHub issue

Conductor version: 2.19.0

Recently, I found that there was a serious synchronization lock problem during ES operation, which caused many threads to be blocked

workflow.elasticsearch.index.batchSize=10

ElasticSearchRestDAOV6.java

    private synchronized void indexBulkRequest(String docType) {
       .....
    }

For example:

  • Updateworkflow of system task worker - * blocked by indexBulkRequest method
  • Workflowsweeper blocked by indexBulkRequest method
  • Startworkflow blocked by indexBulkRequest 26603.stack.zip method

Do you have any plans to improve this situation? Thanks.

Attachment is stack information

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:7 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
manan164commented, Nov 18, 2019

Also, you can try having a higher queue size.

1reaction
manan164commented, Nov 18, 2019

Hi @manheiba . From the stack trace looks like you are using sync configuration but environment is high throughput. Can you please try enable async indexing

Read more comments on GitHub >

github_iconTop Results From Across the Web

Elasticsearch Bulk API - Stack Overflow
I tried to break the records into 5000 and send 5000 everytime inside a While loop and every 5000, I will use Thread.sleep...
Read more >
Break indexDAO to publisherDAO and indexDAO · Issue #1243
manan164 mentioned this issue on Nov 17, 2019. Asking for help,BulkRequest operation , which caused many threads to be blocked #1399.
Read more >
Indexing fails with `error: elastic: Error 429 (Too Many ... - GitLab
On the "Indexing concurrency" approach: I lowered the number of Bulk request concurrency to 4 and expected no more than 16 indexing threads...
Read more >
Bulk API | Java REST Client [6.8] - Elastic
A BulkRequest can be used to execute multiple index, update and/or delete operations using a single request. It requires at least one operation...
Read more >
Failed to execute bulk request . - how to solve related issues
This guide will help you check for common problems that cause the log ... when using the Bulk API it is possible to...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found