Asking for help,BulkRequest operation , which caused many threads to be blocked
See original GitHub issueConductor version: 2.19.0
Recently, I found that there was a serious synchronization lock problem during ES operation, which caused many threads to be blocked
workflow.elasticsearch.index.batchSize=10
ElasticSearchRestDAOV6.java
private synchronized void indexBulkRequest(String docType) {
.....
}
For example:
- Updateworkflow of system task worker - * blocked by indexBulkRequest method
- Workflowsweeper blocked by indexBulkRequest method
- Startworkflow blocked by indexBulkRequest 26603.stack.zip method
Do you have any plans to improve this situation? Thanks.
Attachment is stack information
Issue Analytics
- State:
- Created 4 years ago
- Comments:7 (5 by maintainers)
Top Results From Across the Web
Elasticsearch Bulk API - Stack Overflow
I tried to break the records into 5000 and send 5000 everytime inside a While loop and every 5000, I will use Thread.sleep...
Read more >Break indexDAO to publisherDAO and indexDAO · Issue #1243
manan164 mentioned this issue on Nov 17, 2019. Asking for help,BulkRequest operation , which caused many threads to be blocked #1399.
Read more >Indexing fails with `error: elastic: Error 429 (Too Many ... - GitLab
On the "Indexing concurrency" approach: I lowered the number of Bulk request concurrency to 4 and expected no more than 16 indexing threads...
Read more >Bulk API | Java REST Client [6.8] - Elastic
A BulkRequest can be used to execute multiple index, update and/or delete operations using a single request. It requires at least one operation...
Read more >Failed to execute bulk request . - how to solve related issues
This guide will help you check for common problems that cause the log ... when using the Bulk API it is possible to...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Also, you can try having a higher queue size.
Hi @manheiba . From the stack trace looks like you are using sync configuration but environment is high throughput. Can you please try enable async indexing