Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Chart Generator: Support multiple indexing tasks in the same challenge

See original GitHub issue

The Rally chart generator does not appear to support parallel tasks, as defined in the dense vector track. The following command does not produce charts for operations in parallel tasks:

$ esrally generate charts --configuration-name=nightly --chart-spec-path=~/dev/src/night-rally/night_rally/resources/race-configs-group-3.json --chart-type=time-series --output-path=~/tmp/nightly-dv-charts.ndjson

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/

[INFO] Loading track data...
[INFO] Generating charts...
[INFO] Found [bulk-index] of type [raw-bulk] in [logging-indexing], adding it to indexing dashboard.

[INFO] Found [bulk-index-initial-load] of type [raw-bulk] in [security-indexing-querying], adding it to indexing dashboard.

[INFO] Found [bulk-index] of type [raw-bulk] in [security-indexing], adding it to indexing dashboard.

@pquentin @mayya-sharipova this is required for dense vector track chart generation.

Issue Analytics

State:
Created a year ago
Reactions:1
Comments:5 (4 by maintainers)

Top GitHub Comments

1reaction

pquentincommented, Jun 17, 2022

We actually have the same issue for the many-shards benchmark, so we’ll support this eventually but for now we’ll manually edit the labels before uploading.

1reaction

pquentincommented, Jun 16, 2022

The issue isn’t about parallel tasks, we support them just fine. What’s going on is that we have two indexing tasks in the same challenge, which isn’t supported (the query chart is OK). So we do have two indexing tasks, but they’re both named “add-4g-1node”. We can see this clearly when looking at the generated charts with a 90 days interval:

There are two ways to get out of here:

Modify the track to include two challenges, each one of them performing a single indexing task (one parallel, one not)
Modify all the indexing charts to add the name of the task in the legend, eg we would have “add-4g-1node-index-append” and “add-4g-1node-index-update-concurrent-with-searches” which would be verbose but explicit.

@mayya-sharipova Is it important to you to have a single challenge?