question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Allow task to request data using quality=original

See original GitHub issue

My actions before raising this issue

  • Read/searched the docs
  • Searched past issues

We have projects with large (5MP) images. There is a significant delay loading new images if a new chunk needs to be requested, even locally. When the page requests a new chunk it uses something like: api/v1/tasks/68/data?type=chunk&number=2&quality=compressed

Looking at the network logs, there are two parts to the delay: waiting for a response and downloading the data. On a local connection, the delay is mostly due to waiting for a response.

If I make a request manually using quality=original, the delay waiting for a response is almost zero.

  • Is there all ready an option when creating a task to make it use quality=original? I tried all permutations of the settings and couldn’t find a combination that worked.
  • If not, could this option be added? (Which area of the code do I need to look to change this?)

Issue Analytics

  • State:open
  • Created 2 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

3reactions
nmanoviccommented, Jun 9, 2021

@geometrikal , you see the overhead because chunks are created on the fly. When you create a task, you can uncheck “Use cache” in advanced options. CVAT will prepare all necessary data when a task is created. The primary idea behind “Use cache” is to reduce consumption of disk space, reduce time to create a task.

image

2reactions
geometrikalcommented, Jun 9, 2021

@nmanovic Thanks for your reply and for the work on CVAT.

Unfortunately our datasets are very large (> 1TB and growing) and are regularly updated so we have to use the cache. (We are using CVAT as the front end for annotating large environmental datasets - can discuss offline)

I found where to change the code (line 824 in https://github.com/openvinotoolkit/cvat/blob/develop/cvat-core/src/server-proxy.js) and now it works well for internal use (e.g. fast network)

If this could be made an option in the task creation that would be great. Hopefully we can make the changes ourselves and submit a pull request. Another wish would be that CVAT pre-caches chunks around the currently selected image, so that there is no delay waiting for a new chunk to be requested.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Allow task to request data using quality=original #3303 - GitHub
My actions before raising this issue Read/searched the docs Searched past issues We have projects with large (5MP) images.
Read more >
Use data quality tasks | Dataplex - Google Cloud
This document shows you how to create Dataplex data quality tasks that let you schedule and run data quality checks for your built-in...
Read more >
AWS DMS data validation - AWS Database Migration Service
To collect data validation information from CloudWatch enabled statistics, select Enable CloudWatch logs when you create or modify a task using the console....
Read more >
Why should I need Task inside Transport Request?
I created the report in my dev client, that need to be moved to QAS client. In order to do that, I need...
Read more >
Data Quality Management: Roles, Processes, Tools - AltexSoft
Learn how to implement data quality management, what data quality tools ... In the first case, that's accurate order details that you need....
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found