Timeout: Incomplete Dataset Export Download

My actions before raising this issue

• Read/searched the docs
• Searched past issues

So I’ve been trying to export one of my datasets for the past several days but in vain. Dumping annotations seems to be working fine, but exporting the dataset always results in a timeout halfway through the download. I’ve attempted this on two different networks and browsers with no success.

Expected Behaviour

Trying to export a dataset (4.1 GB) should download successfully, without any timeouts or errors.

Current Behaviour

Download timeouts after 2GB have downloaded and the resulting file is unusable.

Possible Solution

I’ve tried circumventing the timeout in multiple ways, which have all resulted in failure. 1 - I tried using wget in Colab to get around what I thought was slow internet, but the downloaded file is empty and of Length = 0 .

!wget --no-check-certificate  --load-cookies cookies.txt \
'https://cvat.org/api/v1/tasks/###/dataset?format=TFRecord%201.0&action=download'

2- Update: This only worked once and now times out at 2GB as well.

I tried using Selenium on Colab and it was successful:

from selenium import webdriver

options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')

options.add_experimental_option("prefs", {
  "download.default_directory": r"/content/",
  "download.prompt_for_download": False,
  "download.directory_upgrade": True,
  "safebrowsing.enabled": True
})

wd = webdriver.Chrome('chromedriver',options=options)
wd.get("https://cvat.org/auth/login")

username = wd.find_element_by_id("username")
password = wd.find_element_by_id("password")

username.send_keys("username")
password.send_keys("password")
element = wd.find_element_by_css_selector('.login-form-button')
wd.execute_script("arguments[0].click();", element)

from selenium.webdriver import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

wait = WebDriverWait(wd, 10)
actions = ActionChains(wd)

open = wd.find_element_by_css_selector("#root > section > main > div > div:nth-child(3) > div > div:nth-child(1) > div:nth-child(4) > div:nth-child(1) > div > a")
wd.execute_script("arguments[0].click();", open)

action_button = wd.find_element_by_css_selector("#root > section > main > div > div > div.ant-row-flex.ant-row-flex-space-between.ant-row-flex-middle.cvat-task-top-bar > div:nth-child(2) > button")
actions.move_to_element(action_button).perform()

export_data = wd.find_element_by_css_selector("body > div:nth-child(9) > div > div > ul > li:nth-child(3) > div")
actions.move_to_element(export_data).perform()

data = wd.find_element_by_xpath("//*[@id='export_task_dataset$Menu']/li[1]")
wd.execute_script("arguments[0].click();", data)

Steps to Reproduce (for bugs)

1. Create Task
2. Annotate a few things
3. Export dataset
4. Download timeouts before successful download.

Context

I can’t export my data for use in research.

Your Environment

CVAT.org

[cyrilzakka]

@memirerdol @ConstantSun For some odd reason, curl worked for me today - I was able to download 64 GB worth of my annotated data. Regular downloads using the GUI still times out after 1GB

[azhavoro]

@lgg, we discussed internally and I’m going to disable this limit for cvat.org today