Cannot download IWSLT dataset

Running

train_data, valid_data, test_data = IWSLT.splits(exts = ('.de', '.en'), fields = (SRC, TRG))

on Google Colab leads to this error:

TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

NewConnectionError                        Traceback (most recent call last)
NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f9467921c18>: Failed to establish a new connection: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

MaxRetryError                             Traceback (most recent call last)
MaxRetryError: HTTPSConnectionPool(host='wit3.fbk.eu', port=443): Max retries exceeded with url: /archive/2016-01//texts/de/en/de-en.tgz (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f9467921c18>: Failed to establish a new connection: [Errno 110] Connection timed out',))

During handling of the above exception, another exception occurred:

ConnectionError                           Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    514                 raise SSLError(e, request=request)
    515 
--> 516             raise ConnectionError(e, request=request)
    517 
    518         except ClosedPoolError as e:

ConnectionError: HTTPSConnectionPool(host='wit3.fbk.eu', port=443): Max retries exceeded with url: /archive/2016-01//texts/de/en/de-en.tgz (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f9467921c18>: Failed to establish a new connection: [Errno 110] Connection timed out',))

[garyhlai]

@ghlai9665 thanks for the new url. Can you submit a PR and update the IWSLT dataset url with the new one? The new link will download all the files and we will need to fetch specific languages from the local. Let me know if you need any help there.

I used download_from_url func and it works:

url = 'https://drive.google.com/uc?id=1l5y6Giag9aRPwGtuZHswh3w5v3qEz8D8'
torchtext.utils.download_from_url(url)

Remember to add a CI test in test/data/test_builtin_datasets.py (similar to test_multi30k). We have an experimental IWSLT dataset (here).

Just saw this. Working on it!

[zhangguanheng66]

Fixed by https://github.com/pytorch/text/pull/1115