Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BacktranslationDataset doesn't work with multiple workers

See original GitHub issue

if num_workers > 0, there will be an error: AttributeError: Can't pickle local object 'SemisupervisedTranslationTask.build_model.<locals>.backtranslate_fn'

Issue Analytics

State:
Created 4 years ago
Comments:5 (2 by maintainers)

Top GitHub Comments

3reactions

pipibjccommented, Jun 14, 2019

Thanks for reporting! I will need to find some workaround of it. For now we can only use one worker if we want to use BacktranslationDataset

1reaction

mnoukhovcommented, Aug 8, 2022

This is still an issue

Top Results From Across the Web

Data Augmentation in NLP Using Back Translation With ...

Temporary translation: translate each of the original training labeled data into a different language. In our case, it will be from English to...

Back-Translation for Cross-Cultural Research - SAGE Journals

Abstract. Two aspects of translation were investigated: (1) factors that affect translation quality, and (2) how equivalence between source and target versions ...

dataset - Python Google Translate API error - Stack Overflow

My problem might come from the fact that I am using multi-threading : 100 workers for translating the whole dataset. If they work...

Increasing Lilt's Machine Translation Throughput - Lilt Labs

- Split large documents into chunks and process the chunks in parallel using multiple workers. - Return partial results during processing, so ...

Machine Translation: Data Cleaning | Defined.ai Blog

Learn how to clean data for machine translation from Defined.ai. In this definitive guide, you will learn about dirty data and data-cleaning ...