Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Using multiple binary files for training
See original GitHub issueHi, Just like how the workers generate I have multiple train bin and idx files like
train1.en-de.en.bin
train1.en-de.en.idx
train1.en-de.de.bin
train1.en-de.de.idx
train2.en-de.en.bin
train2.en-de.en.idx
train2.en-de.de.bin
train2.en-de.de.idx
Now I want to run the training on these multiple files like a single bin and idx file in each case. Is it possible in a straight forward way to do so or else should I need to change something ?
Thanks.
Issue Analytics
- State:
- Created 4 years ago
- Reactions:2
- Comments:5 (2 by maintainers)
Top Results From Across the Web
Using multiple binary files for training · Issue #678 - GitHub
Now I want to run the training on these multiple files like a single bin and idx file in each case. Is it...
Read more >One-vs-Rest and One-vs-One for Multi-Class Classification
The One-vs-One strategy splits a multi-class classification into one binary classification problem per each pair of classes. Kick-start your ...
Read more >Go Beyond Binary Classification with Multi-Class and Multi ...
You might want to collect more data before training your model in real life. Multi-Label Classification. Next, we'll look at a multi-label ...
Read more >Multiple binary classifiers combining - Stack Overflow
Based on my understanding for OneVsAll, we create a binary classifier per label and train the classifier with the training data.
Read more >Binary and Multiclass Classification in Machine Learning
Binary classification is a task of classifying objects of a set into two groups. Learn about binary classification in ML and its differences ......
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
The following two steps may solve your problems.
train1andtrain2.:to separate directories, which will be iterated upon during epochs in a round-robin manner. For example,fairseq-train train1:train2@luofuli’s answer is the recommended approach.