question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Training on Private Dataset

See original GitHub issue

Thank you for sharing this code with us.

Can I train on my own dataset with only 11 classes? I am overriding config.num_classes with 11 and overriding config.image_size with 512 but I get very bad results, it is almost like the model is not even aware of the image…

I made sure to put in the boxes as yxyx in the dataset and also made sure the classes start from 1 as I think it is needed due to how fast_collate is working.

Would appreciate your kind help, thank you.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:25 (2 by maintainers)

github_iconTop GitHub Comments

3reactions
pasandreicommented, Aug 24, 2020

After 40 epochs with 50 images with batch size of 2, lr 0.001, AdamW:

  1. 50 images is WAY too few images. I would suggest using at least a few thousand.
  2. 40 epochs is too little for efficientdet
  3. Because of the way BatchNorm works, a batch size of 2 is also too little. I think 32 is the hotspot.
  4. You could also increase the learning rate to something like 0.01

Just as a reference, I trained EfficientDet-Lite0 with ~12000 images, 300 epochs, batch size of 40 and with lr ~0.05

1reaction
jinfagangcommented, Sep 29, 2020

@ofekp cascased maskrcnn’s Mask Head with single stage detector seems not a good practise, since the bboxes are too much and not high quality as 2 stages does. Instead, if you try Yolact++ style, you can get a very nice && very easy to deploy instance segmentation model.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Can a Model Be Differentially Private and Fair?
Training with differential privacy limits the information about any one data point that is extractable but in some cases there's an unexpected side-effect: ......
Read more >
Training Machine Learning Models From Sensitive Data
However, in a commercial setting, we often have to train machine learning models from private or sensitive data.
Read more >
Creating your own dataset - Hugging Face Course
Creating your own dataset · Exploring how long it takes to close open issues or pull requests · Training a multilabel classifier that...
Read more >
Training DETR on Your Own Dataset - Towards Data Science
Depending on the number of samples you have in your dataset, it may be preferable to retrain the model from scratch.
Read more >
How to deploy machine learning with differential privacy?
Through the lens of differential privacy, we can design machine learning algorithms that responsibly train models on private data.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found