Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Pretrained models have terrible performance

See original GitHub issue

I evaluate all the efficient architecture with the pre-trained weights available on this repo and the performances are different than the ones shared in this repo.
What is going on?

|                 |    top1 |    top5 |   time (s) |
|:----------------|--------:|--------:|-----------:|
| efficientnet-b0 | 0.7465  | 0.91932 |    78.2806 |
| efficientnet-b1 | 0.7461  | 0.91616 |   120.922  |
| efficientnet-b2 | 0.79394 | 0.9459  |   159.07   |
| efficientnet-b3 | 0.81212 | 0.95526 |   253.05   |
| efficientnet-b4 | 0.82788 | 0.96234 |   542.88   |
| efficientnet-b5 | 0.8345  | 0.9663  |  1049.4    |
| efficientnet-b6 | 0.84024 | 0.96908 |  1853.14   |
| efficientnet-b7 | 0.84106 | 0.9692  |  3245.83   |

Images are resized using the following code

resize_size = {

    'efficientnet-b0': 224,
    'efficientnet-b1': 240,
    'efficientnet-b2': 260,
    'efficientnet-b3': 300,
    'efficientnet-b4': 380,
    'efficientnet-b5': 456,
    'efficientnet-b6': 528,
    'efficientnet-b7': 600,

}

transform = Compose([
            Resize(size, Image.BICUBIC),
            CenterCrop(size),
            ToTensor(),
            Normalize(
            mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
   ])

valid_dataset = ImageNet(root='/home/zuppif/Downloads/ImageNet', split='val', transform=transform)


valid_loader = torch.utils.data.DataLoader(valid_dataset, batch_size=batch_size, shuffle=False,
                                                num_workers=12, pin_memory=True)

My code to compute (I used sotabencheval) top-1 and top-5 is correct.

Issue Analytics

State:
Created 3 years ago
Comments:11 (3 by maintainers)

Top GitHub Comments

1reaction

lukemelascommented, Dec 21, 2020

😃

0reactions

daixiangzicommented, Dec 21, 2020

Haha，maybe

我没有什么天赋，我只是生而由来的执着。

—Original— From: “Francesco Saverio Zuppichini”<notifications@github.com> Date: Mon, Dec 21, 2020 21:07 PM To: “lukemelas/EfficientNet-PyTorch”<EfficientNet-PyTorch@noreply.github.com>; Cc: “Mention”<mention@noreply.github.com>;“daixiangzi”<543826458@qq.com>; Subject: Re: [lukemelas/EfficientNet-PyTorch] Pretrained models have terrible performance (#237)

It may be that resnet18 is better for your task, you have to consider that this is rocket science, sota models are sota only for one specific task

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

Top Results From Across the Web

Why is there a decrease in the performance of pre-trained ...

A pretrained model, even if it sucks and produces sucky results on your data, can help you get the end-2-end system up and...

Approach pre-trained deep learning models with caution

Pre-trained models are easy to use, but are you glossing over details that could impact your model performance? How many times have you...

Successfully-loaded pre-trained model perform awful - vision

I came across some problems when I tried to fine-tuning a VGG19 network on Image-net. I load the model's parameters from “vgg19-dcbb9e9d.pth” by: ......

Using Pre-Training Can Improve Model Robustness ... - arXiv

Surprisingly, pre-training provides no performance bene- ... The pre-trained models have comparable clean accuracy to adversarially trained models from ...

Approach pre-trained deep learning models with caution

Pre-trained models are easy to use, but are you glossing over details that could impact your model performance?