Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Use YoloR with swin transformer as backbone.

See original GitHub issue

@leondgarse I am trying to get inference using yolor with swin backbone but getting the following results. What can be the issue?

from keras_cv_attention_models import efficientnet, yolor
from keras_cv_attention_models import swin_transformer_v2

from keras_cv_attention_models import efficientnet, yolor
bb = swin_transformer_v2.SwinTransformerV2Small_window16(input_shape=(256, 256, 3), num_classes=1000)
model = yolor.YOLOR(backbone=bb) 

from keras_cv_attention_models import test_images
imm = test_images.dog_cat()
preds = model(model.preprocess_input(imm))
bboxs, lables, confidences = model.decode_predictions(preds)[0]

from keras_cv_attention_models.coco import data
data.show_image_with_bboxes(imm, bboxs, lables, confidences)

resulting output download

Issue Analytics

State:
Created a year ago
Comments:10 (5 by maintainers)

Top GitHub Comments

4reactions

leondgarsecommented, Sep 3, 2022

If you really want, this yolor_swin.h5 is a model trained for 10 epochs.

3reactions

leondgarsecommented, Sep 3, 2022

This is a command I’ve just tested. Detail usage for coco_train_script.py is explained in COCO training and evaluating.

CUDA_VISIBLE_DEVICES='0' ./coco_train_script.py --backbone swin_transformer_v2.SwinTransformerV2Small_window16 \
--det_header yolor.YOLOR --anchors_mode yolor -s yolor_swin

Here is a test result after only runing 9 epochs:

from keras_cv_attention_models import yolor, swin_transformer_v2, test_images

bb = swin_transformer_v2.SwinTransformerV2Small_window16(input_shape=(256, 256, 3), pretrained=None, num_classes=0)
model = yolor.YOLOR(backbone=bb, input_shape=(256, 256, 3), rescale_mode='torch')  # Default rescale_mode from coco_train_script.py is "torch"
model.load_weights('checkpoints/yolor_swin_latest.h5')  # Load the trained weights

# Detect
imm = test_images.dog_cat()
preds = model(model.preprocess_input(imm))
bboxs, lables, confidences = model.decode_predictions(preds)[0]

# Show
from keras_cv_attention_models.coco import data
data.show_image_with_bboxes(imm, bboxs, lables, confidences)

yolor_swin

Top Results From Across the Web

YOLOX with Swin-Transformer backbone - GitHub

YOLOX is an anchor-free version of YOLO, with a simpler design but better performance. I rewrote the version with Swin-Transformer as backbone following ......

Using Swin Transformer as Backbone - IceVision

The script installs IceVision, IceData, the MMDetection library, and Yolo v5 as well as the fastai and pytorch lightning engines. Install from pypi......

Swin-Transformer-YOLOv5 for Real-Time Wine Grape Bunch ...

Swin -transformer is a novel backbone network of hierarchical Vision Transformer, using a multi-head self-attention mechanism that can focus ...

Swin Transformer - Hugging Face

This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision.

ViT-YOLO:Transformer-Based YOLO for Object Detection

Recently, the design of vision backbone architectures that use self- attention is an exciting topic. In this work, an improved backbone MHSA-Darknet is...