Embed a single image

Hi, I’m trying to write a script to embed a single image based on your code, it’s look something like this:

import json
import os
from importlib import import_module

import cv2
import tensorflow as tf
import numpy as np

sess = tf.Session()

# Read config
config = json.loads(open(os.path.join(
    '<exp_root>', 'args.json'), 'r').read())

# Input img
net_input_size = (
    config['net_input_height'], config['net_input_width'])
img = tf.placeholder(tf.float32, (None, net_input_size[0], net_input_size[1], 3))

# Create the model and an embedding head.
model = import_module('nets.' + config['model_name'])
head = import_module('heads.' + config['head_name'])

endpoints, _ = model.endpoints(img, is_training=False)
with tf.name_scope('head'):
    endpoints = head.head(endpoints, config['embedding_dim'], is_training=False)

# Initialize the network/load the checkpoint.
checkpoint = tf.train.latest_checkpoint(config['experiment_root'])
print('Restoring from checkpoint: {}'.format(checkpoint))
tf.train.Saver().restore(sess, checkpoint)


raw_img = cv2.imread('<img>')
raw_img = cv2.resize(raw_img, net_input_size)
raw_img = np.swapaxes(raw_img, 0, 1)
raw_img = np.expand_dims(raw_img, axis=0)

emb = sess.run(endpoints['emb'],  feed_dict={img: raw_img})[0]

But the result for a same image with my code and your code are not the same.

Note that there is no any augmentation added when I compute the embedding vector.

Am I missing anything here? Thanks you for the help

[Pandoro]

The only thing that comes to mind right now, is that by the default we use test time augmentation, which you don’t. But that depends on how you are using our embed script to create comparable embeddings in this’ll case.

On Mon, Jun 3, 2019, 10:20 Hoàng Tùng Lâm (Linus) [e-mail] wrote:

Quick update, I’ve just found out that you guys used tf.image.decode_jpeg and tf.image.resize_images instead of OpenCV, I switched to it, the output result is different but still not the same as your code.

Am I missing something like normalization ?? Here is what I’ve changed:

path = tf.placeholder(tf.string) image_encoded = tf.read_file(path) image_decoded = tf.image.decode_jpeg(image_encoded, channels=3) image_resized = tf.image.resize_images(image_decoded, net_input_size) img = tf.expand_dims(image_resized, axis=0)

Thanks 😉

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/VisualComputingInstitute/triplet-reid/issues/81?email_source=notifications&email_token=AAOJDTKVVJNQMXNIJVCBXJLPYTH5ZA5CNFSM4HSFMPL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWYVD7I#issuecomment-498160125, or mute the thread https://github.com/notifications/unsubscribe-auth/AAOJDTKPBI5V64CRLIFKBADPYTH5ZANCNFSM4HSFMPLQ .

[mazatov]

@lamhoangtung I think I figured out the first problem.

1. cv2 loads the image in BGR style so you need to convert it to RGB.
2. There seem to be some differences in the way cv2 and tensorflow load jpeg image. Check https://stackoverflow.com/questions/45516859/differences-between-cv2-image-processing-and-tf-image-processing

So, to get cv2 load embeddings close to the embed.py values, I did the following.

raw_img = cv2.imread(os.path.join(config['image_root'],'query', '0001_c1s1_001051_00.jpg'))
raw_img = cv2.cvtColor(raw_img, cv2.COLOR_BGR2RGB)
raw_img = cv2.resize(raw_img, (net_input_size[1], net_input_size[0]))
raw_img = np.expand_dims(raw_img, axis=0)

If you want to get the exactly same values you can load the image with TF instead of CV2

image_encoded = tf.read_file(os.path.join(config['image_root'],'query', '0001_c1s1_001051_00.jpg'))
image_decoded = tf.image.decode_jpeg(image_encoded, channels=3)
image_resized = tf.image.resize_images(image_decoded, net_input_size)
img = tf.expand_dims(image_resized, axis=0)

# Create the model and an embedding head.
model = import_module('nets.' + config['model_name'])
head = import_module('heads.' + config['head_name'])

endpoints, _ = model.endpoints(img, is_training=False)
with tf.name_scope('head'):
    endpoints = head.head(endpoints, config['embedding_dim'], is_training=False)

tf.train.Saver().restore(sess, os.path.join(config['experiment_root'],'checkpoint-25000') )



emb = sess.run(endpoints['emb'])[0]

I got almost identical embeddings this way