Not able to convert trained model for Tensorflow Serving API
See original GitHub issueFirst of all, thanks for taking the time to make this amazing repo.
While using this repo I am stuck in the following situation, I am trying to convert a trained keras model to a compatible format to enable running it using tensorflow serving. I have the following two pieces of code with me which I am trying to use to convert.
- Using model variable:
from __future__ import print_function
import os
import shutil
from glob import glob
from keras.models import load_model
from keras.preprocessing.image import img_to_array
from keras.applications import imagenet_utils
from tensorflow.python.keras.estimator import model_to_estimator
import tensorflow as tf
import keras_resnet
from tensorflow.python.keras._impl.keras.models import Model
from keras_retinanet import models
import keras_resnet.models
from keras_retinanet.utils.image import read_image_bgr, preprocess_image, resize_image
from keras_retinanet.utils.visualization import draw_box, draw_caption
from keras_retinanet.utils.colors import label_color
def export_for_serving(model_path, model):
'''
Converts model to the TensorFlow estimator and saves it to the disk
:param model: keras model to prepare for serving
'''
export_dir = 'tf_serving_model/'
if os.path.exists(export_dir):
shutil.rmtree(export_dir)
tf_estimator = model_to_estimator(keras_model=model)
tf_estimator.export_savedmodel(
export_dir,
serving_input_receiver_fn,
strip_default_attrs=True)
model_path = os.path.join('.', 'snapshots', 'resnet50_csv_15_model.h5')
model = Model(models.load_model(model_path, backbone_name='resnet50'))
print("loaded model")
export_for_serving(model_path=model_path, model=model)
Running this throws the following error:
Traceback (most recent call last): File "keras_to_tensorflow_serving.py", line 42, in <module>
model = Model(models.load_model(model_path, backbone_name='resnet50'))
TypeError: __init__() missing 1 required positional argument: 'outputs'
- Using the .h5 file stored on disk:
from __future__ import print_function
import os
import shutil
from glob import glob
from keras.models import load_model
from keras.preprocessing.image import img_to_array
from keras.applications import imagenet_utils
from tensorflow.python.keras.estimator import model_to_estimator
import tensorflow as tf
import keras_resnet
from keras_retinanet import models
import keras_resnet.models
from keras_retinanet.utils.image import read_image_bgr, preprocess_image, resize_image
from keras_retinanet.utils.visualization import draw_box, draw_caption
from keras_retinanet.utils.colors import label_color
def export_for_serving(model_path, model):
'''
Converts model to the TensorFlow estimator and saves it to the disk
:param model: keras model to prepare for serving
'''
export_dir = 'tf_serving_model/'
if os.path.exists(export_dir):
shutil.rmtree(export_dir)
tf_estimator = model_to_estimator(keras_model_path=model_path)
tf_estimator.export_savedmodel(
export_dir,
serving_input_receiver_fn,
strip_default_attrs=True)
model_path = os.path.join('.', 'snapshots', 'resnet50_csv_15_model.h5')
model = models.load_model(model_path, backbone_name='resnet50')
print("Loaded Model")
export_for_serving(model_path=model_path, model=model)
I am receiving the following error:
Traceback (most recent call last):
File "keras_to_tensorflow_serving.py", line 42, in <module>
export_for_serving(model_path=model_path, model=model)
File "keras_to_tensorflow_serving.py", line 32, in export_for_serving
tf_estimator = model_to_estimator(keras_model_path=model_path, custom_objects=custom_objects)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/_impl/keras/estimator.py", line 456, in model_to_estimator
keras_model = models.load_model(keras_model_path)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/_impl/keras/engine/saving.py", line 240, in load_model
model = model_from_config(model_config, custom_objects=custom_objects)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/_impl/keras/engine/saving.py", line 317, in model_from_config
return deserialize(config, custom_objects=custom_objects)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/_impl/keras/layers/serialization.py", line 63, in deserialize
printable_module_name='layer')
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/_impl/keras/utils/generic_utils.py", line 171, in deserialize_keras_object
list(custom_objects.items())))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/_impl/keras/engine/network.py", line 1060, in from_config
process_layer(layer_data)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/_impl/keras/engine/network.py", line 1046, in process_layer
layer = deserialize_layer(layer_data, custom_objects=custom_objects)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/_impl/keras/layers/serialization.py", line 63, in deserialize
printable_module_name='layer')
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/_impl/keras/utils/generic_utils.py", line 173, in deserialize_keras_object
return cls.from_config(config['config'])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/_impl/keras/engine/base_layer.py", line 473, in from_config
return cls(**config)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/_impl/keras/layers/normalization.py", line 107, in __init__
**kwargs
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/layers/normalization.py", line 146, in __init__
name=name, trainable=trainable, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/keras/_impl/keras/engine/base_layer.py", line 128, in __init__
raise TypeError('Keyword argument not understood:', kwarg)
TypeError: ('Keyword argument not understood:', u'freeze')
I am using the latest version of the repo using Keras 2.2.0. Also, the model I am trying to convert is trained using the same latest version of the repo.
Can anyone please point out the mistake I am making. Any help would be greatly appreciated. Thanks in advance.
Issue Analytics
- State:
- Created 5 years ago
- Comments:29 (8 by maintainers)
Top Results From Across the Web
Train and serve a TensorFlow model with ...
This guide trains a neural network model to classify images of clothing, like sneakers and shirts, saves the trained model, and then serves ......
Read more >Serving TensorFlow models with TF Serving
TensorFlow Serving is a flexible, high-performance serving system for machine learning models, designed for production environments.
Read more >Convert a TensorFlow model in a format that can be served
tf.saved_model.simple_save => In this case, only Predict API is supported during Serving. Example of this is mentioned by KrisR89 in his answer.
Read more >How to Serve Machine Learning Models With TensorFlow ...
In this tutorial, I'm going to show you how to serve ML models using Tensorflow Serving, an efficient, flexible, high-performance serving system for...
Read more >Issue in tensorflow serving java API when using keras ...
As load the model using tensorflow serving java apis, we encountered the log “The specified SavedModel has no variables; no checkpoints were restored.”.It...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I use the following to create a TensorFlow model for TensorFlow serving. The code below assumes that you have a RetinaNet model with 8 classes using the
resnet101
backbone with weights saved in a file calledmodel.h5
. You’ll need to edit those for your use case. It will create a folder calledretinanet_savedmodel
with the TensorFlow files inside of it.After executing the above, you can use TensorFlow Serving using the following command (note the
1
which is important).I hope this helps someone! 😃
I haven’t found a solution, but I think I’ve found the root of the problem.
Retinanet implements batch norm with it’s own layer that allows freezing, found here: https://github.com/broadinstitute/keras-resnet/blob/master/keras_resnet/layers/_batch_normalization.py It passes one extra boolean variable
freeze
that controls the freezing. The file is pretty simple but I can explain it if someone doesn’t get it.So when tensorflow tries to load up the custom objects, it gets confused when it gets batch norm with one extra variable name “freeze,” and thus returns the error. My layer’s batch norm was all default values, so I tried doing something like so:
but ended up STILL getting the error. Based on the traceback, which I’ll put below, I was pretty certain that replacing the custom BatchNormalization would do the trick. I looked at the other layers in custom_objects and they didn’t have anything like “freeze,” and the final lines of the traceback imply that the problem is in defining the normalization. The fact that it’s a keyword argument further reinforces my idea that it’s the batch norm layer.
I didn’t figure it out but I think we can get this working!!
Here’s the full traceback error I got. Happy to provide more info if it’d help.