TensorFlow model cannot be parsed within the memory limit

Hello,

I’m trying to upload a model generated with TFRanking (32Mb) to BigQuery which I saved like this:

signatures = {
    'serving_default':
        make_keras_tft_serving_fn(
            ranker,
            tf_transform_output,
            context_cols,
            example_cols
        ).get_concrete_function(
            tf.TensorSpec(
                shape=[None],
                dtype=tf.string,
                name='examples'
            )
        ),
}
ranker.load_weights(checkpoint)
ranker.save(model_dir, save_format='tf', signatures=signatures)

but I got this error:

Error while reading data, error message: TensorFlow model cannot be parsed within the memory limit; try reducing the model size

Previously I managed to upload bigger model (>200Mb) created with regular TF 1.13 code, so I don’t understand the message.

Did someone already encounter this ?

Thanks

On Ubuntu 18.04, Python 3.7.3

tensorflow==2.4.1
tensorflow-addons==0.12.1
tensorflow-datasets==4.2.0
tensorflow-estimator==2.4.0
tensorflow-hub==0.11.0
tensorflow-metadata==0.29.0
tensorflow-model-optimization==0.5.0
tensorflow-ranking==0.3.3
tensorflow-serving-api==2.4.1
tensorflow-transform==0.29.0

[tanguycdls]

We think we found a workaround but we’re still not sure it’s viable: we tried transforming our keras models to the old (tf1 format) frozen model that we then re attach to a savedmodel. It seems to reduce the ram.

take a look at this: https://leimao.github.io/blog/Save-Load-Inference-From-TF2-Frozen-Graph/

and then when you have your concrete function reattach it to a tf.Module.

full_model = tf.function(lambda x: model(x))
full_model = full_model.get_concrete_function(
    tf.TensorSpec(model.inputs[0].shape, model.inputs[0].dtype)) # you should fix that to the correct input shapes

# Get frozen ConcreteFunction
frozen_func = convert_variables_to_constants_v2(full_model)
frozen_func.graph.as_graph_def()

module = tf.Module()
module.func = frozen_func
tf.savedmodel.save()... # must specify signature

again we’re still working on the topic so we dont have any long term view of the solution: there might be an issue somewhere…

If you find an issue or have a better idea please tell us !

and some links: https://github.com/search?q=convert_variables_to_constants_v2&type=code

[tanguycdls]

Hello @tanguycdls Thank you again for your help. Did you find any proper solution ? Yours does not work for us as it raise another exception.

We still use the solution above this + some Grappler optims fixed the issue for most domains. You’re using a model that cannot be freeze ?