Getting error when loading on flask | Session issue

I am getting session management issue when try to use it in flask application, can you please help me

ValueError: Tensor(“TensorSliceDataset:0”, shape=(), dtype=variant) must be from the same graph as Tensor(“Iterator:0”, shape=(), dtype=resource).

Sample code structure

from io import StringIO
import flask
import pandas as pd
import tensorrec as tr
import tensorflow as tf


class ScoringService(object):
    model = None                
	model_path = 'model path'
    @classmethod
    def get_model(cls):
        if cls.model == None:
            cls.model = tr.TensorRec.load_model(model_path)
           
        return cls.model

    @classmethod
    def get_reco(cls, model,use_ft, ite_ft):
            tf.reset_default_graph()
            predictions = model.predict(use_ft, ite_ft)
        return predictions

    @classmethod
    def predict(cls, input):
        clf = cls.get_model()
        n_reco_test = cls.get_reco(clf, input,use_ft, ite_ft)
        return n_reco_test

# The flask app for serving predictions
app = flask.Flask(__name__)

@app.route('/invocations', methods=['POST'])
def transformation():
    data = None

    # Convert from CSV to pandas
    if flask.request.content_type == 'text/csv':
        data = flask.request.data.decode('utf-8')
        s = StringIO(data)
        data = pd.read_csv(s, header=None)
    else:
        return flask.Response(response='This predictor only supports CSV data', status=415, mimetype='text/plain')
    # Do the prediction
    predictions = ScoringService.predict(data)
    return flask.Response(response=predictions, status=200,mimetype='text/csv')

[gallmerci]

@jaiswalvineet I got the same problem. The tf graph is not thread safe and needs to be globally defined and always reused when doing the prediction. Here is how I solved it (actually solution is coming from https://github.com/keras-team/keras/issues/2397#issuecomment-306687500):

I inserted a global model variable, loaded the model and stored the graph after model loading:

import tensorflow as tf
from tensorrec import TensorRec
model = TensorRec.load_model(directory_path=model_path)
graph = tf.get_default_graph()

and then in every call, I used the same graph as loaded in graph with a new session:

with graph.as_default():
    session = tf.Session()
    with session.as_default():
        # Now do the prediction
        predictions = ScoringService.predict(data)

[jaiswalvineet]

I removed the tf.reset_default_graph() bus still same error, seems the flask app does persist the graph, it’s strange that its working if I directly run it but does not work if I call it from flask app, if its running one time for you from flask then refresh it on browser, you will get the error, if not then please share your flask app …thanks in advance