model error in web service using Flask

Check List

• I have searched in existing issues but did not find the same one.
• I have read the documents

Environment

• OS [Linux ubuntu 16.04]:

Issue Description

What

When I deploy kashgari.tasks.classification model on the web service using Flask, I encounter an error showing below: tensorflow.python.framework.errors_impl.InvalidArgumentError: Tensor Input-Token:0, specified in either feed_devices or fetch_devices was not found in the Graph

and source code:

@app.route('/api/deep_classify/multi', methods=['GET', 'POST'])
def deep_classify_all():
      text = unquote(request.values.get("text"))
      categories = kashgari.tasks.classification.BiGRU_Model.predict_top_k_class([segment(text)])
        
        return categories

Other Comment

After research a while, I think the reason may be this： []https://kobkrit.com/tensor-something-is-not-an-element-of-this-graph-error-in-keras-on-flask-web-server-4173a8fe15e1

[Kircheis17]

I also encountered this error using flask. The way that solved this problem in my case is keeping the graph & session to be the same for loading the model and doing the prediction, like this:

graph = tf.Graph()
with graph.as_default():
    session = tf.Session()
    with session.as_default():
        # load the model

...

def deep_classify_all():
      text = unquote(request.values.get("text"))
      with graph.as_default():
          with session.as_default():
              categories = model.predict_top_k_class([segment(text)])
      
      return categories

It can’t possibly be the best way. But it works for me. @alexwwang 's point sounds quite convincing, I think that’s why specifying the graph & session helps.

[hhxx2015]

init = tf.global_variables_initializer()
graph = tf.get_default_graph()
session = tf.Session()
session.run(init)
set_session(session)


def predict_prob(self, corpus):
       x = [tokenize(i) for i in corpus]
       logger.info(x)

       global kmodel
       global graph
       global session
       with graph.as_default():
            set_session(session)
            with session.as_default():
                pre = kmodel.predict_top_k_class(x, batch_size=1)
                prelabel = [p["label"] for p in pre]
                return prelabel