the speed of prediction get much slower after its deployment on the server

My model is converted from Keras. The speed is normal, less than 1s when I test it on localhost. But after I deployed it on my remote Ubuntu server using tomcat(https), the speed of prediction is much slower than localhost, nearly 10s.

    const tensor = tf.tensor(data,[1,93],"int32");
    let value = sentiment_predict(tensor);

    async function sentiment_predict(tensor){
        const model = await tf.loadLayersModel(MODEL_URL);
        let result = model.predict(tensor).toString();
        let num = result.split('[')[2].split(',');
        let neg = parseFloat(num[0]);
        let pos = parseFloat(num[1]);
        return pos - neg;
    }

Maybe because the size of model (6.3M) ? Maybe because I load the model every time I want to predict? So why that and what should I do to? I’ll appreciate it if you could help me.

[adwellj]

Sounds like you might be trying to make a prediction before the model has finished loading.

[Swocky]

@Swocky, I believe that model is going to be the promise for loadLayersModel() instead of the actual model. It likely works in your first example because of the await statement.

Yeah, you are right. And I try this way, the problem has basically been solved. Thanks.

var model;
    loadModel();
    async function loadModel() {
        model = await tf.loadLayersModel(MODEL_URL);
    }

    async function sentiment_predict(tensor){
        let result = model.predict(tensor).toString();
        let num = result.split('[')[2].split(',');
        let neg = parseFloat(num[0]);
        let pos = parseFloat(num[1]);
        return pos - neg;
    }