Model.predict() behavior

TensorFlow.js version: 0.12.3 Browser version: Chrome 67.0.3396.99

Hello, I’ll try to be brief, if this is a non-known behavior I’ll provide all the information needed to reproduce it. But if it’s known, I would suggest to document it.

In my application main loop I call model.predict(…). If later I call the resulting_tensor.data(), then model.predict takes around 10 miliseconds and tensor.data() around 300 miliseconds. So far so good. However, if I don’t call tensor.data(), and just dispose it, next time model.predict() is called, takes 300 miliseconds instead of 10 ms.

Is this expected? If so, any chance it can be documented?

// test 1
let model = ...
let loop = () => {
    let input = tf.fromPixels(...)
    let t = model.predict(input) // 10ms
    t.data().then((data) => { // 300ms
        input.dispose()
        t.dispose()
        requestAnimationFrame(loop)
    })
}
loop()

// test 2
let model = ...
let loop = () => {
    let input = tf.fromPixels(...)
    let t = model.predict(input) // 300ms
    // use t
    input.dispose()
    t.dispose()
    requestAnimationFrame(loop)
}
loop()

[nsthorat]

@justadudewhohacks is correct. The time of actually calling “predict” or for that matter any ops is non-blocking. They simply enqueue programs to be run on the GPU. When you call .data(), that is when you introduce a sync between the CPU and the GPU, which means the timing you get after that call will be the “real” timing that it takes to execute that op.

Hope that helps.

[bileschi]

+nsthorat for fromPixels expertise