Implement ops for converted audio models: Mfcc, DecodeWav, AudioSpectrogram

TensorFlow.js version

0.3.0

Describe the problem or feature request

I generated a model using the “Simple Audio Recognition” tutorial, which I’d like to run in a browser. I tried to convert it for use with TensorFlow.js, but I had some issues loading the model when I ran the tensorflowjs-converter, as described in this issue: https://github.com/tensorflow/tfjs/issues/246.

I tried resolving this by adding the line: from tensorflow.contrib.framework.python.ops import audio_ops as contrib_audio to converters/converter.py in the tensorflowjs pip package. This fixed the exception when loading the model and now the converter lists the missing ops:

Unsupported Ops in the model
Mfcc, DecodeWav, AudioSpectrogram

Could these ops be implemented for TensorFlow.js?

[davidsoergel]

Thanks for the request! We are working to expand the coverage of TensorFlow Ops in TF.js. I can’t say when these will be done, but user feedback does help up prioritize.

I vaguely recall that spectrograms are a special case-- @nsthorat, was this one of those situations where the browser implementation differs subtly from the TF implementation and so breaks imported models?

[kongkip]

Hi, Instead of doing that, I created a package by extending tf.keras.layers.Layer by creating an audio preprocessor which generates spectrograms and it turned out that the ops are in TFlite.

On DecodeWav, I preprocess the data before basing through the spectrogram layer hence the not included when converting the model to TFlite.

On Tue, 22 Oct 2019, 19:34 Naveen Singh, [e-mail] wrote:

@kongkip https://github.com/kongkip You got anything yet?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tensorflow/tfjs/issues/271?email_source=notifications&email_token=AIELQNOMQ535BXTJSSQUERLQP4TQXA5CNFSM4E6V7AQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEB6MI6Q#issuecomment-545047674, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIELQNN4V4VV3BR425SEHCLQP4TQXANCNFSM4E6V7AQQ .