Keras RNN Import Error

Issue Description

Trying to import a Keras LSTM model from a ByteArrayInputStream of an hdf5, but it fails with Unsupported keras layer type RNN. Please file an issue at https://github.com/eclipse/deeplearning4j/issues.

The reason I’m importing from an InputStream and not from a file is because I will be getting the model hdf5 from a remote location in production (a database), so I cannot load it from a file.

(Sample) Code:

import org.deeplearning4j.nn.multilayer.MultiLayerNetwork;
import org.nd4j.linalg.api.ndarray.INDArray;
import org.nd4j.linalg.factory.Nd4j;
import org.deeplearning4j.nn.modelimport.keras.*;
import org.deeplearning4j.nn.modelimport.keras.utils.*;
import org.deeplearning4j.util.ModelSerializer;
import org.deeplearning4j.nn.graph.ComputationGraph;
import org.deeplearning4j.nn.modelimport.keras.exceptions.*;

import java.io.*;
import java.io.FileInputStream;
import java.io.InputStream;
import java.io.ByteArrayInputStream;
import java.nio.file.*;
public class KerasLoader{

    public static void main(String[] args) throws IOException,UnsupportedKerasConfigurationException, InvalidKerasConfigurationException{

        Path path = FileSystems.getDefault().getPath("LSTM.h5");
        byte[] modelByteArray = Files.readAllBytes(path);

        InputStream modelByteStream = new ByteArrayInputStream(modelByteArray);
        MultiLayerNetwork model = KerasModelImport.importKerasSequentialModelAndWeights(modelByteStream);
        System.out.println("done");
    }
}

Version Information

• Deeplearning4j version 1.0.0-beta7
• Platform information Ubuntu 18.04.4

Additional Information

Where applicable, please also provide:

• Full log or exception stack trace (ideally in a Gist: gist.github.com)

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Exception in thread "main" org.deeplearning4j.nn.modelimport.keras.exceptions.UnsupportedKerasConfigurationException: Unsupported keras layer type RNN. Please file an issue at https://github.com/eclipse/deeplearning4j/issues.
	at org.deeplearning4j.nn.modelimport.keras.utils.KerasLayerUtils.getKerasLayerFromConfig(KerasLayerUtils.java:334)
	at org.deeplearning4j.nn.modelimport.keras.KerasModel.prepareLayers(KerasModel.java:218)
	at org.deeplearning4j.nn.modelimport.keras.KerasSequentialModel.<init>(KerasSequentialModel.java:110)
	at org.deeplearning4j.nn.modelimport.keras.KerasSequentialModel.<init>(KerasSequentialModel.java:57)
	at org.deeplearning4j.nn.modelimport.keras.utils.KerasModelBuilder.buildSequential(KerasModelBuilder.java:322)
	at org.deeplearning4j.nn.modelimport.keras.KerasModelImport.importKerasSequentialModelAndWeights(KerasModelImport.java:223)
	at org.deeplearning4j.nn.modelimport.keras.KerasModelImport.importKerasSequentialModelAndWeights(KerasModelImport.java:113)
	at KerasLoader.main(KerasLoader.java:24)

• pom.xml file or similar (also in a Gist)

    <properties>
        <scala.binary.version>2.11</scala.binary.version>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
     </properties>
<dependencies>
<dependency>
   <groupId>org.nd4j</groupId>
   <artifactId>nd4j-native</artifactId>
   <version>1.0.0-beta7</version>
   <classifier>linux-x86_64-avx2</classifier>
 </dependency>
 <dependency>
    <groupId>org.deeplearning4j</groupId>
    <artifactId>deeplearning4j-modelimport</artifactId>
    <version>1.0.0-beta7</version>
</dependency>
 <dependency>
    <groupId>org.deeplearning4j</groupId>
    <artifactId>deeplearning4j-core</artifactId>
    <version>1.0.0-beta7</version>
</dependency>
</dependencies>

Update I tried to read with JSON and H5 just to be sure that wasn’t the problem (even though this solution wouldn’t work for my situation). MultiLayerNetwork model = KerasModelImport.importKerasSequentialModelAndWeights("./LSTM.json","./LSTM.h5");

But I received the same error.

Update I noticed that this isn’t a problem for LSTM layers, only RNN layers so I changed the title

[treo]

tf.keras.layers.RNN(tf.keras.layers.LSTMCell(units), input_shape=(None, input_dim))

You should replace that with tf.keras.layers.LSTM(units) instead.

Is there a reason why the standard RNN layer isn’t supported?

It isn’t the standard in a sense, the docs (https://keras.io/guides/working_with_rnns/) say:

Mathematically, RNN(LSTMCell(10)) produces the same result as LSTM(10). In fact, the implementation of this layer in TF v1.x was just creating the corresponding RNN cell and wrapping it in a RNN layer. However using the built-in GRU and LSTM layers enables the use of CuDNN and you may see better performance.

And https://keras.io/api/layers/ doesn’t even mention the RNN(Cell) structure at all.

For this reason, when we import Keras models, we support importing the layers, but not the composites that are based on RNN(Cell).

[Ben-Epstein]

Got it, thanks for the clarification!