Loading models with 2 layers sharing weights fails

If model has shared weights and 2 layers calling each other with shared weights, it introduces a cyclic dependency.

z = keras.layers.Input(shape=(2,))
A = keras.layers.Dense(2)
B = keras.layers.Dense(2)
output = A(B(A(B(z))))
M = keras.models.Model(input=z, output=output)
M.save("test.hd5")
keras.models.load_model("test.hd5")

Then keras/engine/topology.py:2362 assert inbound_layer_name in created_layers, 'Missing layer: %s' % inbound_layer_name throws

AssertionError: Missing layer: dense_1

Is there a simple fix? Worth noting that M.load_weights(“test.hd5”) does work, but unfortunately I need load_model to work

[kudkudak]

Here is a solution I hacked for that. Fix is changing keras.engine.topology.Container.from_config to step iteratively through graph. Each run of step_through_graph (through whole network) will call at least one layer so after all_calls it should have called all layers. I am pretty sure there is a nicer solution, but that was fast to code 😃 I think each application of the layer should be stored as a separate entry in config, then from_config could be left almost unchanged, but that is hard to introduce as it seems to be a breaking change.

def from_config(cls, config, custom_objects={}):
    '''Instantiates a Model from its config (output of `get_config()`).
    TODO: support for custom objects

        TODO: shouldnt modify config in place
    '''
    from keras.utils.layer_utils import layer_from_config

    # layer instances created during
    # the graph reconstruction process
    created_layers = {}

    def step_through_graph(layer_data):
        # instantiate layer
        layer_name = layer_data['name']
        if layer_name not in created_layers:
            layer = layer_from_config(layer_data, custom_objects=custom_objects)
            created_layers[layer_name] = layer
        else:
            layer = created_layers[layer_name]

        # already processed?
        inbound_nodes_data = layer_data['inbound_nodes']
        if len(inbound_nodes_data) == 0:
            return True

        # try to do single step
        node_data = inbound_nodes_data[0]
        input_tensors = []
        for input_data in node_data:
            inbound_layer_name, inbound_node_index, inbound_tensor_index = input_data

            if inbound_layer_name not in created_layers \
                    or len(created_layers[inbound_layer_name].inbound_nodes) <= inbound_node_index:
                return False
            else:
                inbound_layer = created_layers[inbound_layer_name]
                inbound_node = inbound_layer.inbound_nodes[inbound_node_index]
                input_tensors.append(inbound_node.output_tensors[inbound_tensor_index])

        # call layer on its inputs, thus creating the node
        # and building the layer if needed
        if input_tensors:
            if len(input_tensors) == 1:
                layer(input_tensors[0])
            else:
                layer(input_tensors)

        # Ok, all good, pop it
        inbound_nodes_data.pop(0)
        return True


    # Just try iteratively, poping shared copies from the stack
    all_calls = sum(map(lambda layer: len(layer['inbound_nodes']), config['layers']))
    for i in range(all_calls): # Each step is guaranteed to make a progress
        for layer_data in config['layers']:
            result = step_through_graph(layer_data)
    assert result == True, "Each run of step_through_graph should progress at least once"

    name = config.get('name')
    input_tensors = []
    output_tensors = []
    for layer_data in config['input_layers']:
        layer_name, node_index, tensor_index = layer_data
        assert layer_name in created_layers
        layer = created_layers[layer_name]
        layer_output_tensors = layer.inbound_nodes[node_index].output_tensors
        input_tensors.append(layer_output_tensors[tensor_index])

    for layer_data in config['output_layers']:
        layer_name, node_index, tensor_index = layer_data
        assert layer_name in created_layers
        layer = created_layers[layer_name]
        layer_output_tensors = layer.inbound_nodes[node_index].output_tensors
        output_tensors.append(layer_output_tensors[tensor_index])

    return cls(input=input_tensors, output=output_tensors, name=name)

[fchollet]

@kudkudak note that you should have sent a PR rather than leaving a code fix in an issues thread… Feel free to check out my fix and compare to yours. I’m sure mine can be improve.