Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

DDPG example fails adding BatchNormalization with Tensorflow

See original GitHub issue

If I take the DDPG example and add a BatchNormalization layer to either the actor or critic model, I get a TypeError with Tensorflow (but not Theano):

Using TensorFlow backend.
Traceback (most recent call last):
  File "", line 27, in <module>
    actor.add(BatchNormalization())         # Fails with Tensorflow
  File "/python/lib/python3.4/site-packages/keras/", line 332, in add
    output_tensor = layer(self.outputs[0])
  File "/python/lib/python3.4/site-packages/keras/engine/", line 572, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
  File "/python/lib/python3.4/site-packages/keras/engine/", line 635, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
  File "/python/lib/python3.4/site-packages/keras/engine/", line 166, in create_node
    output_tensors = to_list([0], mask=input_masks[0]))
  File "/python/lib/python3.4/site-packages/keras/layers/", line 133, in call
    broadcast_running_mean = K.reshape(self.running_mean, broadcast_shape)
  File "/python/lib/python3.4/site-packages/keras/backend/", line 1463, in reshape
    return tf.reshape(x, shape)
  File "/python/lib/python3.4/site-packages/tensorflow/python/ops/", line 2448, in reshape
  File "/python/lib/python3.4/site-packages/tensorflow/python/framework/", line 493, in apply_op
    raise err
  File "/python/lib/python3.4/site-packages/tensorflow/python/framework/", line 490, in apply_op
  File "/python/lib/python3.4/site-packages/tensorflow/python/framework/", line 669, in convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/python/lib/python3.4/site-packages/tensorflow/python/framework/", line 176, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/python/lib/python3.4/site-packages/tensorflow/python/framework/", line 165, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/python/lib/python3.4/site-packages/tensorflow/python/framework/", line 441, in make_tensor_proto
    tensor_proto.string_val.extend([compat.as_bytes(x) for x in proto_values])
  File "/python/lib/python3.4/site-packages/tensorflow/python/framework/", line 441, in <listcomp>
    tensor_proto.string_val.extend([compat.as_bytes(x) for x in proto_values])
  File "/python/lib/python3.4/site-packages/tensorflow/python/util/", line 65, in as_bytes
TypeError: Expected binary or unicode string, got 1

Code to reproduce it (only two lines have been added):

import numpy as np
import gym

from keras.models import Sequential, Model
from keras.layers import Dense, Activation, Flatten, Input, merge, BatchNormalization
from keras.optimizers import Adam

from rl.agents import DDPGAgent
from rl.memory import SequentialMemory
from rl.random import OrnsteinUhlenbeckProcess

ENV_NAME = 'Pendulum-v0'

# Get the environment and extract the number of actions.
env = gym.make(ENV_NAME)
assert len(env.action_space.shape) == 1
nb_actions = env.action_space.shape[0]

# Next, we build a very simple model.
actor = Sequential()
actor.add(Flatten(input_shape=(1,) + env.observation_space.shape))
actor.add(BatchNormalization())         # Fails with Tensorflow

action_input = Input(shape=(nb_actions,), name='action_input')
observation_input = Input(shape=(1,) + env.observation_space.shape, name='observation_input')
flattened_observation = Flatten()(observation_input)
x = merge([action_input, flattened_observation], mode='concat')
x = BatchNormalization()(x)            # Fails with Tensorflow
x = Dense(32)(x)
x = Activation('relu')(x)
x = Dense(32)(x)
x = Activation('relu')(x)
x = Dense(32)(x)
x = Activation('relu')(x)
x = Dense(1)(x)
x = Activation('linear')(x)
critic = Model(input=[action_input, observation_input], output=x)

# Finally, we configure and compile our agent. You can use every built-in Keras optimizer and
# even the metrics!
memory = SequentialMemory(limit=100000, window_length=1)
random_process = OrnsteinUhlenbeckProcess(size=nb_actions, theta=.15, mu=0., sigma=.3)
agent = DDPGAgent(nb_actions=nb_actions, actor=actor, critic=critic, critic_action_input=action_input,
                  memory=memory, nb_steps_warmup_critic=100, nb_steps_warmup_actor=100,
                  random_process=random_process, gamma=.99, target_model_update=1e-3)
agent.compile(Adam(lr=.001, clipnorm=1.), metrics=['mae'])

# Okay, now it's time to learn something! We visualize the training here for show, but this
# slows down training quite a lot. You can always safely abort the training prematurely using
# Ctrl + C., nb_steps=50000, visualize=True, verbose=1, nb_max_episode_steps=200)

# After training is done, we save the final weights.
agent.save_weights('ddpg_{}_weights.h5f'.format(ENV_NAME), overwrite=True)

# Finally, evaluate our algorithm for 5 episodes.
agent.test(env, nb_episodes=5, visualize=True, nb_max_episode_steps=200)

I’m on the latest master of everything.

$ python -V; pip freeze | egrep -i 'keras|theano'
Python 3.4.3+

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:6 (1 by maintainers)

github_iconTop GitHub Comments

ParthaEthcommented, Nov 6, 2017

Hi, Since in Keras2.0 mode option in BatchNormalization has been removed it would be great to be able to support mode=0, which is the only mode supported in Keras 2.0. Is there any plan on that?

chpohlcommented, Aug 9, 2018

I ran into a similar situation. I get a different error message, but I guess the underlying problem is the same:

tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'flatten_1_input' with dtype float and shape [?,1,17]
	 [[Node: flatten_1_input = Placeholder[dtype=DT_FLOAT, shape=[?,1,17], _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]
	 [[Node: sequential_1/activation_3/Tanh/_401 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_839_sequential_1/activation_3/Tanh", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

A temporary fix to use Keras > 2.0 is setting trainable=False for the BatchNormalization. I tried debugging this and found out that the if DDPGAgent.uses_learning_phase is true, something goes wrong once the warmup steps for the actor are completed. I followed the error to keras.backend.tensorflow_backend._call(), until where everything runs appropriately. But once tf_session.TF_SessionRunCallable() is called the error is thrown and I can’t figure out why. Hope that helps someone.

Read more comments on GitHub >

github_iconTop Results From Across the Web

DDPG and Batch Normalization in TensorFlow - Reddit
The batch norm layers are designed such that the moving average is actually only updated when it is told it is in training...
Read more >
Batch Normalization Tensorflow Keras Example | by Cory Maklin
To make the problem simpler, we will assume we have a neural network consisting of two layers, each with a single neuron. We...
Read more >
The Batch Normalization layer of Keras is broken - Datumbox
Below I describe exactly what is the problem and I sketch out the technical implementation for solving it. I also provide a few...
Read more >
Batch Normalization TensorFlow [10 Amazing Examples]
Batch normalization is the process of adding additional layers to a deep neural network to speed up and stabilize neural networks. On the...
Read more >
tf.keras.layers.BatchNormalization | TensorFlow v2.11.0
Batch normalization applies a transformation that maintains the mean output ... center, If True, add offset of beta to normalized tensor.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found