How does Adagrad work in keras?

Hi all,

I post a question about keras’ optimizer in stackoverflow. Can anybody help me to answer that?

The function get_update() in Adagrad seems one step update. However should the accumulators be stored the history information? Why it has been initialized to zeros at each step? How it can be an accumulator through the whole training process?

What does this line do? self.weights = accumulators It seems self.weights is never been called anymore.

Thanks.

[bstriner]

You should do some reading on how theano and tensorflow build graphs. get_update builds a graph that can be used to get updates, it doesn’t actually perform the updates. Those updates are then run by the training function.

It is a weird way to start thinking about things. Python code builds the graph and then tells the GPU each time it needs to be run.

The accumulator is created locally by get_updates and initialized to zeros. get_updates is called once. Each time the updates returned from get_updates are run, they update that same local variable.

Cheers

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs, but feel free to re-open it if needed.