Cannot get cross-entropy to work

Hi,

I’m playing around with this code for a research project and everything works fine with mean-squared-error, however as soon as I switch to cross-entropy (which I really want), it does not converge and loss gets bigger over time… I tried numerous parameters but nothing seems to work. I’m using MNIST with the following model.

model = StackedAutoEncoder(
    dims=[100],
    activations=['softmax'], 
    noise='gaussian', 
    epoch=[1000],
    loss='cross-entropy',
    lr=0.005,
    batch_size=100,
    print_step=100
)

Do you know why this isn’t working?

Thanks!

[elggem]

I can confirm that using a linear layer and scaled data, global loss decreases over time as expected! On visualization of the filters I still get essentially random noise (rmse shows nice detectors) and loss gets stuck at around 5.0, but that must be a problem with either the visualization or parameters. I am aiming for filters like the ones in the paper you cite in Readme (Vincent et al. 2010) on page 3390.

For now I consider this issue as closed, thank you for the pointer in the right direction, it is much appreciated! 👍

[rajarsheem]

@elggem: @Nilabhra has pushed a commit . Hope it solves your issue. Get back to us in case of any help.