Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Output Layer Type

See original GitHub issue

I’m working on implementing a beam decoder for this, and just realized that the output values do not appear to be posteriors. In model.py I see the output layer is a Linear layer. Why not a Softmax or LogSoftmax activation? I suppose such a layer is not strictly necessary when doing a Greedy decode (and less efficient), but will be necessary for more complicated decoders. Just wondering if there’s a specific reason or if there’s something I’m missing.

Thanks!

Issue Analytics

State:
Created 6 years ago
Comments:9 (5 by maintainers)

Top GitHub Comments

1reaction

ryanlearycommented, May 23, 2017

Agreed. This can be closed I think – perhaps adding a note to the README/wiki in the future just to highlight that there is no softmax activation on the output layer would be prudent.

0reactions

SeanNarencommented, May 26, 2017

Added disclaimer, thanks Ryan!

Top Results From Across the Web

List of Deep Learning Layers - MATLAB & Simulink - MathWorks

Deep Learning Layers · Input Layers · Convolution and Fully Connected Layers · Sequence Layers · Activation Layers · Normalization Layers · Utility...

Configuring a Neural Network Output Layer | Enthought, Inc.

If your data has a target that resides in a single vector, the number of output nodes in your neural network will be...

What is an Output Layer? - Definition from Techopedia

A typical traditional neural network has three types of layers: one or more input layers, one or more hidden layers, and one or...

Four Common Types of Neural Network Layers

The four most common types of neural network layers are Fully connected, Convolution, Deconvolution, and Recurrent, and below you will find what ...

The output layer - Deep Learning: Getting Started - LinkedIn

- The output layer is the final layer in the neural network where desired predictions are obtained. There is one output layer in...