neupy.layers.LSTM

class neupy.layers.LSTM[source]

Long Short Term Memory (LSTM) Layer.

Parameters:
n_units : int

Number of hidden units in the layer.

only_return_final : bool

If True, only return the final sequential output (e.g. for tasks where a single target value for the entire sequence is desired). In this case, Tensorfow makes an optimization which saves memory. Defaults to True.

input_weights : Initializer, ndarray

Weight parameters for input connection. Defaults to HeNormal().

hidden_weights : Initializer, ndarray

Weight parameters for hidden connection. Defaults to HeNormal().

cell_weights : Initializer, ndarray

Weight parameters for cell connection. Require only when peepholes=True otherwise it will be ignored. Defaults to HeNormal().

bias : Initializer, ndarray

Bias parameters for all gates. Defaults to Constant(0).

ingate : function

Activation function for the input gate. Defaults to tf.nn.sigmoid.

forgetgate : function

Activation function for the forget gate. Defaults to tf.nn.sigmoid.

outgate : function

Activation function for the output gate. Defaults to tf.nn.sigmoid.

cell : function

Activation function for the cell. Defaults to tf.tanh.

learn_init : bool

If True, make cell_init and hidden_init trainable variables. Defaults to False.

cell_init : array-like, Tensorfow variable, scalar or Initializer

Initializer for initial cell state (\(c_0\)). Defaults to Constant(0).

hidden_init : array-like, Tensorfow variable, scalar or Initializer

Initializer for initial hidden state (\(h_0\)). Defaults to Constant(0).

backwards : bool

If True, process the sequence backwards and then reverse the output again such that the output from the layer is always from \(x_1\) to \(x_n\). Defaults to False

peepholes : bool

If True, the LSTM uses peephole connections. When False, cell parameters are ignored. Defaults to False.

unroll_scan : bool

If True the recursion is unrolled instead of using scan. For some graphs this gives a significant speed up but it might also consume more memory. When unroll_scan=True, backpropagation always includes the full sequence, so n_gradient_steps must be set to -1 and the input sequence length must be known at compile time (i.e., cannot be given as None). Defaults to False.

gradient_clipping : float or int

If nonzero, the gradient messages are clipped to the given value during the backward pass. Defaults to 0.

name : str or None

Layer’s name. Can be used as a reference to specific layer. When value specified as None than name will be generated from the class name. Defaults to None

Notes

Code was adapted from the Lasagne library.

Examples

Sequence classification

>>> from neupy.layers import *
>>>
>>> n_time_steps = 40
>>> n_categories = 20
>>> embedded_size = 10
>>>
>>> network = join(
...     Input(n_time_steps),
...     Embedding(n_categories, embedded_size),
...     LSTM(20),
...     Sigmoid(1),
... )
>>> network
(?, 40) -> [... 4 layers ...] -> (?, 1)
backwards = None[source]
biases = None[source]
cell = None[source]
cell_init = None[source]
cell_weights = None[source]
create_variables(input_shape)[source]
forgetgate = None[source]
gradient_clipping = None[source]
hidden_init = None[source]
hidden_weights = None[source]
ingate = None[source]
input_weights = None[source]
learn_init = None[source]
options = {'backwards': Option(class_name='LSTM', value=Property(name="backwards")), 'biases': Option(class_name='LSTM', value=ParameterProperty(name="biases")), 'cell': Option(class_name='LSTM', value=Property(name="cell")), 'cell_init': Option(class_name='LSTM', value=ParameterProperty(name="cell_init")), 'cell_weights': Option(class_name='LSTM', value=ParameterProperty(name="cell_weights")), 'forgetgate': Option(class_name='LSTM', value=Property(name="forgetgate")), 'gradient_clipping': Option(class_name='LSTM', value=NumberProperty(name="gradient_clipping")), 'hidden_init': Option(class_name='LSTM', value=ParameterProperty(name="hidden_init")), 'hidden_weights': Option(class_name='LSTM', value=ParameterProperty(name="hidden_weights")), 'ingate': Option(class_name='LSTM', value=Property(name="ingate")), 'input_weights': Option(class_name='LSTM', value=ParameterProperty(name="input_weights")), 'learn_init': Option(class_name='LSTM', value=Property(name="learn_init")), 'n_units': Option(class_name='BaseRNNLayer', value=IntProperty(name="n_units")), 'name': Option(class_name='BaseLayer', value=Property(name="name")), 'only_return_final': Option(class_name='BaseRNNLayer', value=Property(name="only_return_final")), 'outgate': Option(class_name='LSTM', value=Property(name="outgate")), 'peepholes': Option(class_name='LSTM', value=Property(name="peepholes")), 'unroll_scan': Option(class_name='LSTM', value=Property(name="unroll_scan"))}[source]
outgate = None[source]
output(input, **kwargs)[source]
peepholes = None[source]
unroll_scan = None[source]