neupy.layers.GRU

class neupy.layers.GRU[source]

Gated Recurrent Unit (GRU) Layer.

Parameters:
n_units : int

Number of hidden units in the layer.

only_return_final : bool

If True, only return the final sequential output (e.g. for tasks where a single target value for the entire sequence is desired). In this case, Tensorfow makes an optimization which saves memory. Defaults to True.

input_weights : Initializer, ndarray

Weight parameters for input connection. Defaults to HeNormal().

hidden_weights : Initializer, ndarray

Weight parameters for hidden connection. Defaults to HeNormal().

biases : Initializer, ndarray

Bias parameters for all gates. Defaults to Constant(0).

resetgate : function

Activation function for the reset gate. Defaults to tf.nn.sigmoid.

updategate : function

Activation function for the update gate. Defaults to tf.nn.sigmoid.

hidden_update : function

Activation function for the hidden state update. Defaults to tf.tanh.

learn_init : bool

If True, make hidden_init trainable variable. Defaults to False.

hidden_init : array-like, Tensorfow variable, scalar or Initializer

Initializer for initial hidden state (\(h_0\)). Defaults to Constant(0).

backwards : bool

If True, process the sequence backwards and then reverse the output again such that the output from the layer is always from \(x_1\) to \(x_n\). Defaults to False.

unroll_scan : bool

If True the recursion is unrolled instead of using scan. For some graphs this gives a significant speed up but it might also consume more memory. When unroll_scan=True, backpropagation always includes the full sequence, so n_gradient_steps must be set to -1 and the input sequence length must be known at compile time (i.e., cannot be given as None). Defaults to False.

name : str or None

Layer’s name. Can be used as a reference to specific layer. When value specified as None than name will be generated from the class name. Defaults to None

Notes

Code was adapted from the Lasagne library.

Examples

Sequence classification

>>> from neupy.layers import *
>>>
>>> n_time_steps = 40
>>> n_categories = 20
>>> embedded_size = 10
>>>
>>> network = join(
...     Input(n_time_steps),
...     Embedding(n_categories, embedded_size),
...     GRU(20),
...     Sigmoid(1),
... )
>>> network
(?, 40) -> [... 4 layers ...] -> (?, 1)
backwards = None[source]
biases = None[source]
create_variables(input_shape)[source]
gradient_clipping = None[source]
hidden_init = None[source]
hidden_update = None[source]
hidden_weights = None[source]
input_weights = None[source]
learn_init = None[source]
options = {'backwards': Option(class_name='GRU', value=Property(name="backwards")), 'biases': Option(class_name='GRU', value=ParameterProperty(name="biases")), 'gradient_clipping': Option(class_name='GRU', value=NumberProperty(name="gradient_clipping")), 'hidden_init': Option(class_name='GRU', value=ParameterProperty(name="hidden_init")), 'hidden_update': Option(class_name='GRU', value=Property(name="hidden_update")), 'hidden_weights': Option(class_name='GRU', value=ParameterProperty(name="hidden_weights")), 'input_weights': Option(class_name='GRU', value=ParameterProperty(name="input_weights")), 'learn_init': Option(class_name='GRU', value=Property(name="learn_init")), 'n_units': Option(class_name='BaseRNNLayer', value=IntProperty(name="n_units")), 'name': Option(class_name='BaseLayer', value=Property(name="name")), 'only_return_final': Option(class_name='BaseRNNLayer', value=Property(name="only_return_final")), 'resetgate': Option(class_name='GRU', value=Property(name="resetgate")), 'unroll_scan': Option(class_name='GRU', value=Property(name="unroll_scan")), 'updategate': Option(class_name='GRU', value=Property(name="updategate"))}[source]
output(input, **kwargs)[source]
resetgate = None[source]
unroll_scan = None[source]
updategate = None[source]