neupy.layers.GRU

class neupy.layers.GRU[source]

Gated Recurrent Unit (GRU) Layer.

Parameters:
n_units : int

Number of hidden units in the layer.

only_return_final : bool

If True, only return the final sequential output (e.g. for tasks where a single target value for the entire sequence is desired). In this case, Tensorfow makes an optimization which saves memory. Defaults to True.

input_weights : Initializer, ndarray

Weight parameters for input connection. Defaults to HeNormal().

hidden_weights : Initializer, ndarray

Weight parameters for hidden connection. Defaults to HeNormal().

biases : Initializer, ndarray

Bias parameters for all gates. Defaults to Constant(0).

resetgate : function

Activation function for the reset gate. Defaults to tf.nn.sigmoid.

updategate : function

Activation function for the update gate. Defaults to tf.nn.sigmoid.

hidden_update : function

Activation function for the hidden state update. Defaults to tf.tanh.

learn_init : bool

If True, make hidden_init trainable variable. Defaults to False.

hidden_init : array-like, Tensorfow variable, scalar or Initializer

Initializer for initial hidden state (\(h_0\)). Defaults to Constant(0).

backwards : bool

If True, process the sequence backwards and then reverse the output again such that the output from the layer is always from \(x_1\) to \(x_n\). Defaults to False.

unroll_scan : bool

If True the recursion is unrolled instead of using scan. For some graphs this gives a significant speed up but it might also consume more memory. When unroll_scan=True, backpropagation always includes the full sequence, so n_gradient_steps must be set to -1 and the input sequence length must be known at compile time (i.e., cannot be given as None). Defaults to False.

name : str or None

Layer’s name. Can be used as a reference to specific layer. Name Can be specified as:

  • String: Specified name will be used as a direct reference to the layer. For example, name=”fc”
  • Format string: Name pattern could be defined as a format string and specified field will be replaced with an index. For example, name=”fc{}” will be replaced with fc1, fc2 and so on. A bit more complex formatting methods are acceptable, for example, name=”fc-{:<03d}” will be converted to fc-001, fc-002, fc-003 and so on.
  • None: When value specified as None than name will be generated from the class name.

Defaults to None.

Notes

Code was adapted from the Lasagne library.

Examples

Sequence classification

>>> from neupy.layers import *
>>>
>>> n_time_steps = 40
>>> n_categories = 20
>>> embedded_size = 10
>>>
>>> network = join(
...     Input(n_time_steps),
...     Embedding(n_categories, embedded_size),
...     GRU(20),
...     Sigmoid(1),
... )
>>> network
(?, 40) -> [... 4 layers ...] -> (?, 1)
backwards = None[source]
biases = None[source]
create_variables(input_shape)[source]
gradient_clipping = None[source]
hidden_init = None[source]
hidden_update = None[source]
hidden_weights = None[source]
input_weights = None[source]
learn_init = None[source]
options = {'backwards': Option(class_name='GRU', value=Property(name="backwards")), 'biases': Option(class_name='GRU', value=ParameterProperty(name="biases")), 'gradient_clipping': Option(class_name='GRU', value=NumberProperty(name="gradient_clipping")), 'hidden_init': Option(class_name='GRU', value=ParameterProperty(name="hidden_init")), 'hidden_update': Option(class_name='GRU', value=Property(name="hidden_update")), 'hidden_weights': Option(class_name='GRU', value=ParameterProperty(name="hidden_weights")), 'input_weights': Option(class_name='GRU', value=ParameterProperty(name="input_weights")), 'learn_init': Option(class_name='GRU', value=Property(name="learn_init")), 'n_units': Option(class_name='BaseRNNLayer', value=IntProperty(name="n_units")), 'name': Option(class_name='BaseLayer', value=Property(name="name")), 'only_return_final': Option(class_name='BaseRNNLayer', value=Property(name="only_return_final")), 'resetgate': Option(class_name='GRU', value=Property(name="resetgate")), 'unroll_scan': Option(class_name='GRU', value=Property(name="unroll_scan")), 'updategate': Option(class_name='GRU', value=Property(name="updategate"))}[source]
output(input, **kwargs)[source]
resetgate = None[source]
unroll_scan = None[source]
updategate = None[source]