neupy.layers.GRU

class neupy.layers.GRU[source]

Gated Recurrent Unit (GRU) Layer.

Parameters:
size : int

Number of hidden units in the layer.

input_weights : Initializer, ndarray

Weight parameters for input connection. Defaults to HeNormal().

hidden_weights : Initializer, ndarray

Weight parameters for hidden connection. Defaults to HeNormal().

bias : Initializer, ndarray

Bias parameters for all gates. Defaults to Constant(0).

activation_functions : dict, callable

Activation functions for different gates. Defaults to:

# import tensorflow as tf
dict(
    resetgate=tf.nn.sigmoid,
    updategate=tf.nn.sigmoid,
    hidden_update=tf.tanh,
)

If application requires modification to only one parameter then it’s better to specify the one that you need to modify and ignore other parameters

dict(resetgate=tf.tanh)

Other parameters like updategate or hidden_update will be equal to their default values.

learn_init : bool

If True, make hidden_init trainable variable. Defaults to False.

hidden_init : array-like, Tensorfow variable, scalar or Initializer

Initializer for initial hidden state (\(h_0\)). Defaults to Constant(0).

only_return_final : bool

If True, only return the final sequential output (e.g. for tasks where a single target value for the entire sequence is desired). In this case, Tensorfow makes an optimization which saves memory. Defaults to True.

backwards : bool

If True, process the sequence backwards and then reverse the output again such that the output from the layer is always from \(x_1\) to \(x_n\). Defaults to False.

unroll_scan : bool

If True the recursion is unrolled instead of using scan. For some graphs this gives a significant speed up but it might also consume more memory. When unroll_scan=True, backpropagation always includes the full sequence, so n_gradient_steps must be set to -1 and the input sequence length must be known at compile time (i.e., cannot be given as None). Defaults to False.

name : str or None

Layer’s identifier. If name is equal to None than name will be generated automatically. Defaults to None.

Notes

Code was adapted from the Lasagne library.

Examples

Sequence classification

from neupy import layers, algorithms

n_time_steps = 40
n_categories = 20
embedded_size = 10

network = algorithms.RMSProp(
    [
        layers.Input(n_time_steps),
        layers.Embedding(n_categories, embedded_size),
        layers.GRU(20),
        layers.Sigmoid(1),
    ]
)
activation_functions = None[source]
backwards = None[source]
biases = None[source]
gradient_clipping = None[source]
hidden_init = None[source]
hidden_weights = None[source]
initialize()[source]

Initialize connection

input_weights = None[source]
learn_init = None[source]
options = {'activation_functions': Option(class_name='GRU', value=MultiCallableProperty(name="activation_functions")), 'backwards': Option(class_name='GRU', value=Property(name="backwards")), 'biases': Option(class_name='GRU', value=ParameterProperty(name="biases")), 'gradient_clipping': Option(class_name='GRU', value=NumberProperty(name="gradient_clipping")), 'hidden_init': Option(class_name='GRU', value=ParameterProperty(name="hidden_init")), 'hidden_weights': Option(class_name='GRU', value=ParameterProperty(name="hidden_weights")), 'input_weights': Option(class_name='GRU', value=ParameterProperty(name="input_weights")), 'learn_init': Option(class_name='GRU', value=Property(name="learn_init")), 'name': Option(class_name='BaseLayer', value=Property(name="name")), 'only_return_final': Option(class_name='BaseRNNLayer', value=Property(name="only_return_final")), 'size': Option(class_name='BaseRNNLayer', value=IntProperty(name="size")), 'unroll_scan': Option(class_name='GRU', value=Property(name="unroll_scan"))}[source]
output(input_value)[source]

Return output base on the input value.

Parameters:
input_value
unroll_scan = None[source]