neupy.layers.BatchNorm

class neupy.layers.BatchNorm[source]

Batch-normalization layer.

Parameters:
axes : int, tuple with int or None

The axis or axes along which normalization is applied. None means that normalization will be applied over all axes except the first one. In case of 4D tensor it will be equal to (0, 1, 2). Defaults to None.

epsilon : float

Epsilon is a positive constant that adds to the standard deviation to prevent the division by zero. Defaults to 1e-5.

alpha : float

Coefficient for the exponential moving average of batch-wise means and standard deviations computed during training; the closer to one, the more it will depend on the last batches seen. Value needs to be between 0 and 1. Defaults to 0.1.

gamma : array-like, Tensorfow variable, scalar or Initializer

Default initialization methods you can find here. Defaults to Constant(value=1).

beta : array-like, Tensorfow variable, scalar or Initializer

Default initialization methods you can find here. Defaults to Constant(value=0).

running_mean : array-like, Tensorfow variable, scalar or Initializer

Default initialization methods you can find here. Defaults to Constant(value=0).

running_inv_std : array-like, Tensorfow variable, scalar or Initializer

Default initialization methods you can find here. Defaults to Constant(value=1).

name : str or None

Layer’s identifier. If name is equal to None than name will be generated automatically. Defaults to None.

References

[1]Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, http://arxiv.org/pdf/1502.03167v3.pdf
Attributes:
input_shape : tuple

Returns layer’s input shape in the form of a tuple. Shape will not include batch size dimension.

output_shape : tuple

Returns layer’s output shape in the form of a tuple. Shape will not include batch size dimension.

training_state : bool

Defines whether layer in training state or not. Training state will enable some operations inside of the layers that won’t work otherwise.

parameters : dict

Parameters that networks uses during propagation. It might include trainable and non-trainable parameters.

graph : LayerGraph instance

Graphs that stores all relations between layers.

Methods

disable_training_state() Context manager that switches off trainig state.
initialize() Set up important configurations related to the layer.
alpha = None[source]
axes = None[source]
beta = None[source]
epsilon = None[source]
gamma = None[source]
initialize()[source]

Initialize connection

options = {'alpha': Option(class_name='BatchNorm', value=ProperFractionProperty(name="alpha")), 'axes': Option(class_name='BatchNorm', value=AxesProperty(name="axes")), 'beta': Option(class_name='BatchNorm', value=ParameterProperty(name="beta")), 'epsilon': Option(class_name='BatchNorm', value=NumberProperty(name="epsilon")), 'gamma': Option(class_name='BatchNorm', value=ParameterProperty(name="gamma")), 'name': Option(class_name='BaseLayer', value=Property(name="name")), 'running_inv_std': Option(class_name='BatchNorm', value=ParameterProperty(name="running_inv_std")), 'running_mean': Option(class_name='BatchNorm', value=ParameterProperty(name="running_mean"))}[source]
output(input_value)[source]

Return output base on the input value.

Parameters:
input_value
running_inv_std = None[source]
running_mean = None[source]