# neupy.layers.GroupNorm

class neupy.layers.GroupNorm[source]

Group Normalization layer. This layer is a simple alternative to the Batch Normalization layer for cases when batch size is small.

Parameters: n_groups : int During normalization all the channels will be break down into separate groups and mean and variance will be estimated per group. This parameter controls number of groups. gamma : array-like, Tensorfow variable, scalar or Initializer Scale. Default initialization methods you can find here. Defaults to Constant(value=1). beta : array-like, Tensorfow variable, scalar or Initializer Offset. Default initialization methods you can find here. Defaults to Constant(value=0). epsilon : float Epsilon ensures that input rescaling procedure that uses estimated variance will never cause division by zero. Defaults to 1e-5. name : str or None Layer’s name. Can be used as a reference to specific layer. Name Can be specified as: String: Specified name will be used as a direct reference to the layer. For example, name=”fc” Format string: Name pattern could be defined as a format string and specified field will be replaced with an index. For example, name=”fc{}” will be replaced with fc1, fc2 and so on. A bit more complex formatting methods are acceptable, for example, name=”fc-{:<03d}” will be converted to fc-001, fc-002, fc-003 and so on. None: When value specified as None than name will be generated from the class name. Defaults to None.

References

 [1] Group Normalization, Yuxin Wu, Kaiming He, https://arxiv.org/pdf/1803.08494.pdf

Examples

Convolutional Neural Networks (CNN)

>>> from neupy.layers import *
>>> network = join(
...     Input((28, 28, 1)),
...     Convolution((3, 3, 16)) >> GroupNorm(4) >> Relu(),
...     Convolution((3, 3, 16)) >> GroupNorm(4) >> Relu(),
...     Reshape(),
...     Softmax(10),
... )

Attributes: variables : dict Variable names and their values. Dictionary can be empty in case if variables hasn’t been created yet.

Methods

 variable(value, name, shape=None, trainable=True) Initializes variable with specified values. get_output_shape(input_shape) Computes expected output shape from the layer based on the specified input shape. output(*inputs, **kwargs) Propagates input through the layer. The kwargs variable might contain additional information that propagates through the network.
beta = None[source]
create_variables(input_shape)[source]
epsilon = None[source]
gamma = None[source]
get_output_shape(input_shape)[source]
n_groups = None[source]
options = {'beta': Option(class_name='GroupNorm', value=ParameterProperty(name="beta")), 'epsilon': Option(class_name='GroupNorm', value=NumberProperty(name="epsilon")), 'gamma': Option(class_name='GroupNorm', value=ParameterProperty(name="gamma")), 'n_groups': Option(class_name='GroupNorm', value=IntProperty(name="n_groups")), 'name': Option(class_name='BaseLayer', value=Property(name="name"))}[source]
output(input)[source]