# neupy.algorithms.step_decay

neupy.algorithms.step_decay(initial_value, reduction_freq, start_iter=0, name='step')[source]

Algorithm minimizes learning step monotonically after each iteration.

$\alpha_{t + 1} = \frac{\alpha_{0}}{1 + \frac{t}{m}}$

where $$\alpha$$ is a step, $$t$$ is an iteration number and $$m$$ is a reduction_freq parameter.

step = initial_value / (1 + current_iteration / reduction_freq)

Parameters: initial_value : float Initial value for the learning rate. It’s the learning rate returned during the first iteration. reduction_freq : int Parameter controls step reduction frequency. The larger the value the slower step parameter decreases. For instance, if reduction_freq=100 and step=0.12 then after 100 iterations step is going to be equal to 0.06 (which is 0.12 / 2), after 200 iterations step is going to be equal to 0.04 (which is 0.12 / 3) and so on. start_iter : int Start iteration. At has to be equal to 0 when network just started the training. Defaults to 0. name : str Learning rate’s variable name. Defaults to step.

Notes

Step will be reduced faster when you have smaller training batches.

Examples

>>> from neupy import algorithms
>>> from neupy.layers import *
>>>
>>> optimizer = algorithms.Momentum(
...     Input(5) >> Relu(10) >> Sigmoid(1),
...     step=algorithms.step_decay(
...         initial_value=0.1,
...         reduction_freq=100,
...     )
... )