neupy.algorithms.gd.rprop module
- class neupy.algorithms.gd.rprop.RPROP[source]
Resilient backpropagation (RPROP) is an optimization algorithm for supervised learning.
RPROP algorithm takes into account only direction of the gradient and completely ignores its magnitude. Every weight values has a unique step size associated with it (by default all of the are equal to step).
The rule is following, when gradient direction changes (sign of the gradient) we decrease step size for specific weight multiplying it by decrease_factor and if sign stays the same than we increase step size for this specific weight multiplying it by increase_factor.
The step size is always bounded by minstep and maxstep.
Parameters: minstep : float
Minimum possible value for step. Defaults to 0.001.
maxstep : float
Maximum possible value for step. Defaults to 10.
increase_factor : float
Increase factor for step in case when gradient doesn’t change sign compare to previous epoch.
decrease_factor : float
Decrease factor for step in case when gradient changes sign compare to previous epoch.
connection : list, tuple or LayerConnection instance
Network’s architecture. There are a few ways to define it.
- List of layers. For instance, [Input(2), Tanh(4), Relu(1)].
- Construct layer connections. For instance, Input(2) > Tanh(4) > Relu(1).
- Tuple of integers. Each integer defines Sigmoid layer and it’s input size. For instance, value (2, 4, 1) means that network has 3 layers with 2 input units, 4 hidden units and 1 output unit.
error : str or function
Error/loss function. Defaults to mse.
- mae - Mean Absolute Error.
- mse - Mean Squared Error.
- rmse - Root Mean Squared Error.
- msle - Mean Squared Logarithmic Error.
- rmsle - Root Mean Squared Logarithmic Error.
- categorical_crossentropy - Categorical cross entropy.
- binary_crossentropy - Binary cross entropy.
- binary_hinge - Binary hinge entropy.
- categorical_hinge - Categorical hinge entropy.
- Custom function which accepts two mandatory arguments. The first one is expected value and the second one is predicted value. Example:
def custom_func(expected, predicted): return expected - predicted
step : float
Learning rate, defaults to 0.1.
show_epoch : int or str
This property controls how often the network will display information about training. There are two main syntaxes for this property.
- You can define it as a positive integer number. It defines how offen would you like to see summary output in terminal. For instance, number 100 mean that network shows summary at 100th, 200th, 300th ... epochs.
- String defines number of times you want to see output in terminal. For instance, value '2 times' mean that the network will show output twice with approximately equal period of epochs and one additional output would be after the finall epoch.
Defaults to 1.
shuffle_data : bool
If it’s True class shuffles all your training data before training your network, defaults to True.
epoch_end_signal : function
Calls this function when train epoch finishes.
train_end_signal : function
Calls this function when train process finishes.
verbose : bool
Property controls verbose output interminal. True enables informative output in the terminal and False - disable it. Defaults to False.
addons : list or None
The list of addon algortihms. None by default. If this option is not empty it will generate new class which will inherit all from this list. Support two types of addon algorithms: weight update and step update.
See also
- IRPROPPlus
- iRPROP+ algorithm.
- GradientDescent
- GradientDescent algorithm.
Notes
Algorithm doesn’t work with mini-batches.
Examples
>>> import numpy as np >>> from neupy import algorithms >>> >>> x_train = np.array([[1, 2], [3, 4]]) >>> y_train = np.array([[1], [0]]) >>> >>> rpropnet = algorithms.RPROP((2, 3, 1)) >>> rpropnet.train(x_train, y_train)
Attributes
errors (ErrorHistoryList) Contains list of training errors. This object has the same properties as list and in addition there are three additional useful methods: last, previous and normalized. train_errors (ErrorHistoryList) Alias to the errors attribute. validation_errors (ErrorHistoryList) The same as errors attribute, but it contains only validation errors. last_epoch (int) Value equals to the last trained epoch. After initialization it is equal to 0. Methods
predict(input_data) Predicts output for the specified input. train(input_train, target_train, input_test=None, target_test=None, epochs=100, epsilon=None) Train network. You can control network’s training procedure with epochs and epsilon parameters. The input_test and target_test should be presented both in case of you need to validate network’s training after each iteration. fit(*args, **kwargs) Alias to the train method. - decrease_factor = None[source]
- increase_factor = None[source]
- init_train_updates()[source]
Initialize updates that would be applied after each training epoch.
- maxstep = None[source]
- minstep = None[source]
- options = {'verbose': Option(class_name='Verbose', value=VerboseProperty(name="verbose")), 'step': Option(class_name='BaseNetwork', value=NumberProperty(name="step")), 'show_epoch': Option(class_name='BaseNetwork', value=ShowEpochProperty(name="show_epoch")), 'shuffle_data': Option(class_name='BaseNetwork', value=Property(name="shuffle_data")), 'epoch_end_signal': Option(class_name='BaseNetwork', value=Property(name="epoch_end_signal")), 'train_end_signal': Option(class_name='BaseNetwork', value=Property(name="train_end_signal")), 'error': Option(class_name='ConstructibleNetwork', value=FunctionWithOptionsProperty(name="error")), 'addons': Option(class_name='BaseGradientDescent', value=Property(name="addons")), 'minstep': Option(class_name='RPROP', value=BoundedProperty(name="minstep")), 'maxstep': Option(class_name='RPROP', value=BoundedProperty(name="maxstep")), 'increase_factor': Option(class_name='RPROP', value=BoundedProperty(name="increase_factor")), 'decrease_factor': Option(class_name='RPROP', value=ProperFractionProperty(name="decrease_factor"))}[source]
- update_prev_delta(prev_delta)[source]
- class neupy.algorithms.gd.rprop.IRPROPPlus[source]
iRPROP+ is an optimization algorithm for supervised learning. This is a variation of the RPROP algorithm.
Parameters: minstep : float
Minimum possible value for step. Defaults to 0.001.
maxstep : float
Maximum possible value for step. Defaults to 10.
increase_factor : float
Increase factor for step in case when gradient doesn’t change sign compare to previous epoch.
decrease_factor : float
Decrease factor for step in case when gradient changes sign compare to previous epoch.
addons : list or None
The list of addon algortihms. None by default. If this option is not empty it will generate new class which will inherit all from this list. Support two types of addon algorithms: weight update and step update.
connection : list, tuple or LayerConnection instance
Network’s architecture. There are a few ways to define it.
- List of layers. For instance, [Input(2), Tanh(4), Relu(1)].
- Construct layer connections. For instance, Input(2) > Tanh(4) > Relu(1).
- Tuple of integers. Each integer defines Sigmoid layer and it’s input size. For instance, value (2, 4, 1) means that network has 3 layers with 2 input units, 4 hidden units and 1 output unit.
error : str or function
Error/loss function. Defaults to mse.
- mae - Mean Absolute Error.
- mse - Mean Squared Error.
- rmse - Root Mean Squared Error.
- msle - Mean Squared Logarithmic Error.
- rmsle - Root Mean Squared Logarithmic Error.
- categorical_crossentropy - Categorical cross entropy.
- binary_crossentropy - Binary cross entropy.
- binary_hinge - Binary hinge entropy.
- categorical_hinge - Categorical hinge entropy.
- Custom function which accepts two mandatory arguments. The first one is expected value and the second one is predicted value. Example:
def custom_func(expected, predicted): return expected - predicted
show_epoch : int or str
This property controls how often the network will display information about training. There are two main syntaxes for this property.
- You can define it as a positive integer number. It defines how offen would you like to see summary output in terminal. For instance, number 100 mean that network shows summary at 100th, 200th, 300th ... epochs.
- String defines number of times you want to see output in terminal. For instance, value '2 times' mean that the network will show output twice with approximately equal period of epochs and one additional output would be after the finall epoch.
Defaults to 1.
shuffle_data : bool
If it’s True class shuffles all your training data before training your network, defaults to True.
epoch_end_signal : function
Calls this function when train epoch finishes.
train_end_signal : function
Calls this function when train process finishes.
verbose : bool
Property controls verbose output interminal. True enables informative output in the terminal and False - disable it. Defaults to False.
See also
- RPROP
- RPROP algorithm.
- GradientDescent
- GradientDescent algorithm.
Notes
Algorithm doesn’t work with mini-batches.
References
- [1] Christian Igel, Michael Huesken (2000)
- Improving the Rprop Learning Algorithm
Examples
>>> import numpy as np >>> from neupy import algorithms >>> >>> x_train = np.array([[1, 2], [3, 4]]) >>> y_train = np.array([[1], [0]]) >>> >>> rpropnet = algorithms.IRPROPPlus((2, 3, 1)) >>> rpropnet.train(x_train, y_train)
Methods
predict(input_data) Predicts output for the specified input. train(input_train, target_train, input_test=None, target_test=None, epochs=100, epsilon=None) Train network. You can control network’s training procedure with epochs and epsilon parameters. The input_test and target_test should be presented both in case of you need to validate network’s training after each iteration. fit(*args, **kwargs) Alias to the train method. - init_variables()[source]
Initialize Tensorflow variables.
- on_epoch_start_update(epoch)[source]
Function will be triggered before the training epoch procedure.
Parameters: epoch : int
Current epoch number.
- options = {'verbose': Option(class_name='Verbose', value=VerboseProperty(name="verbose")), 'step': Option(class_name='BaseNetwork', value=NumberProperty(name="step")), 'show_epoch': Option(class_name='BaseNetwork', value=ShowEpochProperty(name="show_epoch")), 'shuffle_data': Option(class_name='BaseNetwork', value=Property(name="shuffle_data")), 'epoch_end_signal': Option(class_name='BaseNetwork', value=Property(name="epoch_end_signal")), 'train_end_signal': Option(class_name='BaseNetwork', value=Property(name="train_end_signal")), 'error': Option(class_name='ConstructibleNetwork', value=FunctionWithOptionsProperty(name="error")), 'addons': Option(class_name='BaseGradientDescent', value=Property(name="addons")), 'minstep': Option(class_name='RPROP', value=BoundedProperty(name="minstep")), 'maxstep': Option(class_name='RPROP', value=BoundedProperty(name="maxstep")), 'increase_factor': Option(class_name='RPROP', value=BoundedProperty(name="increase_factor")), 'decrease_factor': Option(class_name='RPROP', value=ProperFractionProperty(name="decrease_factor"))}[source]
- update_prev_delta(prev_delta)[source]