# neupy.algorithms.LevenbergMarquardt

class neupy.algorithms.LevenbergMarquardt[source]

Levenberg-Marquardt algorithm is a variation of the Newton’s method. It minimizes MSE error. The algorithm approximates Hessian matrix using dot product between two jacobian matrices.

Parameters: connection : list, tuple or LayerConnection instance Network’s architecture. There are a few ways to define it. List of layers. For instance, [Input(2), Tanh(4), Relu(1)]. Construct layer connections. For instance, Input(2) > Tanh(4) > Relu(1). Tuple of integers. Each integer defines Sigmoid layer and it’s input size. For instance, value (2, 4, 1) means that network has 3 layers with 2 input units, 4 hidden units and 1 output unit. mu : float Control invertion for J.T * J matrix, defaults to 0.1. mu_update_factor : float Factor to decrease the mu if update decrese the error, otherwise increse mu by the same factor. Defaults to 1.2 error : {mse} Levenberg-Marquardt works only for quadratic functions. Defaults to mse. show_epoch : int or str This property controls how often the network will display information about training. There are two main syntaxes for this property. You can define it as a positive integer number. It defines how offen would you like to see summary output in terminal. For instance, number 100 mean that network shows summary at 100th, 200th, 300th … epochs. String defines number of times you want to see output in terminal. For instance, value '2 times' mean that the network will show output twice with approximately equal period of epochs and one additional output would be after the finall epoch. Defaults to 1. shuffle_data : bool If it’s True class shuffles all your training data before training your network, defaults to True. epoch_end_signal : function Calls this function when train epoch finishes. train_end_signal : function Calls this function when train process finishes. verbose : bool Property controls verbose output interminal. True enables informative output in the terminal and False - disable it. Defaults to False. addons : list or None The list of addon algortihms. None by default. If this option is not empty it will generate new class which will inherit all from this list. Support two types of addon algorithms: weight update and step update.

Notes

• Method requires all training data during propagation, which means it’s not allowed to use mini-batches.
• Network minimizes only Mean Squared Error (MSE) loss function.
• Efficient for small training datasets, because it computes gradient per each sample separately.
• Efficient for small-sized networks.

Examples

>>> import numpy as np
>>> from neupy import algorithms
>>>
>>> x_train = np.array([[1, 2], [3, 4]])
>>> y_train = np.array([[1], [0]])
>>>
>>> lmnet = algorithms.LevenbergMarquardt((2, 3, 1))
>>> lmnet.train(x_train, y_train)

Attributes: errors : ErrorHistoryList Contains list of training errors. This object has the same properties as list and in addition there are three additional useful methods: last, previous and normalized. train_errors : ErrorHistoryList Alias to the errors attribute. validation_errors : ErrorHistoryList The same as errors attribute, but it contains only validation errors. last_epoch : int Value equals to the last trained epoch. After initialization it is equal to 0.

Methods

 predict(input_data) Predicts output for the specified input. train(input_train, target_train, input_test=None, target_test=None, epochs=100, epsilon=None) Train network. You can control network’s training procedure with epochs and epsilon parameters. The input_test and target_test should be presented both in case of you need to validate network’s training after each iteration. fit(*args, **kwargs) Alias to the train method.
error = None[source]

Initialize updates that would be applied after each training epoch.

init_variables()[source]

Initialize Tensorflow variables.

mu = None[source]
mu_update_factor = None[source]
on_epoch_start_update(epoch)[source]

Function will be triggered before the training epoch procedure.

Parameters: epoch : int Current epoch number.
options = {'addons': Option(class_name='BaseGradientDescent', value=Property(name="addons")), 'epoch_end_signal': Option(class_name='BaseNetwork', value=Property(name="epoch_end_signal")), 'error': Option(class_name='LevenbergMarquardt', value=ChoiceProperty(name="error")), 'mu': Option(class_name='LevenbergMarquardt', value=BoundedProperty(name="mu")), 'mu_update_factor': Option(class_name='LevenbergMarquardt', value=BoundedProperty(name="mu_update_factor")), 'show_epoch': Option(class_name='BaseNetwork', value=ShowEpochProperty(name="show_epoch")), 'shuffle_data': Option(class_name='BaseNetwork', value=Property(name="shuffle_data")), 'train_end_signal': Option(class_name='BaseNetwork', value=Property(name="train_end_signal")), 'verbose': Option(class_name='Verbose', value=VerboseProperty(name="verbose"))}[source]
step = None[source]