Storage for Neural Networks

Save and load layer parameters

NeuPy allows storing network’s parameters in a pickle file. Here is an example

from neupy import layers, storage

network = layers.join(
    layers.Input(10),
    layers.Relu(20),
    layers.Relu(30),
    layers.Softmax(10),
)
storage.save(network, filepath='/path/to/file.pickle')

To be able to load parameters you need to have predefined network structure. Using layer names NeuPy can restore parameters from the pickle file.

storage.load(network, filepath='/path/to/file.pickle')

Since parameters are stored in a regular pickle files it’s possible to load them without NeuPy.

import pickle

with open('/path/to/file.pickle', 'rb') as f:
    parameters = pickle.load(f)

The stored object in the pickle file has the following format.

{
    'layer-name-1': {
        'weight': np.array([...]),
        'bias': np.array([...]),
    },
    'layer-name-2': {
        'weight': np.array([...]),
        'bias': np.array([...]),
    },
    ...
}

Save and load algorithms

After succesful learning you can save network and later use it for prediction tasks. There already exists awesome library - dill. As a pickle library dill provides similar interface to serialize and de-serialize built-in python objects. The main advantage of this is a functionality that can store a network class and attributes without additional functionality.

First of all you need to install dill library

$ pip install dill>=0.2.3

There is a simple example for network serialization.

>>> import dill
>>> import numpy as np
>>> from sklearn import datasets, preprocessing
>>> from neupy import algorithms
>>>
>>> np.random.seed(0)
>>>
>>> # Prepare the data
... data, target = datasets.make_regression(n_features=2, n_targets=1)
>>> data = preprocessing.MinMaxScaler().fit_transform(data)
>>> target = preprocessing.MinMaxScaler().fit_transform(target)
>>>
>>> # Init and train network
... bpnet = algorithms.GradientDescent((2, 5, 1), step=0.1, verbose=False)
>>> bpnet.train(data, target, epochs=10000)
>>>
>>> predicted = bpnet.predict(data)
>>> bpnet.error(predicted, target.reshape(target.size, 1))
0.000756823576315
>>>
>>> with open('network-storage.dill', 'wb') as f:
...     dill.dump(bpnet, f)
...

And then you can load it from the network-storage.dill file and try to reproduce the same error rate.

>>> import dill
>>> import numpy as np
>>> from sklearn import datasets, preprocessing
>>>
>>> np.random.seed(0)
>>>
>>> # Get the same data set because we use the same seed number.
... data, target = datasets.make_regression(n_features=2, n_targets=1)
>>> data = preprocessing.MinMaxScaler().fit_transform(data)
>>> target = preprocessing.MinMaxScaler().fit_transform(target)
>>>
>>> with open('network-storage.dill', 'rb') as f:
...     bpnet = dill.load(f)
...
>>> predicted = bpnet.predict(data)
>>> bpnet.error(predicted, target.reshape(target.size, 1))
0.00075682357631507964

The interface for dill library is the same as for pickle, so functions like dumps or loads are available.