neupy.layers.embedding module

class neupy.layers.embedding.Embedding[source]

Embedding layer accepts indices as an input and returns rows from the weight matrix associated with these indices. It’s useful when inputs are categorical features or for the word embedding tasks.

Parameters:
input_size : int

Layer’s input vector dimension. It’s, typically, associated with number of categories or number of unique words that input vector has.

output_size : int

Layer’s output vector dimension.

weight : array-like, Tensorfow variable, scalar or Initializer

Defines layer’s weights. Default initialization methods you can find here. Defaults to HeNormal().

name : str or None

Layer’s name. Can be used as a reference to specific layer. Name Can be specified as:

  • String: Specified name will be used as a direct reference to the layer. For example, name=”fc”
  • Format string: Name pattern could be defined as a format string and specified field will be replaced with an index. For example, name=”fc{}” will be replaced with fc1, fc2 and so on. A bit more complex formatting methods are acceptable, for example, name=”fc-{:<03d}” will be converted to fc-001, fc-002, fc-003 and so on.
  • None: When value specified as None than name will be generated from the class name.

Defaults to None.

Examples

This example converts dataset that has only categorical variables into format that suitable for Embedding layer.

>>> import numpy as np
>>> from neupy.layers import *
>>>
>>> dataset = np.array([
...     ['cold', 'high'],
...     ['hot',  'low'],
...     ['cold', 'low'],
...     ['hot',  'low'],
... ])
>>>
>>> unique_value, dataset_indices = np.unique(
...     dataset, return_inverse=True
... )
>>> dataset_indices = dataset_indices.reshape((4, 2))
>>> dataset_indices
array([[0, 1],
       [2, 3],
       [0, 3],
       [2, 3]])
>>>
>>> n_features = dataset.shape[1]
>>> n_unique_categories = len(unique_value)
>>> embedded_size = 1
>>>
>>> network = join(
...     Input(n_features),
...     Embedding(n_unique_categories, embedded_size),
...     # Output from the embedding layer is 3D
...     # To make output 2D we need to reshape dimensions
...     Reshape(),
... )
Attributes:
variables : dict

Variable names and their values. Dictionary can be empty in case if variables hasn’t been created yet.

Methods

variable(value, name, shape=None, trainable=True) Initializes variable with specified values.
get_output_shape(input_shape) Computes expected output shape from the layer based on the specified input shape.
output(*inputs, **kwargs) Propagates input through the layer. The kwargs variable might contain additional information that propagates through the network.
create_variables(input_shape)[source]
get_output_shape(input_shape)[source]
input_size = None[source]
options = {'input_size': Option(class_name='Embedding', value=IntProperty(name="input_size")), 'name': Option(class_name='BaseLayer', value=Property(name="name")), 'output_size': Option(class_name='Embedding', value=IntProperty(name="output_size")), 'weight': Option(class_name='Embedding', value=ParameterProperty(name="weight"))}[source]
output(input_value, **kwargs)[source]
output_size = None[source]
weight = None[source]