Hierarchy

class pyogmaneo.Hierarchy

The sparse predictive hierarchy (SPH). Can be thought of as the “agent” when used for reinforcement learning.

Hierarchy.__init__(self, cs, prog, inputSizes, inputTypes, firstLayerDesc, higherLayerDescs)

Initialize a hierarchy of given structure.

Parameters:
  • cs – (ComputeSystem) compute system to use
  • prog – (ComputeProgram) compute program containing the necessary kernels (e.g. from neoKernels.cl)
  • inputSizes – ([Int3]) list of input sizes. Defines the size of each input layer
  • inputTypes –

    ([int]) list of enums defining the type of each input layer. Must be the same size as inputSizes. Possible enums are:

    pyogmaneo._inputTypeNone - used solely as an input (no additional functionality)

    pyogmaneo._inputTypePrediction - used as input and can also predict next timestep of input (t + 1)

    pyogmaneo._inputTypeAction - used as input and can also perform reinforcement learning (produces actions)

  • firstLayerDesc – (FirstLayerDesc) A filled-out descriptor (FirstLayerDesc object) describing how the first layer in the hierarchy should look
  • higherLayerDescs – ([HigherLayerDesc]) A list of filled-out descriptors (HigherLayerDesc objects) describing how all the higher layers (all but first) in the hierarchy should look
Hierarchy.__init__(self, cs, prog, name)

Initialize a hierarchy given a save file.

Parameters:
  • cs – (ComputeSystem) compute system to use
  • prog – (ComputeProgram) compute program containing the necessary kernels (e.g. from neoKernels.cl)
  • name – (string) save file name
Hierarchy.step(self, cs, inputCs, learnEnabled=True, reward=0.0)

Perform a simulation step of the hierarchy. This will produce new predictions or actions if those are being used.

Parameters:
  • cs – (ComputeSystem) compute system to use
  • inputCs – ([IntBuffer]) list of input integer buffers representing the CSDRs of the dimensions described in the initialization
  • learnEnabled – (boolean) whether or not to enable learning (if False, will only perform inference). Defaults to True
  • reward – (float32) reward signal, if action input layers (pyogmaneo._inputTypeAction) are present this will be used to update those to maximize reward. Defaults to 0.0
Hierarchy.save(self, cs, name)

Save the hierarchy to a file.

Parameters:
  • cs – (ComputeSystem) compute system to use
  • name – (string) save file name
Hierarchy.getNumLayers(self)

Return the number of layers (encoders) the hierarchy has. Will be equal to (len(higherLayerDescs) + 1)

Return type:(int32) number of layers
Hierarchy.getPredictionCs(self, i)

Get a prediction for a given input layer index

Parameters:i – (int32) index of the input layer. Will error if input layer does not have type pyogmaneo._inputTypePrediction
Return type:(IntBuffer) integer buffer containing predictions
Hierarchy.getActionCs(self, i)

Get an action for a given input layer index

Parameters:i – (int32) index of the input layer. Will error if input layer does not have type pyogmaneo._inputTypeAction
Return type:(IntBuffer) integer buffer containing actions
Hierarchy.getUpdate(self, l)

Return whether a given layer has updated (clocked) in the last timestep

Parameters:l – (int32) index of the layer
Return type:(boolean) True if updated, False otherwise
Hierarchy.getHiddenCs(self, l)

Get the CSDR of the encoder of a certain layer

Parameters:l – (int32) index of the layer
Return type:(IntBuffer) integer buffer containing encoder hidden layer states
Hierarchy.getHiddenSize(self, l)

Get the size of the encoder of a certain layer

Parameters:l – (int32) index of the layer
Return type:(Int3) size of layer. Will always be equal to the size if was initialzed to
Hierarchy.getTicks(self, l)

Get current tick of a layer (clock value)

Parameters:l – (int32) index of the layer
Return type:(int32) tick value. Will always be less the value produced by Hierarchy.getTicksPerUpdate(l) Will always be 0 for the first layer (l = 0)
Hierarchy.getTicksPerUpdate(self, l)

Number of ticks required before a layer clocks. Typically 2

Parameters:l – (int32) index of the layer
Return type:(int32) number of ticks. Will always be 1 for the first layer (l = 0)
Hierarchy.getNumInputLayers(self)

Get the number of input layers. Will never deviate from the length of inputSizes given during initialization.

Return type:(int32) number of input layers
Hierarchy.getNumPLayers(self, l)

Get the number of predictor (top down) sub-layers at a given layer

Parameters:l – (int32) index of the layer. Must be greater than 0 as the first layer does not have regular predictors
Return type:(int32) number of input layers
Hierarchy.getNumSCVisibleLayers(self, l)

Get the number of visible (sub) layers of a sparse coder (SC), AKA encoder. This will be equal to Hierarchy.getTicksPerUpdate(l) for all l except for 0, where it will be equal to Hierarchy.getNumInputLayers()

Parameters:l – (int32) index of the layer
Return type:(int32) number of visible layers
Hierarchy.pLayerExists(self, i)

Determine whether there is a predictor at the i-th input layer. This will be True if the inputTypes[i] == pyogmaneo._inputTypePrediction during initialization

Parameters:i – (int32) index of the input layer
Return type:(boolean) True if exists, False otherwise
Hierarchy.aLayerExists(self, i)

Determine whether there is a action layer at the i-th input layer. This will be True if the inputTypes[i] == pyogmaneo._inputTypeAction during initialization

Parameters:i – (int32) index of the input layer
Return type:(boolean) True if exists, False otherwise
Hierarchy.setSCAlpha(self, l, alpha)

Set the alpha (learning rate) of a sparse coder (SC) (AKA encoder)

Parameters:
  • l – (int32) index of the layer
  • alpha – (float32) value to set
Hierarchy.getSCAlpha(self, l)

Get the alpha (learning rate) of a sparse coder (SC) (AKA encoder)

Parameters:l – (int32) index of the layer
Return type:(float32) alpha
Hierarchy.setPAlpha(self, l, alpha)

Set the alpha (learning rate) of a predictor (P) (AKA decoder)

Parameters:
  • l – (int32) index of the layer. This function is used for predictors above the first layer, so l > 0
  • alpha – (float32) value to set
Hierarchy.getPAlpha(self, l)

Get the alpha (learning rate) of a predictor (P) (AKA decoder)

Parameters:l – (int32) index of the layer. This function is used for predictors above the first layer, so l > 0
Return type:(float32) alpha
Hierarchy.setBPAlpha(self, i, alpha)

Set the alpha (learning rate) of a predictor (P) (AKA decoder) at the bottom (B) of the hierarchy (input layer)

Parameters:
  • i – (int32) index of the input layer
  • alpha – (float32) value to set
Hierarchy.getBPAlpha(self, i)

Get the alpha (learning rate) of a predictor (P) (AKA decoder) at the bottom (B) of the hierarchy (input layer)

Parameters:i – (int32) index of the input layer
Return type:(float32) alpha
Hierarchy.setBAAlpha(self, i, alpha)

Set the alpha (value learning rate) of an action layer (A) at the bottom (B) of the hierarchy (input layer)

Parameters:
  • i – (int32) index of the input layer
  • alpha – (float32) value to set
Hierarchy.getBAAlpha(self, i)

Get the alpha (value learning rate) of an action layer (A) at the bottom (B) of the hierarchy (input layer)

Parameters:i – (int32) index of the input layer
Return type:(float32) alpha
Hierarchy.setBABeta(self, i, beta)

Set the beta (action learning rate) of an action layer (A) at the bottom (B) of the hierarchy (input layer)

Parameters:
  • i – (int32) index of the input layer
  • beta – (float32) value to set
Hierarchy.getBABeta(self, i)

Get the beta (action learning rate) of an action layer (A) at the bottom (B) of the hierarchy (input layer)

Parameters:i – (int32) index of the input layer
Return type:(float32) beta
Hierarchy.setBAGamma(self, i, gamma)

Set the gamma (discount factor) of an action layer (A) at the bottom (B) of the hierarchy (input layer)

Parameters:
  • i – (int32) index of the input layer
  • beta – (float32) value to set
Hierarchy.getBAGamma(self, i)

Get the gamma (discount factor) of an action layer (A) at the bottom (B) of the hierarchy (input layer)

Parameters:i – (int32) index of the input layer
Return type:(float32) gamma