In this context it is of interest that previous studies demonstrated that feedforward propagation of activity to higher cortical areas mainly utilizes the AMPA receptor, whereas feedback effects rely more on the NMDA receptor [85], which plays an important role in synaptic plasticity .

NMDA receptors also modify neuronal activity in lower areas, and another candidate receptor that could have a specific role in the influence of feedback connections on plasticity are metabotropic glutamate receptors, which are prominent in feedback pathways [86,87] and known to influence synaptic plasticity [88].

If the outcome deviates from the reward-prediction, a neuro-modulatory signal that codes the global reward-prediction error (RPE) gates synaptic plasticity in order to change the Q-value, in accordance with experimental findings [9—12].

The first two factors are pre and post-syn-aptic activity of the units and there are two additional “gating factors” that enable synaptic plasticity .

Dopamine is often implicated because it is released if reward expectancy increases and it influences synaptic plasticity [10,38].

There is also a potential role for acetylcholine because cholinergic cells project diffusely to cortex, respond to rewards [61—63] and influence synaptic plasticity [61,64].

The hypothesis that synaptic plasticity requires a sequence of events [66,67] is supported by the synapses’ complex biochemical machinery.

Neuromodulatory signals influence synaptic plasticity even if released seconds or minutes later than the plasticity-inducing event [15,17,32] , which supports the hypothesis that they interact with some form of tag.

To test this idea, we introduced STDP-type synaptic plasticity in lateral excitatory connections and feedback inhibitory connections and investigated how different STDP rules cause different structures in the circuit.

We first studied synaptic plasticity at the feed-forward connections (connections from the input layer to the output layer), while fixing lateral connections (i.e., connections from the output layer to the lateral layer and connections from the lateral layer to the output layer).

Synaptic plasticity .

For most of this study, we focused on synaptic plasticity in the feedfor-ward connection WX, with fixed lateral synaptic weights WY and W2.

Such basic geometric requirement, which was explicitly recognized in Donald Hebb’s original formulation of synaptic plasticity , is not usually accounted for in neural network learning rules.

Such basic geometric requirement was explicitly recognized in Hebb’s original formulation of synaptic plasticity , yet is not usually accounted for in neural network learning rules.

More generally, circuit connectivity, synaptic plasticity , and neuronal placement are interrelated in a broad class of other common neural network approaches, including Kohonen-type self-organizing maps [17].