Biol Cybern DOI 10.1007/s00422-013-0581-3

ORIGINAL PAPER

A feedforward model for the formation of a grid field where spatial information is provided solely from place cells Luísa Castro · Paulo Aguiar

Received: 19 June 2013 / Accepted: 20 December 2013 © Springer-Verlag Berlin Heidelberg 2014

Abstract Grid cells (GCs) in the medial entorhinal cortex (mEC) have the property of having their firing activity spatially tuned to a regular triangular lattice. Several theoretical models for grid field formation have been proposed, but most assume that place cells (PCs) are a product of the grid cell system. There is, however, an alternative possibility that is supported by various strands of experimental data. Here we present a novel model for the emergence of gridlike firing patterns that stands on two key hypotheses: (1) spatial information in GCs is provided from PC activity and (2) grid fields result from a combined synaptic plasticity mechanism involving inhibitory and excitatory neurons mediating the connections between PCs and GCs. Depending on the spatial location, each PC can contribute with excitatory or inhibitory inputs to GC activity. The nature and magnitude of the PC input is a function of the distance to the place field center, which is inferred from rate decoding. A biologically plausible learning rule drives the evolution of the connection strengths from PCs to a GC. In this model, PCs compete for GC activation, and the plasticity rule favors efficient packing of the space representation. This leads to gridlike firing patterns. In a new environment, GCs continuously recruit new PCs to cover the entire space. The model described here makes important predictions and can represent the feedforward connections from hippocampus CA1 to deeper mEC layers. L. Castro Departamento de Matemática, Faculdade de Ciências, Universidade do Porto, Porto, Portugal L. Castro · P. Aguiar (B) Centro de Matemática, Universidade do Porto, Porto, Portugal e-mail: [email protected] P. Aguiar Instituto de Biologia Molecular e Celular, Porto, Portugal

Keywords Grid cells · Place cells · Inhibitory neurons plasticity · Combined plasticity rule · Efficient packing · Competition

1 Introduction Grid cells (GCs) are principal cells of the medial entorhinal cortex (mEC) that exhibit a multipeak firing rate map with the nodes in a geometrically regular, triangular lattice that tiles the entire arena (Hafting et al. 2005). It is commonly believed that these spatial cells were recorded previously (Quirk et al. 1992) but failed to reveal their hexagonal tessellating pattern due to the unfortunate combination of small environments with cells in ventral portions of the mEC (where the spacing between nodes is larger). Each grid map can be characterized by three parameters: phase or position of the grid vertices in the plane; orientation, defined by the lines that intersect the grid nodes; and spacing between the centers of activity (Moser and Moser 2008). GCs in rodents have been found essentially in the mEC, throughout all principal cell layers (II, III, V, VI) intermingled with other navigation-activityrelated cells such as head-direction cells, conjunctive cells, and border cells (Sargolini et al. 2006). More recently, the same types of cells were also found in the pre- and parasubiculum (Boccara et al. 2010). Experiments show that for complete coverage of the available environment it is enough to record a small number of GCs in the same recording location. There is no evidence of a topographic relation between the relative position of GCs in the mEC and their receptive fields’ vertex locations. Nevertheless, neighboring cells show fields with similar orientation, field size, and spacing (Hafting et al. 2005). In GC firing maps, the spacing and size of the receptive fields increase from dorsal to ventral recording locations

123

Biol Cybern

(Stensola et al. 2012; Barry et al. 2007). Interestingly, an analogous relation has been reported for place cells (PCs) in the hippocampus CA1 and CA3 regions, where receptive fields are found to be larger in ventral locations when compared to the most dorsal ones (Jung et al. 1994; Abarbanel et al. 2002; Kjelstrup et al. 2008). Since they were initially reported, GCs have motivated the development of models addressing different mechanisms for their formation. The first generations of published models were divided into two main classes: those relying on the interference of oscillators with similar frequency (Blair et al. 2008; Burgess et al. 2007; Giocomo et al. 2007; Hasselmo et al. 2007; Zilli and Hasselmo 2010) and those depending on continuous attractor network dynamics (Burak and Fiete 2009; Fuhs and Touretzky 2006; Guanella et al. 2007; McNaughton et al. 2006; Navratilova et al. 2012). Kropff and Treves developed yet another type of model where connections from PCs are slowly modified through self-organization, with firing adaptation playing a major role (Kropff and Treves 2008; Si et al. 2012). Other very recent models for GC formation are based on inputs coming from stripe cells and can be considered self-organizing models in the way synaptic weights are learned. These models also possess features of the classes mentioned previously such as ring attractors and recurrent connectivity (Grossberg and Pilly 2012; Mhatre et al. 2012). Most existing models assume that GCs perform path integration and, more importantly, consider PCs to be a product of the GC system. There is, however, an alternative possibility that is in fact supported by various strands of experimental data: that GCs represent the output of a PC network. First, results from experiments with rat pups (experiencing their first excursion from the nest) provide evidence that head direction cells develop first, followed by PCs and, finally, GCs (Langston et al. 2010; Wills et al. 2010). This development hierarchy might suggest that place fields could provide the spatial inputs to the GC firing field formation. Experimental studies have also been carried out to assess the effect of environment modifications on the receptive fields of both GCs and PCs. One example, made with PCs in CA3 and GCs in mEC layer II, has shown that when rate remapping occurs in PCs, grid vertices remain stable in mEC cells. Moreover, when global remapping occurs, e.g., a different set of PCs is activated, grid fields realign, shifting their vertices’ location without losing their hexagonal spatial phase arrangement (Fyhn et al. 2007). It has also been shown that grid firing patterns are deformed when the geometry of a familiar environment is manipulated (Barry et al. 2007), indicating that GCs receive sensory input that presumably comes from PCs. Finally, one recent study found that inactivation of the hippocampus led to an extinction of the grid pattern (Bonnevie et al. 2013), indicating that the excitatory drive from the hip-

123

pocampus is a prerequisite for the formation of grid patterns in the mEC. The results described previously motivated the model described here, where cells in the deeper layers of the entorhinal cortex are driven by inputs coming from PCs located downstream in the hippocampus, namely, in the CA1 field. In this model we hypothesize that the feedforward circuit mediating the connection between PCs and GCs is capable of promoting a packed representation of space where each activity node in a grid field is produced by ensembles of spatially coherent PCs.

2 Materials and methods The model was developed, simulated, and analyzed in MATLAB (R2012a, MathWorks, Natick, MA, USA). The model implementation is available through ModelDB (Hines et al. 2004) under accession number 150846. 2.1 Spatial input from place cells In the model, all spatial information reaching the GC network originates from PCs that encode the position of the hypothetical animal in an environment. Experimental reports show that PC firing fields homogeneously cover the training arena (Muller et al. 1987). For reasons of simplification, in most of the experiments presented here, place field centers are positioned on the nodes of a square lattice with 1 cm unit side that covers the entire virtual environment. However, it is also shown that the model is not dependent on the particular arrangement of PC centers, and, specifically, a random uniform distribution of the place centers leads to the same qualitative results of the model. A square environment with 1 m side was used in the simulations, implying a total of 100 × 100 PC units inside the environment. Furthermore, to provide a roughly homogeneous positional input to the GC, we also included PCs with centers that were outside the arena but close to its border. In this way, the average activity level provided by the PCs to the GC close to the enclosure periphery is not smaller than the activity in central regions. A border of 40 cm was considered around the square arena, increasing the total number of simulated PCs to 180 × 180 units regularly distributed in a square lattice. This uniformity in input excitation on the environment could also be generated using border cells. The individual place field firing intensity is defined as a bidimensional Gaussian function. In most experiments, the same (isotropic) standard deviation σ is used in all place fields, and its magnitude can be changed to reflect the hippocampal area that is being modeled (dorsal/ventral). Heterogeneous place fields – where the orthogonal σx and σ y in the bidimensional Gaussian are randomly selected to produce

Biol Cybern

variations in the field area, aspect ratio, and orientation–are shown to lead to the same qualitative results. The trajectory of the animal is described by a function p(t), which sets the position in the environment at each time step t. This function is used to obtain each PC’s firing rate: u i ( p (t)) = e



d 2 ( p(t),θi ) 2σ 2

,

where θi is the position of the ith place field center, d() is the Euclidean distance, and σ sets the scale of the place fields. The firing rate u i of the PCs is considered to be normalized and constrained in the range [0,1]. In other words, this model assumes that the firing rate range in PCs is approximately normalized, meaning that the firing rate coding scheme is shared among the PC population. As a result, the firing rate of a PC is highly informative about the absolute distance to its place field center. 2.2 Network connectivity The model proposes a simple feedforward architecture where one grid cell receives indirect input from N PCs associated with different locations (space sampling). In the simulations, the grid cell receives connections from all PCs. This is not a requirement of the model, however, and this condition can be relaxed to a reduced number of PC inputs as long as a reasonable sampling of the available space is provided. The feedforward connections between each PC and the GC are mediated by two pathways of opposite influence: one excitatory and one inhibitory (Fig. 1). Each PC drives one excitatory neuron (E) and one inhibitory neuron (I1 ) responsible for these two pathways. The excitatory neuron is not essential to the model and serves the purpose of generalizing the excitatory pathway (demonstrating the ability to accommodate different connectivity conditions between PCs and GCs, besides the direct/monosynaptic situation). The inhibitory pathway is subject to an activity level control by a third class of neurons (I2 ). Different studies have addressed the contributions of inhibitory neurons in controlling activity levels in the hippocampus (Castro and Aguiar 2012; Freund and Gulyas 1997). The core assumption of our proposed architecture is that each PC can have a combined excitatory and inhibitory influence over GC’s activity. The nature and strength of the influence is determined by the PC firing activity. 2.3 Grid cell firing rate model The dynamics of GCs are described using the firing-rate model  N   dr = −r + f wi u i , τr dt i=1

Fig. 1 Diagram of model network architecture. The key element in the proposed model is that each PC can both excite and inhibit a GC. In the presented architecture, this modulation is mediated by three types of association neurons: excitatory neuron E, and inhibitory neurons I1 and I2 . All synapses have a static unit weight w0 , except for the synapses directly targeting the GC. Plastic changes to the excitatory synaptic weight w+ and the inhibitory synaptic weight w− depend on the firing rates r E and r1 , respectively. A grid cell receives connections from N PCs, each combined with an E and I1 and I2 neurons (shaded area)

where τr = 20 ms; u i and wi represent respectively the firing rate of the inputs and their associated synaptic efficacies (connection weights). For the transfer function f we used a sigmoid function defined by parameters a (slope) and b (midpoint): φ (z) =

1 1 + e−

z−b a

.

In the GC model, parameter a was set to 0.12 while parameter b was set to 0.5. For the purpose of simplification, in most analyses the dynamics of the association neurons E, I1 , and I2 are not explicitly described by a differential equation and their modulatory action on GCs is instead represented as a transfer function that is applied to the PCs’ output activity. This transfer function reflects the combined action of the individual transfer functions of the association neurons and allows the model to be analyzed with only two neuronal classes, one GC and N PCs, with a simplified direct connection between them. Exclusively in the analysis of the biological plausibility of the synaptic modification rule, the association neurons E, I1 , and I2 are explicitly described by firing-rate models with τr = 100 ms. In the case of the association neurons, parameter a was set to 0.02 for all neurons. Parameter b assumes distinct values, b E , b1 , and b2 , corresponding to the three distinct association neurons E, I1 , and I2 , respectively.

123

Biol Cybern

For the simulations presented here the values b E = 0.25, b1 = 0.50, and b2 = 0.75 were used. 2.4 Synaptic plasticity learning rule All synaptic connections are static and take a unit value (w0 = 1), with the exception of the synapses E-GC and I1 -GC, which are plastic and subject to a fast learning rule (Fig. 1). Learning in these synapses is triggered by a high threshold value r th in the GC firing rate; below this fixed value no learning takes place. Each time the GC’s rate rises above rth , the connections with the PCs whose field center is close to the animal’s current position undergo modifications. These PCs then become responsible for one node of the grid field. This is an association learning rule as both presynaptic and postsynaptic conditions are considered. For the purpose of simplification in the model, plasticity takes place in the collapsed direct connection between PCs and the GC. A synapse between a PC and the GC represents the combined action of w+ and w− (Fig. 1). To emphasize this, the term connection strength is used instead of synaptic efficacy to address the modifications driven by the learning rule. A gain modulation mechanism is assumed to take place in the association circuit between PCs and GC during learning episodes. The gain function is implemented by increasing the spatial scale σ of the place fields to σ˜ = 2.5σ , thereby broadening the spatial tuning provided by the PCs to the GC. This gain modulation is associated with plasticity (it only takes place during learning) and takes into account published studies showing that the place fields in rats foraging in unfamiliar environments are larger than those reported for familiar arenas (Barry et al. 2012; Karlsson and Frank 2008). The plasticity and gain modulation proposed here are in accordance with the action of acetylcholine in learning and memory in the hippocampus and entorhinal cortex (Acquas et al. 1996; Hasselmo 2006; Heys et al. 2012). This cholinergic neuromodulator is known to be released when the animal is introduced in a novel environment and mediates an increase in the place field size and the magnitude of synaptic plasticity present in the current model. Plasticity occurs in a one-step procedure and is dependent on the present weights. The connection strength is considered to be on one of four distinct levels: minimum wmin , baseline wb , switching ws , and maximum wmax , where wmin < wb < ws < wmax . Changes can occur in both directions (potentiation and depression), with sequential transitions between levels. In an unfamiliar environment, connections from PCs are assumed to have similar weights wb . When entering an unfamiliar environment, a GC is assumed to be prone to high firing activity—resulting from the enhanced PC firing rates driven by acetylcholine—such that, independently of the animal’s

123

position, it easily reaches r th and triggers the first plasticity event. In this initial condition, given the homogeneity of the PC connection strengths, every location in the environment is a possible location for the emergence of the first grid field node. Upon the first plasticity event, the strengths of the connections from PCs are pushed toward one of the three modified levels (wmin , ws , or wmax ) according to each PC firing rate. The PCs’ firing rate is divided into ranges 1 > q1 > q2 > q3 > 0, each defining an interval of distances to the place field center. In this first plasticity event, PCs with a rate above q1 have their connections modified to wmax . This sets a central disk, region R1 , centered in the position of the animal. Surrounding this inner disk, a ring-shaped region R2 is formed with all the PCs firing in the interval [q2 , q1 [. The connections of PCs within this firing rate range are modified to a minimal value (which could also be negative) wmin . PCs firing in the range [q3 , q2 [, forming an outer ring R3 , have their connections potentiated to ws . Finally, PCs with a firing rate below q3 , i.e., field centers located in the rest of the environment, are not subject to any modification in their connection strengths, which are kept at wb . The dependence of the synaptic modification on the PC firing rate, triggered only when the GC rate is above r th, is represented in Fig. 2. The radii associated with each of the regions R1 , R2 , and R3 are, respectively, φ1 , φ2 , and φ3 . The initial connection strengths, wmin , ws , and wmax , are set in such a way that, after the first learning event, where the GC was considered to be more prone to high firing rates, the only region where the GC again receives enough excitation to reach r th is on the ring R3 . In other words, a range in the firing rate of the PCs encodes a ring of space around the previously created grid field node where a new node can be created.

Fig. 2 Dependence between PC rates and connection strength modifications. Output of a PC with place field center at 0.5 and σ˜ = 0.125 m for a 1 m length linear path (blue/dark gray) and associated connection strength modifications (green/light gray) (color figure online)

Biol Cybern

As the animal explores the environment, each new learning event adds a new grid field node by recruiting one or more PCs with the field’s center (approximately) at the node’s location. The connections of the PCs to the GC in the inner disk R1 and inhibitory ring R2 are set, respectively, to wmax and wmin , as previously. For PCs in the R3 region (rates in the interval [q3 , q2 [) two types of potentiation may occur: connections that were at level ws now become wmax ; connections that were at level wb now become ws . Connections that were previously wmin or wmax do not undergo modifications. The proposed synaptic plasticity learning rule is summarized in Table 1. This mechanism is repeated as the animal explores the environment, with the GC reaching rth (and thus triggering plasticity) only in the intersecting areas of the outer potentiation rings. The grid map is assembled as the rat explores the environment by recruiting new PCs to activate the nodes of the grid field. By construction, the nodes of the grid field form a regular triangular lattice, as a rate range in PCs encodes a specific distance interval. Different parameterizations can satisfy the constraints of the learning rule and support grid field formation from the recruitment/competition of PCs. The parameterization used here takes into account experimental data such as firing rate and the size and spacing of the grid field nodes (Sargolini et al. 2006). The values used were φ1 = 0.07, φ2 = 0.26, and φ3 = 0.34; q1 = 0.85, q2 = 0.11, and q3 = 0.025; wmin = 0, wb = 0.97 ρ, ws = 0.98 ρ, and wmax = 1.5 ρ. The parameter ρ is a normalization factor: if GC input connection strengths were all ρ, then the firing rate would be rth . The scale 1.5 is used to ensure a GC high firing rate if the inputs came only from R1 (assuming a null contribution from the outer regions). In our model, acetylcholine is responsible for two noveltyrelated mechanisms. The acetylcholine effect on broadening the spatial scale σ of the place fields (gain modulation, σ˜ = 2.5σ ) is maintained during the complete formation of the grid field. It lasts for the entire learning process and is removed only during recall in familiar environments (where the gain modulation is absent). Furthermore, during the forTable 1 Summary of discrete learning rule Learning events PC rate

Modification

[q1 , 1]

w∗ → wmax

[q2 , q1 [

w∗ → wmin

[q3 , q2 [

ws → wmax wb → ws wmax , wmin No change

[0, q3 [

No change

Synaptic modifications only take place when GC firing rate is above rth . The notation w∗ means “any connection strength level”

mation of the first two nodes of the grid, the acetylcholine is responsible for an additional global increase of PC firing rates. In the model, for simplification this temporary global increase is introduced directly at the weight level, instead of at the place field intensity level (which nevertheless would in turn lead to weight potentiation). Only during the formation of the first two nodes of the grid are the weights wb and ws increased by 0.2 ρ. This guarantees that in a novel environment, the first learning episode can take place at any location of the environment, and the second learning episode can take place in the newly formed R3 region. 2.5 Animal space trajectories Two types of trajectory were used to set the PC firing rates: experimentally recorded (real) trajectories and randomly generated (synthetic) trajectories. The real trajectories were obtained from the Moser group GC database, available for download at http://www.ntnu.no/cbm/moser/gridcell, which includes the position times for a rat running in a cylinder having a diameter of 180 cm with a time step of 20 ms. The simulated animal trajectories were constrained to a square environment with a 1 m side and were used in most analyses. The rat’s simulated path inside the square arena for a total time of T ms is defined by p(t) = (x(t), y(t)). After choosing an initial position to start the trajectory p(t0 ) = (x0 , y0 ), with x0 , y0  [0, 1] m, the position of the rat for the following iterations ti is obtained according to the recurrent rule:   cos (αi ) sin (αi ) , yi−1 + , p (ti ) = (xi , yi ) = xi−1 + h h where αi represents an angle in radians and 1/ h = dt × v represents the distance traveled in each step, given that dt is the time duration of each step and v is the simulated rat’s velocity. A constant velocity is used and, unless otherwise stated, is equal to 0.08 m/s. For step 2 ≤ i ≤ T /dt, the direction of the movement of the rat is given recurrently by αi = αi−1 + kdt ε radians, where k is the rate of change in direction, and ε is chosen randomly in each step from a Gaussian distribution with mean 0 and standard deviation 1. The initial direction α0 is set to zero. The rate of change in direction was tuned, taking into account the velocity and the time step duration, dt = 1 ms, in order to generate qualitatively reasonable rat trajectories; a value k = π/120 radians was used. Examples of a real trajectory and a synthetic trajectory are shown in Fig. 3. 3 Results Given the important role and nonstandard form of the learning rule in this model for the formation of grid fields, this results section starts with an analysis supporting the biological plausibility of the rule. Simulation results showing grid

123

Biol Cybern Fig. 3 Real and synthetic animal trajectories. a Real trajectory taken from Moser group database (Hafting et al. 2008) recorded in a circular environment with a diameter of 1.8 m. b Simulated rat trajectory on a square arena with 1.8 m side (for comparison with circular arena). Both real and simulated trajectories are for 3 min of foraging with velocities of 0.20 m/s (average value in real, and constant value in simulated). The time steps are 20 and 1 ms long, respectively

map formation, both for synthetic trajectories and real trajectories, are then presented, together with a demonstration that the model is robust to variable velocities, firing rate noise, and variability in place field locations. Autocorrelograms and gridness scores are presented for the model’s generated grid fields as quality metrics. 3.1 Biological plausibility of learning rule The connection strengths between PCs and GC in our model are obtained using a noncanonical rule. In this subsection, we provide biological support for the plasticity rule and show that this rule can be produced by a combination of standard synaptic plasticity rules applied in a small excitatory/inhibitory neuronal circuit. Instead of using the simplification of a direct connection between each PC and the GC, we consider the full architecture presented in Fig. 1. Given the parameterization b E = 0.25, b1 = 0.50, and b2 = 0.75, the excitatory and inhibitory influences of PCs on a GC are segregated into firing rate domains (Fig. 4). Excitatory modulation is limited to rates above b2 or within the interval [b E , b1 ], while the inhibitory modulation is limited to the interval [b1 , b2 ]. Plasticity takes place in the w+ and w− synapses, is triggered by a high activity level in the GC, and has a modification amplitude, w+ or w− , proportional to the presynaptic firing rate (canonical Hebbian-type learning rule in firing rate models). The distinct frequency domains and the combination of w+ and w− modifications generate the three levels of connection strengths wmax , ws , and wmin . The connection strength modification to wmax results from a strong potentiation in the excitatory (w+ ) pathway. The modification to wmin results from a shunting or competing potentiation in the inhibitory (w− ) pathway. The combined connection strength modification to ws results also from potentiation in the excitatory (w+ ) pathway alone (Fig. 4b–d).

123

Fig. 4 Learning rule rate dependence. a Firing rate of association neurons as a function of PC rate (both normalized). b Synaptic modification in excitatory pathway as a function of PC rate. c Synaptic modification in inhibitory pathway as a function of PC rate. d Overall connection strength modification (combined excitatory and inhibitory modifications) as a function of input rate from PC

Biol Cybern Fig. 5 Connection strength profiles associated with grid field formation. Each point in the maps represents a place field center (cell per pixel), and these are color coded (arrows) according to the connection strength between the associated PC and the GC. Scale bar is 0.6 m. a Result of first plasticity event, generated upon arrival in an unfamiliar environment. b, c, and d Result of subsequent learning events as the animal explores the environment following a specific trajectory. Velocity is not relevant in the process where PCs compete for GC activation. The grid field is continuously built from the recruitment of regularly spaced place field centers (color figure online)

The combined excitatory and inhibitory action of each PC on a GC hypothesized in this model allows the formation of different regions of influence, depending on the distance between the place field center and the animal’s position (encoded in the firing rate): a hot spot of excitation in the neighborhood of the place field center, an adjacent ring of inhibition, a surrounding excitation ring, and a neutral peripheral region. This is therefore a center-surround activation profile modulated by the PC’s firing rate encoding the distance to the place field center. The borders of the regions are defined by the parameters b E , b1 , b2 and σ . 3.2 Grid field formation As the animal is placed at the center of a new/unfamiliar environment, the first grid node is quickly formed, generating the connection strength map shown in Fig. 5a. The connection strength map is used to represent the connection strength between each PC and the GC as a function of the place field center location in space. This should not be confused with a topographic map—morphological neighboring PCs are not required to have neighboring place field centers. As the ani-

mal crosses for the first time the excitatory ring (R3 ), a second field node is formed, and only two possible regions are now available for the third field to be raised (diamond regions in Fig. 5b). As mentioned in the description of the model, at this stage the plasticity enhancement driven by acetylcholine is no longer required; the gain function, however, persists for the entire learning episode. As the animal explores the new environment, PCs compete for GC activation and become progressively recruited to give rise to the grid field nodes (Fig. 5c, d). In this simulation, the GC receives inputs from all the PCs, densely covering the environment area. This allows improved visualization of the connection strength maps by avoiding a patchy representation. But a very large number of PCs’ inputs is not required for the model to work: a reduced number of PCs’ inputs to the GC still produces the same qualitative results as long as the PCs provide a reasonable sampling of the space. The sampling resolution is related to the R1 disk area—for the parameters of the simulations, a few hundred PCs’ inputs (with place field centers uniformly distributed in the environment) are sufficient to support the grid field formation.

123

Biol Cybern Fig. 6 Grid cell firing rate map after learning process. a Firing rate map of a simulated GC after learning using a synthetic trajectory. Rates are presented on top of the animal’s trajectory—central regions have high normalized firing rates of ∼0.90 and the base line is 0.0. b Normalized autocorrelogram of field shown in a. c Grid field formation using real trajectories with variable velocity. Positional data from (Hafting et al. 2008), 30 min long with dt = 20 ms. d Firing rate map of GC in c, with a gridness score of 1.28. Scale bars are 0.5 m

It is important to note that in this model the grid map does not extend automatically to infinity; instead, it is progressively extended to the regions being explored. It is reasonable to speculate that the real trajectory of an animal is implicitly chosen/conditioned in order to more efficiently build the grid field through the recruitment process. Under this hypothesis, exploration gathers the local, auto-generated place fields to construct a global descriptor of the environment. Using this learning rule, under a constant velocity of 0.08 m/s in an environment with 1 m side, after approximately 20 min of animal exploration the available area is fully tiled and no more learning takes place (GC is no longer able to reach rth ). When the animal is placed back in the same environment after a learning episode, a firing rate map for the GC is obtained (Fig. 6a, different simulation). This GC firing rate map is used to compute the gridness score, according to Sargolini’s description (Sargolini et al. 2006), which requires the calculation of the autocorrelogram (Fig. 6b). Grid map measures for this simulation/parameterization, 36 cm for spacing

123

and 20 cm for field diameter, are in line with those reported for dorsal mEC GCs (Sargolini et al. 2006). The associated gridness score is 1.29. Running the model with real rat trajectory data (Hafting et al. 2008) also leads to gridness scores above 1. For a circular arena with a 1.80 m diameter our model was able to generate a perfect grid mapping covering almost the total extent of the environment, using a real rat trajectory and only 30 min of foraging time (Fig. 6c, d). As previously stated, it is reasonable to think that in a real trajectory, the animal would favor areas that were less explored, thus actively improving the grid field tessellation/PC recruitment procedure. The robustness of the model was also tested against three types of perturbations: (i) place field centers randomly distributed in arena space, (ii) noise in place field firing rates, and (iii) variability in place field area and shape modeled as bivariate Gaussians profiles with varying aspect ratio and orientation. If place field centers are displaced from the original (simplified) square lattice by a random vector, thereby producing a stochastic spatial distribution of place fields, the grid fields

Biol Cybern Fig. 7 Model robustness to firing rate noise and place field variability. The present model leads to high gridness scores independently of the place fields’ spatial distribution – square lattice (a) and stochastically distributed (b); side of section is 20 cm. c Gridness scores as a function of noise on place field firing rate (defined as a fraction of the signal amplitude). d Firing rate map of a simulated GC with heterogeneous place fields (level of variability of 24 %, in both area and aspect ratio); same animal’s trajectory as in Fig. 6a; gridness score was 1.16; scale bar is 0.50 m

and grid scores obtained from the model are largely unaffected (Fig. 7a, b). The proposed grid field formation mechanism is also robust to noisy place fields. The gridness score as a function of signal-to-noise ratio in the PCs’ firing rate is presented in Fig. 7c. Noisy place fields were modeled by adding zero mean Gaussian noise with a predefined amplitude (defined as a percentage of the signal amplitude) to the PCs’ firing rate profiles. Gridness scores are kept close to or above 1 for noise levels up to 30 % of the signal amplitude. The firing rate map obtained for a noise level of 30 %, and using the same animal’s trajectory as in Fig. 6, shows that all the nodes of the original map are properly formed under these conditions. With higher noise levels, place fields start to lose spatial consistency and are not properly recruited during grid field formation. This leads to one or more grid nodes not being formed during exploration, motivating the drop on gridness scores observed for noise levels above 40 %. Finally, the model was also tested in a scenario where place fields are not homogeneous. For this we allowed the scale parameters σx and σ y in the bivariate Gaussian function defining each place field to vary randomly but under

constraints for both area (σx × σ y ) and aspect ratio (σx /σ y ). Gaussian distributions were used to generate variability in area and aspect ratio, while a uniform distribution in the range [0, π [ was used to generate variability in the place field orientation. The grid field formation model copes with place field heterogeneity. This ability arises from the property that multiple place fields contribute to the formation of each grid node, allowing some degree of spatial averaging in place fields’ heterogeneity. For a level of variability in both area and aspect ratio of 24 % (set at 3σ of Gaussian distributions) the obtained map produced a gridness score of 1.16 (Fig. 7d). Throughout the experiments shown previously, the animal’s trajectory was the same (except in the case where real data were used) in order to facilitate comparison of manipulations. 4 Discussion In this model, PCs compete for GC activation. The excitatory and inhibitory pathways of a winning (selected) place cell ensure that a GC is forced to fire close to the center of the place field. GCs continuously recruit new PCs to cover the

123

Biol Cybern

space. The recruitment is only possible when the GC is freed from inhibitory action from PCs close to the present position. A key feature of our model, but also present in the model of Kropff and Treves [2008], is that path integration is not required to produce gridlike firing maps. It is important to emphasize the assumption of this model that GCs receive position information from the PC network. In turn, PCs receive spatial information from sources other than the GC network. Supporting this hypothesis is the finding that PC activity remains spatially stable after the disruption of the GC firing patterns (Koenig et al. 2011; Brandon et al. 2011). The role of PCs as the spatial information providers to GCs is also supported by studies stating that grid patterns almost disappear and their firing frequency is substantially diminished when hippocampal cells are deactivated (Bonnevie et al. 2013; Hafting et al. 2008). These results suggest that the grid patterns in mEC are strongly dependent on input from hippocampus. Therefore, this model presents a direct alternative to the assumption that summation of GC activity gives rise to PC firing (Solstad et al. 2006). Rate remapping of PCs is normally observed after small changes are made in the environment such as changing the color of the walls, while global remapping are a consequence of, for example, placing the same arena in different rooms or using arenas with different shapes (Fyhn et al. 2007; Barry et al. 2007). According to our model, for rate remapping, the connectivity strength maps learned previously are not expected to be disrupted, and consequently the grid field configuration is maintained. However, if the environment is now represented by a different set of PCs, then a new grid field map, with distinct phase and orientation, may be generated. Also, given the model dynamics, if a familiar environment is expanded, two situations may occur. If familiarity is maintained, the PCs that were active in the small arena will remain active, and new cells become active to encode the new available portion of the environment. In this situation, our model predicts that the grid field will not be disrupted and is extended to incorporate the new areas, in agreement with Hafting et al. [2005]. However, if the expanded arena is considered to be unfamiliar, then a different set of PCs encodes the space, and a new grid field is generated. A prediction of this model is the existence of interneurons with ring-shaped place fields (canonical place fields but inhibited at the center). Such a profile is absent from presently published experimental results, but data on the spatial modulation of hippocampal and entorhinal cortex interneurons is far from exhaustive. Another prediction of this model arises from the fact that the grid field is constructed by progressively recruiting new grid field nodes: given the summation of errors in the locations of newly recruited nodes, it is expected that the hexagonal lattice regularity will decrease as the grid field covered area increases. Beyond a certain environment area the grid field is expected to lose its fine hexagonal structure.

123

The model presented here for a single GC does not account for important population properties of a GC system such as the aforementioned constant spatial relationship between GC firing fields in different environments. However, the model can be extended to multiple grid maps. Assuming competition between GCs (possibly mediated by inhibitory interneurons), different nonoverlapping grid fields can be created. The grid fields would share the same spacing for the same level of mEC, which in turn would reflect the scale of the place fields. Synaptic connections could be introduced between GCs belonging to the same local population (in accordance with the scale of a recording site). There are no strong excitatory recurrent connections between principal cells in the mEC (Dhillon and Jones 2000), but the connections can be supported by local inhibitory neurons, providing inhibitory recurrent interactions (Couey et al. 2013). In this way, distinct GCs from the same recording location would cover distinct regions of the environment providing the homogeneous coverage experimentally reported (Fyhn et al. 2004). Head-direction cells are known to be present in mEC layers, intermingled with GCs, and are assumed to provide important directional information about navigation (Sargolini et al. 2006). We expect that head-direction cells may drive the different orientation of GCs along different mEC recording locations. Each local population could be fed by a head-direction preference, thereby producing GCs with nodes at different spatial locations but with similar orientations. Moreover, conjunctive cells are likely to appear as a side effect of this network architecture. Acknowledgments Supported by Fundação para a Ciência e a Tecnologia (FCT) through the Centro de Matemática da Universidade do Porto. Luísa Castro was supported by the Grant SFRH/BD/46329/2008 from FCT. The authors would like to thank the reviewers for important and insightful suggestions.

References Abarbanel HD, Huerta R, Rabinovich MI (2002) Dynamical model of long-term synaptic plasticity. Proc Natl Acad Sci USA 99(15):10132–10137. doi:10.1073/pnas.132651299 Acquas E, Wilson C, Fibiger HC (1996) Conditioned and unconditioned stimuli increase frontal cortical and hippocampal acetylcholine release: effects of novelty, habituation, and fear. J Neurosci Off J Soc Neurosci 16(9):3089–3096 Barry C, Ginzberg LL, O’Keefe J, Burgess N (2012) Grid cell firing patterns signal environmental novelty by expansion. Proc Natl Acad Sci USA 109(43):17687–17692. doi:10.1073/pnas.1209918109 Barry C, Hayman R, Burgess N, Jeffery KJ (2007) Experiencedependent rescaling of entorhinal grids. Nat Neurosci 10(6):682– 684. doi:10.1038/nn1905 Blair HT, Gupta K, Zhang K (2008) Conversion of a phase- to a ratecoded position signal by a three-stage model of theta cells, grid cells, and place cells. Hippocampus 18(12):1239–1255. doi:10.1002/hipo. 20509 Boccara CN, Sargolini F, Thoresen VH, Solstad T, Witter MP, Moser EI, Moser MB (2010) Grid cells in pre- and parasubiculum. Nat Neurosci 13(8):987–994. doi:10.1038/nn.2602

Biol Cybern Bonnevie T, Dunn B, Fyhn M, Hafting T, Derdikman D, Kubie JL, Roudi Y, Moser EI, Moser MB (2013) Grid cells require excitatory drive from the hippocampus. Nat Neurosci 16(3):309–317. doi:10. 1038/nn.3311 Brandon MP, Bogaard AR, Libby CP, Connerney MA, Gupta K, Hasselmo ME (2011) Reduction of theta rhythm dissociates grid cell spatial periodicity from directional tuning. Science 332(6029):595– 599. doi:10.1126/science.1201652 Burak Y, Fiete IR (2009) Accurate path integration in continuous attractor network models of grid cells. PLoS Comput Biol 5(2):e1000291. doi:10.1371/journal.pcbi.1000291 Burgess N, Barry C, O’Keefe J (2007) An oscillatory interference model of grid cell firing. Hippocampus 17(9):801–812. doi:10.1002/hipo. 20327 Castro L, Aguiar P (2012) Phase precession through acceleration of local theta rhythm: a biophysical model for the interaction between place cells and local inhibitory neurons. J Comput Neurosci 33(1):141–150. doi:10.1007/s10827-011-0378-0 Couey JJ, Witoelar A, Zhang SJ, Zheng K, Ye J, Dunn B, Czajkowski R, Moser MB, Moser EI, Roudi Y, Witter MP (2013) Recurrent inhibitory circuitry as a mechanism for grid formation. Nat Neurosci 16(3):318–324. doi:10.1038/nn.3310 Hines ML, Morse T, Migliore M, Carnevale NT, Shepherd GM (2004) ModelDB: a database to support computational neuroscience. J Comput Neurosci 17(1):7–11. doi:10.1023/B:JCNS.0000023869.22017. 2e Dhillon A, Jones RS (2000) Laminar differences in recurrent excitatory transmission in the rat entorhinal cortex in vitro. Neuroscience 99(3):413–422 Freund TF, Gulyas AI (1997) Inhibitory control of GABAergic interneurons in the hippocampus. Can J Physiol Pharmacol 75(5):479– 487 Fuhs MC, Touretzky DS (2006) A spin glass model of path integration in rat medial entorhinal cortex. J Neurosci Off J Soc Neurosci 26(16):4266–4276. doi:10.1523/JNEUROSCI.4353-05.2006 Fyhn M, Hafting T, Treves A, Moser MB, Moser EI (2007) Hippocampal remapping and grid realignment in entorhinal cortex. Nature 446(7132):190–194. doi:10.1038/nature05601 Fyhn M, Molden S, Witter MP, Moser EI, Moser MB (2004) Spatial representation in the entorhinal cortex. Science 305(5688):1258– 1264. doi:10.1126/science.1099901 Giocomo LM, Zilli EA, Fransen E, Hasselmo ME (2007) Temporal frequency of subthreshold oscillations scales with entorhinal grid cell field spacing. Science 315(5819):1719–1722. doi:10.1126/science. 1139207 Grossberg S, Pilly PK (2012) How entorhinal grid cells may learn multiple spatial scales from a dorsoventral gradient of cell response rates in a self-organizing map. PLoS Comput Biol 8(10):e1002648. doi:10. 1371/journal.pcbi.1002648 Guanella A, Kiper D, Verschure P (2007) A model of grid cells based on a twisted torus topology. Int J Neural Syst 17(4):231–240 Hafting T, Fyhn M, Bonnevie T, Moser MB, Moser EI (2008) Hippocampus-independent phase precession in entorhinal grid cells. Nature 453(7199):1248–1252. doi:10.1038/nature06957 Hafting T, Fyhn M, Molden S, Moser MB, Moser EI (2005) Microstructure of a spatial map in the entorhinal cortex. Nature 436(7052):801–806. doi:10.1038/nature03721 Hasselmo ME (2006) The role of acetylcholine in learning and memory. Curr Opin Neurobiol 16(6):710–715. doi:10.1016/j.conb.2006.09. 002 Hasselmo ME, Giocomo LM, Zilli EA (2007) Grid cell firing may arise from interference of theta frequency membrane potential oscillations in single neurons. Hippocampus 17(12):1252–1271. doi:10.1002/ hipo.20374

Heys JG, Schultheiss NW, Shay CF, Tsuno Y, Hasselmo ME (2012) Effects of acetylcholine on neuronal properties in entorhinal cortex. Front Behav Neurosci 6:32. doi:10.3389/fnbeh.2012.00032 Jung MW, Wiener SI, McNaughton BL (1994) Comparison of spatial firing characteristics of units in dorsal and ventral hippocampus of the rat. J Neurosci Off J Soc Neurosci 14(12):7347–7356 Karlsson MP, Frank LM (2008) Network dynamics underlying the formation of sparse, informative representations in the hippocampus. J Neurosci Off J Soc Neurosci 28(52):14271–14281. doi:10.1523/ JNEUROSCI.4261-08.2008 Kjelstrup KB, Solstad T, Brun VH, Hafting T, Leutgeb S, Witter MP, Moser EI, Moser M-B (2008) Finite Scale of Spatial Representation in the Hippocampus. Science 321(5885):140–143. doi:10.1126/ science.1157086 Koenig J, Linder AN, Leutgeb JK, Leutgeb S (2011) The spatial periodicity of grid cells is not sustained during reduced theta oscillations. Science 332(6029):592–595. doi:10.1126/science.1201685 Kropff E, Treves A (2008) The emergence of grid cells: Intelligent design or just adaptation? Hippocampus 18(12):1256–1269. doi:10. 1002/hipo.20520 Langston RF, Ainge JA, Couey JJ, Canto CB, Bjerknes TL, Witter MP, Moser EI, Moser MB (2010) Development of the spatial representation system in the rat. Science 328(5985):1576–1580. doi:10.1126/ science.1188210 McNaughton BL, Battaglia FP, Jensen O, Moser EI, Moser MB (2006) Path integration and the neural basis of the ’cognitive map’. Nat Rev Neurosci 7(8):663–678. doi:10.1038/nrn1932 Mhatre H, Gorchetchnikov A, Grossberg S (2012) Grid cell hexagonal patterns formed by fast self-organized learning within entorhinal cortex. Hippocampus 22(2):320–334. doi:10.1002/hipo.20901 Moser EI, Moser MB (2008) A metric for space. Hippocampus 18(12):1142–1156. doi:10.1002/hipo.20483 Muller RU, Kubie JL, Ranck JB Jr (1987) Spatial firing patterns of hippocampal complex-spike cells in a fixed environment. J Neurosci Off J Soc Neurosci 7(7):1935–1950 Navratilova Z, Giocomo LM, Fellous JM, Hasselmo ME, McNaughton BL (2012) Phase precession and variable spatial scaling in a periodic attractor map model of medial entorhinal grid cells with realistic after-spike dynamics. Hippocampus 22(4):772–789. doi:10.1002/ hipo.20939 Quirk G, Muller R, Kubie J, Ranck J (1992) The positional firing properties of medial entorhinal neurons: description and comparison with hippocampal place cells. J Neurosci 12(5):1945–1963 Sargolini F, Fyhn M, Hafting T, McNaughton BL, Witter MP, Moser MB, Moser EI (2006) Conjunctive representation of position, direction, and velocity in entorhinal cortex. Science 312(5774):758–762. doi:10.1126/science.1125572 Si B, Kropff E, Treves A (2012) Grid alignment in entorhinal cortex. Biol Cybern 106(8–9):483–506. doi:10.1007/s00422-012-0513-7 Solstad T, Moser EI, Einevoll GT (2006) From grid cells to place cells: a mathematical model. Hippocampus 16(12):1026–1031. doi:10. 1002/hipo.20244 Stensola H, Stensola T, Solstad T, Froland K, Moser MB, Moser EI (2012) The entorhinal grid map is discretized. Nature 492(7427):72– 78. doi:10.1038/nature11649 Wills TJ, Cacucci F, Burgess N, O’Keefe J (2010) Development of the hippocampal cognitive map in preweanling rats. Science 328(5985):1573–1576. doi:10.1126/science.1188224 Zilli EA, Hasselmo ME (2010) Coupled noisy spiking neurons as velocity-controlled oscillators in a model of grid cell spatial firing. J Neurosci Off J Soc Neurosci 30(41):13850–13860. doi:10.1523/ JNEUROSCI.0547-10.2010

123

A feedforward model for the formation of a grid field where spatial information is provided solely from place cells.

Grid cells (GCs) in the medial entorhinal cortex (mEC) have the property of having their firing activity spatially tuned to a regular triangular latti...
3MB Sizes 1 Downloads 2 Views