Experimental quantum speed-up in reinforcement learning agents.

As the field of artificial intelligence advances, the demand for algorithms that can learn quickly and efficiently increases. An important paradigm within artificial intelligence is reinforcement learning1, where decision-making entities called agents interact with environments and learn by updating their behaviour on the basis of the obtained feedback. The crucial question for practical applications is how fast agents learn2. Although various studies have made use of quantum mechanics to speed up the agent's decision-making process3,4, a reduction in learning time has not yet been demonstrated. Here we present a reinforcement learning experiment in which the learning process of an agent is sped up by using a quantum communication channel with the environment. We further show that combining this scenario with classical communication enables the evaluation of this improvement and allows optimal control of the learning progress. We implement this learning protocol on a compact and fully tunable integrated nanophotonic processor. The device interfaces with telecommunication-wavelength photons and features a fast active-feedback mechanism, demonstrating the agent's systematic quantum advantage in a setup that could readily be integrated within future large-scale quantum communication networks.

An antisense investigation of the role of the gamma-aminobutyric acidB1 receptor subunit in Ca2+ channel modulation in rat sensory neurones.

Activation of gamma-aminobutyric acid(B) (GABA(B)) receptors in dorsal root ganglion (DRG) neurones leads to inhibition of calcium (Ca(2+)) channels. The role of the GABA(B1) receptor subunit was assessed by its depletion achieved by microinjection of DRG neurones with an antisense (A/S) oligodeoxynucleotide (ODN). Control neurones were injected with a scrambled version of the A/S ODN (missense) or were not injected. Patch clamp recordings of Ca(2+) channel current were made two to four days after injection. GABA(B1) A/S substantially reduced the current inhibition induced by baclofen, a GABA(B) agonist. Therefore, most, if not all, native GABA(B) receptors which couple to Ca(2+) channels contain GABA(B1). Moreover, if native receptors are heterodimers of GABA(B1) and GABA(B2), then GABA(B2), in isolation, is unable to sustain coupling to Ca(2+) channels.

Mechanism of block by ZD 7288 of the hyperpolarization-activated inward rectifying current in guinea pig substantia nigra neurons in vitro.

1. The effects of the novel bradycardic agent 4-(N-ethyl-N-phenylamino)-1,2-dimethyl-6-(methylamino) pyrimidinium chloride (ZD 7288) (Zeneca) were investigated on the hyperpolarization-activated cationic current (Ih) in guinea pig substantia nigra pars compacta neurons in vitro, using a single-microelectrode current-clamp/voltage-clamp technique. 2. Under current-clamp conditions, injection of large negative current pulses (0.1-0.5 nA, 400 ms) evoked a slow depolarizing "sag" in the electrotonic potential due to activation of the slow inward (anomalous) rectifier. In voltage-clamp recordings, hyperpolarizing voltage steps from a holding potential of -60 mV (close to resting potential) elicited slow inward current relaxations with kinetic properties similar to those seen for other neuronal Ihs. 3. ZD 7288 (10-100 microM) produced a consistent abolition of the electrotonic potential sag with no effect on membrane potential or spike properties. Under voltage clamp, Ih amplitude was clearly reduced in a time- and concentration-dependent manner (apparent half-maximum blocking concentration = 2 microM); full block of Ih was typically achieved after 10-15 min of exposure to 50 microM ZD 7288, with no significant recovery observed after 1 h of washing. 4. A similar (although more rapid) block of Ih was seen after application of 3-5 mM Cs+ (partially reversible after 30 min of washing). 5. Partial block of Ih by 10 microM ZD 7288 was accompanied by a reduction in the maximum amplitude of the Ih activation curve, a small negative shift in its position on the voltage axis, and a linearization of the steady-state current-voltage relationship. The estimated Ih reversal potential, however, remained unaffected. 6. In 10 microM ZD 7288, the time course of Ih activation and deactivation was significantly slowed (within the range of -70 to -120 mV for the activation time constant and -70 to -90 mV for the inactivation time constant). 7. Blockade of Ih by ZD 7288 or Cs+ was independent of prior Ih activation (i.e., non-use dependent). 8. Intracellular loading with ZD 7288 also abolished the sag in the electrotonic voltage response and Ih relaxations, suggesting an intracellular site of action. By contrast, intracellular Cs+ had no effect on Ih properties. 9. Block of Ih by ZD 7288 (but not Cs+) was relieved by prolonged cell hyperpolarization, manifested as a slowly developing (half-time approximately 20 s) inward current at a holding potential of -100 mV. 10. We propose that ZD 7288, when applied externally, may behave as a "lipophilic" quaternary cation, capable of passing into the cell interior to block Ih channels in their closed state; this compound may thus prove a useful research tool, in place of Cs+, for studying the properties and significance of Ih currents in controlling neuronal function.

Selective blockade of the hyperpolarization-activated cationic current (Ih) in guinea pig substantia nigra pars compacta neurones by a novel bradycardic agent, Zeneca ZM 227189.

The effects of a novel bradycardic agent Zeneca ZM 227189 (4-(N-ethyl-N-phenylamino)-1,2-dimethyl-6-(methylamino) triazinium iodide) were tested on the inward rectifying properties of guinea-pig substantia nigra pas compacta (SNC) and guinea-pig olfactory cortical cells recorded in vitro. In SNC neurones, ZM 227189 (10-100 microM) produced a dose-dependent block of the slow anomalous rectifier; under voltage clamp, a clear reduction was seen in the amplitude of the slow inward current (Ih) relaxation evoked by negative voltage commands from a holding potential of -60 mV. ZM 227189 (50-100 microM) induced an irreversible block of the Ih current after 10-15 min exposure. A similar block of Ih was observed following application of 5 mM Cs+. ZM 227189 had little effect on other membrane properties. By contrast, in olfactory cortical neurones, ZM 227189 (100 microM) induced an increase in the input resistance (approximately 20%) and cell excitability, accompanied by a small (< 2 mV) hyperpolarization; these effects were also not reversible. Activation of the fast (K(+)-mediated) inward rectifier at negative membrane potentials remained unaffected. Lower concentrations (1-10 microM) of ZM 227189 had no obvious effect on cortical cell properties. Our data indicate that ZM 227189 is a potent and apparently selective blocker of Ih in substantia nigra neurones, but has no effect on the fast-type inward rectifier in olfactory cortical cells.

Pharmacological characterisation of excitatory synaptic transmission in the cat red nucleus in vivo.

Extracellular recordings were made from the magnocellular neurones of the red nucleus (mRN) in anaesthetised cats. A study was made of the effects of selective excitatory amino acid receptor antagonists on excitatory monosynaptic responses evoked from the sensorimotor cortex (SMC) and cerebellar interpositus nucleus (IPN). Iontophoretically applied CNQX and NBQX antagonised both SMC and IPN responses whereas, D-AP5 inhibited the SMC response but was ineffective to the IPN. At currents that selectively antagonised NMDA responses, CPPene had no effect on either SMC or IPN responses. 7-chlorokynurenate inhibited both SMC and IPN responses but required currents that antagonised both AMPA and NMDA responses and was therefore acting in a non-selective manner. Iontophoretically applied glycine was inhibitory to both agonist and synaptic responses, whilst D-serine potentiated NMDA responses but did not enhance monosynaptic responses of the SMC. However in the presence of either 7-chlorokynurenate or high currents of CNQX that reduced the SMC synaptic activation of the mRN neurones, D-serine attenuated the inhibitory action of these antagonists. It is concluded that monosynaptic responses from the SMC are mediated by both NMDA and non-NMDA receptors whereas the monosynaptic responses evoked from the IPN are mediated only by non-NMDA receptors. The lack of effect of CPPene is consistent with the postulate that two NMDA receptor subtypes are present on mRN neurones.

Sensitivity of transient outward rectification to ion channel blocking agents in guinea-pig substantia nigra pars compacta neurones in vitro.

The sensitivity of a transient outward rectifier in guinea-pig substantia nigra pars compacta (SNC) neurones in vitro was tested to a variety of ion channel blocking agents. This rectification was not blocked by TEA (10 mM), 4-aminopyridine (4-AP) (5 mM), apamin (0.2-1 microM) or Cd2+ ions (200 microM). Cs+ ions (5 mM) blocked the inward 'anomalous' rectifier but had no effect on the transient outward rectification. However, replacing Ca2+ ions in the bath perfusate by Ba2+ blocked the transient outward but not the inward rectifier. It is suggested that an atypical barium-sensitive conductance is responsible for the transient outward rectification of these neurones.

Cortically evoked excitatory synaptic transmission in the cat red nucleus is antagonised by D-AP5 but not by D-AP7.

Extracellular recordings were made from magnocellular red nucleus neurons (mRN) in alpha-chloralose (50 mg/kg, iv.) anaesthetised cats. Iontophoretically applied N-methyl-D-aspartate (NMDA) excited the neuronal firing which was antagonised by 4 selective NMDA receptor antagonists: 2-amino-5-phosphonopentanoate (AP5), 2-amino-7-phosphonoheptanoate (AP7), RS-4-(phosphonomethyl) piperazine-2-carboxylic acid (PMPC) and R-4-(3-phosphonopropyl) piperazine-2-carboxylic acid (CPP), whereas AMPA responses were uneffected. Monosynaptic excitatory responses were produced by stimulation of the sensorimotor cortex. These responses were reduced and often abolished by AP5 and PMPC but not by AP7 or CPP. It is postulated that two NMDA receptor subtypes exist on mRN neurones.

The electrophysiological properties of substantia nigra pars compacta neurones recorded from 6-hydroxydopamine lesioned guinea-pigs in vitro.

Intracellular recordings were made from substantia nigra pars compacta neurones in vitro from animals with partial unilateral 6-hydroxydopamine lesions of the nigrostriatal tract. Lesions were assessed and grouped according to the severity of the striatal dopamine depletion. No differences were seen between neurones from control and lesioned side nigrae as regards their membrane properties, firing rates, burst activity or percentage of quiescent neurones in any of the lesioned categories. It is concluded that following partial lesioning, the remaining substantia nigra zona compacta neurones in vitro, are functioning normally.

A possible pacemaker mechanism in pars compacta neurons of the guinea-pig substantia nigra revealed by various ion channel blocking agents.

The membrane properties of pars compacta neurons in the in vitro guinea-pig substantia nigra have been studied in the presence of sodium, calcium and potassium channel blockers. The following properties, which have already been described for dopamine-containing substantia nigra zona compacta neurons were observed: high and low threshold calcium spikes; a calcium-activated potassium-mediated transient; inward rectification. Inward rectification was sensitive to caesium ions. An additional property was seen reminiscent of an "A" current, although resistant to 4-aminopyridine. It is suggested that this outward transient is in fact a calcium activated potassium conductance. Under certain conditions calcium-mediated rhythmic depolarizations were observed. It is suggested that at least two of the properties seen (outward rectification and low threshold calcium spike) could interact to provide the basis for a pacemaker mechanism in pars compacta neurons.

The effects of gamma-hydroxybutyrate on the membrane properties of guinea-pig pars compacta neurons in the substantia nigra in vitro.

The effects of gamma-hydroxybutyrate have been studied on the membrane properties of pars compacta neurons within the guinea-pig substantia nigra maintained in vitro. The effects of gamma-hydroxybutyrate are to (i) lower input resistance, (ii) hyperpolarize the cell membrane in a dose-dependent manner and (iii) facilitate calcium conductances. These effects are resistant to the blockade of sodium channels with tetrodotoxin and blockade of potassium channels with tetraethylammonium or 4-aminopyridine. Furthermore, these effects are only partially blocked by high doses of the GABA receptor antagonist bicuculline: indeed the effects of the GABA receptor agonist muscimol can be differentiated from those of gamma-hydroxybutyrate in that the latter is sensitive to application of barium ions. The results suggest that gamma-hydroxybutyrate acts to increase utilization of calcium, which in turn leads to an initiation of calcium-dependent events. The functional consequences of these effects of gamma-hydroxybutyrate are discussed with regard to its possible endogenous modulatory actions.

Electrophysiological evidence for the dendritic localization of a calcium conductance in guinea-pig substantia nigra neurones in vitro.

Within the substantia nigra, anatomical, neurochemical and pharmacological findings strongly suggest that transmitter and protein are secreted from the dendrites of nigrostriatal neurones. This phenomenon may underlie a non classical modulatory cellular mechanism. Two conductances are generated in nigrostriatal neurones independent of somatic action potentials, that might mediate this modulation. However, these conductances have never been directly nor precisely located specifically within the dendrites. The aim of this study was to record the membrane properties of substantia nigra zona compacta neurones in response to selective sectioning of the population of long 'apical' dendrites i.e. the removal of the zona reticulata. Intracellular recordings from substantia nigra zona compacta neurones were made from mesencephalic slices of the guinea-pig brain maintained in vitro. In cells without the apical dendrites, the membrane potential, input resistance and mean firing frequency was not significantly different from the control neurones. However, removal of the substantia nigra zona reticulata virtually abolished one conductance in particular. This conductance, seen in control neurones, is a long lasting slow depolarization which is resistant to tetrodotoxin blockade of sodium channels: rather, it is mediated by the entry of calcium ions and is optimally deinactivated at a hyperpolarised membrane potential. Hence, this study strongly suggests that this conductance is generated exclusively in the 'apical' dendrites. It has been postulated that this long lasting calcium conductance is central to the modulation of nigrostriatal neuronal excitability. Thus, the 'apical' dendrites could play a specific and active role in the functioning of nigrostriatal neurones.

Mustard oil excites but does not inhibit nociceptive dorsal horn neurones in the rat: a presumed effect on A-delta fibres.

1. Mustard oil was applied topically in concentrations of 10-20% to the excitatory and inhibitory nociceptive receptive fields in glabrous and hairy skin of the anesthetized rat while recording the activity of nociceptive dorsal horn neurones. The noxious stimulus was radiant heat which is known to activate C-fibres in glabrous and hairy skin. 2. Mustard oil had little effect when applied to glabrous skin and this was attributed to poor penetration of the skin. 3. Mustard oil excited cells in the dorsal horn which were excited by noxious heat in the receptive field on hairy skin. 4. Mustard oil excited cells in the dorsal horn which were inhibited by noxious heat in the receptive field on hairy skin. 5. Inhibitory effects of mustard oil were never seen, even when applied to receptive fields in which noxious heating caused inhibition. 6. The excitatory effects of mustard oil on cells inhibited by noxious heating of the skin are attributed to the reported activation of A-delta fibres which probably masked any C-fibre activation.

Opiates distinguish spinal excitation from inhibition evoked by noxious heat stimuli in the rat: relevance to theories of analgesia.

1. Experiments were performed to test the hypothesis that a significant part of the action of opiates in reducing responses to noxious stimuli is a reduction in the release of neurotransmitter from primary afferent fibres. 2. The effects of locally and systemically administered opiates were examined on the excitatory and gamma-aminobutyric acid (GABA)-mediated inhibitory responses of spinal dorsal horn neurones to noxious heat stimulation in the anaesthetized rat: the inhibitions are thought to involve the same C-fibre afferents as the excitation. 3. Microionophoretically administered morphine reduced the excitatory response i a small proportion of the cells, reduced the background firing in a larger proportion but was ineffective on the inhibition. 4. Intravenously injected morphine (0.5-6 mg kg-1) or etorphine (0.1-2 micrograms kg-1) invariably attenuated the excitation of dorsal horn neurones by noxious stimuli but had no effect on the inhibition. 5. It was concluded that the data do not support the hypothesis that the production of analgesia is due mainly to a reduction in the release of transmitter from primary afferent fibres.

Zetidoline blocks dopamine autoreceptors in the rat substantia nigra.

Zetidoline, applied microiontophoretically onto neurones in the substantia nigra, zona compacta of anaesthetized rats, reversibly blocked the depressant effect of iontophoretically-applied dopamine. The inhibitory actions of gamma-aminobutyric acid (GABA) and glycine and the excitatory action of glutamate were unaffected by the antagonist. Zetidoline applied by bath perfusion, was similarly an antagonist of the inhibitory effect of bath applications of dopamine on neurones in the zona compacta of slices of substantia nigra; the pA2 for zetidoline in this preparation was 7.0.