Space-variant interconnections based on diffractive optical elements for neural networks: architectures and cross-talk reduction.

Optical architectures for fully connected and limited-fan-out space-variant weighted interconnections based on diffractive optical elements for fixed-connection multilayer neural networks are investigated and compared in terms of propagation lengths, system volumes, connection densities, and interconnection cross talk. For a small overall system volume the limited-fan-out architecture can accommodate a much larger number of input and output nodes. However, the interconnection cross talk of the limited-fan-out space-variant architecture is relatively high owing to noise from the diffractive-optical-element reconstructions. Therefore a cross-talk reduction technique based on a modified design procedure for diffractive optical elements is proposed. It rearranges the reconstruction pattern of the diffractive optical elements such that less noise lands on each detector region. This technique is verified by the simulation of one layer of an interconnection system with 128 x 128 input nodes, 128 x 128 output nodes, and weighted connections that fan out from each input node to the nearest 5 x 5 array of output nodes. In addition to a significant cross-talk reduction, this technique can reduce the propagation length and system volume.

Holographic grating formation in photopolymers.

We introduce a model describing real-time grating formation in holographic photopolymers, under the assumption that the diffusion of free monomers is much faster than the grating formation. This model, which combines polymerization kinetics with results from coupled-wave theory, indicates that the grating formation time depends sublinearly on the average holographic recording intensity, and the beam intensity ratio controls the grating index modulation at saturation. We validate the model by comparing its predictions with the results of experiments in which DuPont HRF-150X001 photopolymer was used.

Convergence of backward-error-propagation learning in photorefractive crystals.

We analytically determine that the backward-error-propagation learning algorithm has a well-defined region of convergence in neural learning-parameter space for two classes of photorefractive-based optical neural-network architectures. The first class uses electric-field amplitude encoding of signals and weights in a fully coherent system, whereas the second class uses intensity encoding of signals and weights in an incoherent/coherent system. Under typical assumptions on the grating formation in photorefractive materials used in adaptive optical interconnections, we compute weight updates for both classes of architectures. Using these weight updates, we derive a set of conditions that are sufficient for such a network to operate within the region of convergence. The results are verified empirically by simulations of the xor sample problem. The computed weight updates for both classes of architectures contain two neural learning parameters: a learning-rate coefficient and a weight-decay coefficient. We show that these learning parameters are directly related to two important design parameters: system gain and exposure energy. The system gain determines the ratio of the learning-rate parameter to decay-rate parameter, and the exposure energy determines the size of the decay-rate parameter. We conclude that convergence is guaranteed (assuming no spurious local minima in the error function) by using a sufficiently high gain and a sufficiently low exposure energy per weight update.

Gain and exposure scheduling to compensate for photorefractive neural-network weight decay.

A gain and exposure schedule that theoretically eliminates the effect of photorefractive weight decay for the general class of outer-product neural-network learning algorithms (e.g., backpropagation, Widrow-Hoff, perceptron) is presented. This schedule compensates for photorefractive diffraction-efficiency decay by iteratively increasing the spatial-light-modulator transfer function gain and decreasing the weight-update exposure time. Simulation results for the scheduling procedure, as applied to backpropagation learning for the exclusive-OR problem, show improved learning performance compared with results for networks trained without scheduling.

Optical parallel-access shared memory system: analysis and experimental demonstration.

An optical implementation of a parallel-access shared memory uses a single photorefractive crystal and can realize the set of memory modules in a digital shared memory computer. This implementation uses two arrays of sources that are individually coherent but mutually incoherent from region to region across each array, and it permits incoherent/coherent double angular multiplexing of data in the crystal. A complete instruction set for its memory access consists of four operations, READ, WRITE, SELECTIVE ERASE, and REFRESH, which can be applied to any memory module independent of (and parallel with) instructions to the other memory modules. In addition, a memory module can execute a sequence of READ operations simultaneously with the execution of a WRITE operation to accommodate differences efficiently in optical recording and readout times common to optical volume storage media. An experimental shared memory system demonstrates two memory modules, each consisting of up to two 5-bit data blocks, implemented in a single Fe:LiNbO(3) crystal. The projected performance of the optical parallel-access shared memory system is analyzed and compared with conventional page-addressed volume holographic memories.

Passive optical interconnection network employing a shuffle-exchange topology.

The design of the optical interconnection network (OIN) system is presented. The network is implemented by the use of only passive optical elements and is based on a shuffle-exchange interconnection pattern. Each passive optical shuffle-exchange stage is designed for cascadability. The OIN switch nodes are capable of broadcast and combine operations in addition to bypass and exchange. The OIN is designed to be controlled by an electronic address computer in a circuit-switched manner. Although the OIN is designed to be used as a subsystem on the shared-memory optical/electronic computer, it may be used as a complete subsystem in other communication or computing architectures.

Shared-memory optical/electronic computer: architecture and control.

The design of a parallel digital computer architecture, the shared-memory optical/electronic computer (SMOEG), and its associated control algorithms are presented. The design is based on the shared-memory model of computation and incorporates an optical interconnection network as an essential element. The arthitecture consists of a novel passive optical shuffle-exchange network, which is detailed in another paper [Appl. Opt. 33, (1994)],.that interconnects electronic processing elements with electronic memory modules and incorporates network control. Improved capability of this optical-electronic multiple-instruction multiple-data (MIMD) architecture over fully electronic implementations stems from the reduced complexity inherent in the optical interconnection network and the resulting memory access capability. In this system the simultaneous development of three main design facets, architecture, hardware, and control algorithms, is crucial in designing an efficient high-performance system.

Digital optical cellular image processor (DOCIP): experimental implementation.

We demonstrate experimentally the concept of the digital optical cellular image processor architecture by implementing one processing element of a prototype optical computer that includes a 54-gate processor, an instruction decoder, and electronic input-output interfaces. The processor consists of a twodimensional (2-D) array of 54 optical logic gates implemented by use of a liquid-crystal light valve and a 2-D array of 53 subholograms to provide interconnections between gates. The interconnection hologram is fabricated by a computer-controlled optical system.

Analysis of weighted fan-out/fan-in volume holographic optical interconnections.

The feasibility of employing volume holographic techniques for the implementation of highly multiplexed weighted fan-out/fan-in interconnections is analyzed on the basis of interconnection fidelity, optical throughput, and complexity of recording schedule or implementation hardware. These feasibility criteria were quantitatively evaluated using the optical beam propagation method to numerically simulate the diffraction characteristics of volume holographic interconnections recorded in a linear holographic material. We find that conventional interconnection architectures (that are based on a single coherent optical source) exhibit a direct trade-off between interconnection fidelity and optical throughput on the one hand, and recording schedule or hardware complexity on the other. In order to circumvent this trade-off we describe and analyze in detail an incoherent/coherent double angularly multiplexed interconnection architecture that is based on the use of multiple-source array of individually coherent but mutually incoherent sources. This architecture either minimizes or avoids several key sources of cross talk, permits simultaneous recording of interconnection weights or weight updates, and provides enhanced fidelity, interchannel isolation, and thróughput performance.

Visual cortex operations and their implementation using the incoherent optical neuron model.

The incoherent optical neuron model uses two different device responses, an inhibitory response and a nonlinear output response, to realize a complete neuron unit that has both inhibitory and excitatory inputs. We describe its use to implement a model of simple cells of the visual cortex. Such simple cells perform the operations of edge detection, orientation selection, and in the case of moving objects, direction and speed selection. Experiments are described that utilize two Hughes liquid-crystal light valves to perform the functions of input transduction and optical neuron unit-array implementation. A multiplexed dichromated gelatin hologram serves as a holographic optical element that forms space-invariant (but otherwise arbitrary) point-spread functions for the network interconnections. Changing the holographic interconnection pattern permits implementation of different simple cells performing, e.g., transient response, edge detection, orientation preference, and direction and speed preference. Experimental results of these operations are presented.

Incoherent dynamic lenslet array processor.

The use of a dynamic lenslet array processor for the implementation of unipolar and bipolar analog inner product, outer product, and vector sum operations is described. Its matrix-vector operations are used as a basis for neural networks and digital circuits. Experimental results of two circuits are presented: a unipolar neural network that computes parity of a 3-bit input word and a digital 3-to-8 decoder circuit.

Single-step copying process for multiplexed volume holograms.

A technique for copying the index or absorption modulation of a multiplexed volume holographic element into a second volume holographic medium is presented. The technique utilizes a set of coherent but mutually incoherent optical sources and can perform the copy process in a single exposure step. The concept is demonstrated experimentally for the case of three angularly multiplexed holographic gratings in dichromated gelatin.

Optical computing: introduction by the feature editors.

This feature of Applied Optics includes papers on components, subsystems, and systems for analog and digital optical computing. Many of the papers are an outgrowth of papers presented at the Fourth Topical Meeting on Optical Computing, March 1991, in Salt Lake City. This introduction first gives an overview of the evolution of recent optical-computing research and then an overview of the feature issue papers.

Digital optical processor based on symbolic substitution using holographic matched filtering.

We propose a digital optical arithmetic processor design based on symbolic substitution using holographic matched and space-invariant filters. The proposed system performs Boolean logic, binary addition, and subtraction in a highly parallel manner; i.e., the processing time depends on word size but not array size. Algorithms for performing binary addition and subtraction in parallel are presented. A skew problem occurring when symbolic substitution is applied to binary addition and subtraction with space-invariant systems is addressed, and its solution is suggested. Crosstalk in symbolic substitution is described, and new symbols which can prevent the crosstalk are introduced. System analysis and fundamental limitations of the proposed system are also presented in terms of processing time, overall light efficiency, and the maximum array size of the input data plane. The performance of the proposed system with that of the current electronic supercomputers has been compared by combining information about the processing time and maximum array size.

Subtracting incoherent optical neuron model: analysis, experiment, and applications.

To fully use the advantages of optics in optical neural networks, an incoherent optical neuron (ION) model is proposed. The main purpose of this model is to provide for the requisite subtraction of signals without the phase sensitivity of a fully coherent system and without the cumbrance of photon-electron conversion and electronic subtraction. The ION model can subtract inhibitory from excitatory neuron inputs by using two device responses. Functionally it accommodates positive and negative weights, excitatory and inhibitory inputs, non-negative neuron outputs, and can be used in a variety of neural network models. This technique can implement conventional inner-product neuron units and Grossberg's mass action law neuron units. Some implementation considerations, such as the effect of nonlinearities on device response, noise, and fan-in/fan-out capability, are discussed and simulated by computer. An experimental demonstration of optical excitation and inhibition on a 2-D array of neuron units using a single Hughes liquid crystal light valve is also reported.

Image algebra representation of parallel optical binary arithmetic.

A binary image algebra (BIA) that gives a mathematical description of parallel processing operations is described. Rigorous and concise BIA representations of parallel arithmetic and symbolic substitution operations are given. A sequence of programming steps for implementation of these operations on a parallel architecture is specified by the BIA representation. Examples of arithmetic operations implemented on a digital optical cellular image processor architecture are given.

Model for an incoherent optical neuron that subtracts.

An incoherent optical neuron is proposed that subtracts inhibitory inputs from excitatory inputs optically by utilizing two separate device responses. Functionally it accommodates positive and negative weights, excitatory and inhibitory inputs, and nonnegative neuron outputs, and it can be used in a variety of neural network models. An extension is given to include bipolar neuron outputs in the case of fully connected networks.