Stage-Distributed Time-Division Permutation Routing in a Multistage Optically Interconnected Switching Fabric

1. {c2, id} c3 c4 input c4 c3 output c1 c2 c2 c1 input plane 1 1 1 1 1 6 4 6 6 input 5 8 5 5 0.9 0.9 0.9 0.9 0.9 5 4 cross crossbar crossbar 0.8 0.8 0.8 4 0.8 0.8 4 I4 6 0.7 0.7 0.7 0.7 0.7 3 straight switching region 3 0.6 0.6 0.6 0.6 0.6 3 Interconnect 1 Interconnect 2 Interconnect 3 Interconnect N Interconnect 1 Interconnect 2 Interconnect 3 Interconnect N Interconnect 1 Interconnect 2 Interconnect 3 Interconnect N SEMIN output plane 0.5 0.5 0.5 0.5 0.5 Buffer size Buffer size Probability of packet acceptance Probability of packet acceptance 2 L2 E2 0 0.4 0.4 (b) 0.4 0.4 0.4 (a) 2 2 GSMIN {c4,id} 1 1 0.3 0.3 0.3 0.3 0.3 Size: 128 x 128 Size: 128 x 128 {c3, id} 0 2 {c2, id} 0.2 0.2 0.2 0.2 0.2 1 1 SEMIN (alternate) GSMIN (alternate) Multistage architecture parallel computers switching networks 3 {c1,id} (2) SEMIN/GSMIN (forced alternate) 0.1 0.1 0.1 0.1 GSMIN (forced alternate) 0.1 0 0 0 0 0 0 0 0 0 0.1 0.1 0.1 0.1 0.2 0.2 0.2 0.2 0.3 0.3 0.3 0.3 0.4 0.4 0.4 0.4 0.5 0.5 0.5 0.5 0.6 0.6 0.6 0.6 0.7 0.7 0.7 0.7 0.8 0.8 0.8 0.8 0.9 0.9 0.9 0.9 1 1 1 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 + Input request probability per unit time () Input request probability per unit time () Dense optical interconnect: interconnection folded in 2D Optical Multistage Architecture Paradigm Routing parameters 6 6 5 crossbar crossbar 5 4 4 3 SEMIN SEMIN Probability of packet acceptance Probability of packet acceptance 3 2 Buffer size 2 GSMIN GSMIN Buffer size 1 0 Depth Analysis = 1 Depth Transfer = 1 Depth Analysis = Buffer Size Depth Transfer = Buffer Size 1 0 0 Input request probability per unit time () Input request probability per unit time () 6 6 5 5 4 crossbar 4 3 3 SEMIN Buffer size Probability of packet acceptance 2 2 1 GSMIN 0 Depth Analysis = Buffer Size Depth Transfer = Buffer Size 1 0 Input request probability per unit time () …topology mapped on a plane (optical interconnects, VLSI integration) four-dimensional hypercube connected multiprocessor… Output (to CCD) Displacement stages Input (from VCSEL array) Exit first module Input second module time Permutation appearance period Time slot Stage-Distributed Time-Division Permutation Routing in a Multistage Optically Interconnected Switching Fabric Alvaro Cassinelli(1), Makoto Naruse(2), Masatoshi Ishikawa(1) 1: University of Tokyo, Dept. Information Physics and Computing, 7-3-1 Hongo Bunkyo-ku, Tokyo 113-0033, Japan. (alvaro@k2.t.u-tokyo.ac.jp) 2: Communications Research Laboratory, 4-2-1 Nukui-kita, Koganei, Tokyo 184-8795, Japan. ECOC 2003 / We4.P.137 (C) (A) Column-controlled buffered MIN architecture Introduction Plane-to-plane guide-wave based optical interconnections Fiber-based Modules vs. Free-Space Bandwidth, cross-talk and volume consumption considerations, all advocate for plane-to-plane optical interconnects within massively interconnected electronic system as a promising substitute to conventional electronic circuitry in the near future. Multistage Interconnection Networks (MINs) are capable of supporting arbitrary input-output interconnections, and have been long proposed as an interesting alternative to the full crossbar and the time-shared bus (in terms of performance, cost and scalability) for use as permutation networks for multiprocessors but also as point-to-point switching fabrics for telecom applications. As a consequence, multistage architectures using optical plane-to-plane regular interconnections may well represent a theoretically optimal architecture for use within a massively interconnected systems. The work presented here develops on some fundamental aspects of such optical multistage architecture: Routing Strategy Conflicts are not resolved individually at the switch level, but globally at the stage level by a “tournament” between all the incoming requests to that particular stage. Provided that these request are uniformly directed to any possible output, "votes" leading to the adoption of one of the two possible states of the global-switch will be evenly distributed. Such behaviour takes place for all stages of the network, so that at each stage, half of the requests will be dropped and half will be able to pass to the next stage. This means there is an enormous number of discarded packets, certainly much bigger than that occurring by internal blocking in a standard SEMIN. However, if one considers a buffered architecture, then presumably there will be no need to provide it with a large buffer memory, because the packets that have been retained in the buffers are very likely to go forward in the following tournament. • better efficiency (than holograms) for long-range interconnections. • no cross-talk in 3D, just like free-space optics. • No space-invariance required. • Theoretically more volume efficient than free-space • Precise and robustalignment possible… • multiple interleaved permutations possible. …an optical “3D optical wiring” module between 2D VLSI arrays. • Maybe “hard” to build? Boring, but not a fundamentally difficult (can be automated, can be done by “layers”, see below). • Alignment ofboth output and input needed. • Power dissipation may be a fundamental limitation, but we are far from these limits. Stacking layers Stacking planar lightwave circuits to produce 2D guided-wave interconnection modules In the case of “column/row-decomposable” permutations (see left), which happen to be the ones required in most regular interconnected MINs, 2D interconnection modules can be easily implemented by stacking layers of printed lightwave circuits containing 1D permutations (see right). (A)Guide-wave based interconnection modules While many demonstrator have been built to illustrate the advantages of free-space optics over electronics for dense plane-to-plane interconnections, there has been relatively little research on the use of two-dimensional wave-guide-based interconnections, (or guide-wave based interconnection modules). Yet, these can easily achieve better transmission efficiency than holographic-based interconnections while almost completely cancelling cross-talk, and contrary to the common belief they may be more volume efficient than free-space optics for both space-invariant and space-variant interconnects. “Brute” folding and row/column decomposed folding of the perfect shuffle inter-stage permutation Performance analysis of a buffered GSMIN (B)Transparent column-controlled Optical Multistage Network The use of “modular” inter-stage interconnections leads naturally to consider a new paradigm for the optical MIN architecture: interconnection modules containing several interleaved global interconnections (i.e. permutations). Addressing of the required permutation can be done by mechanical displacement of the whole multi-permutation module. Cascading such modules without intermediate optoelectronic arrays gives a transparent ”globally-stage switched” multistage interconnection network (GSMIN) that can be used as a circuit-switched permutation network for multiprocessor communications (this architecture represents an original optical implementation of a column-controlled shuffle-exchange MIN). Experimental results Figures represent performance (normalized amount of satisfied requests) as a function of the input load (computed as the probability of a request being issued at any input per unit time) for the MIN with stage-distributed switching, and for a standard Shuffle Exchange MIN with independent control of switches (both 128x128 large networks). The input traffic follows the uniform request model. Recently, we tested this approach by implementing several 4x4 fiber-based modules each integrating a different fixed topology. Automatic alignment of modules, both dynamically and statically (pre-aligned ”plug-and-play” exchangeable blocks), is a critical issue now being studied. “Fair” selection mode (the state of the global-switch is chosen randomly if the poll is a draw) Optimal situation: both “depth of analysis” and “depth of transfer” are equal to the actual buffer size. The performance of a "fair" operated GSMIN is superior to that of a standard ("fair" operated) MIN for buffer sizes larger than one. Also, for a buffer size equal to three, the GSMIN network gives better performance than the crossbar, even at full-load. (C) Buffered column-controlled Optical Multistage Network Column-controlled unbuffered MINs have been extensively studied for circuit-switched applications. We found that an inter-stage buffered column-controlled MIN architecture may also represent an interesting alternative to the well-known Shuffle-Exchange MINs (SEMINs) for point-to-point packet-routed communications, both from the point of view of its implementation complexity and simplicity of routing protocol. Only the older packets waiting on the queues are given attention in the selection phase, and that only these packets are candidates to be transferred during the transfer phase. (B) Transparent column-controlled Multistage Network Mechanically Reconfigurable wave-guide-based interconnections “Alternate” selection mode(the state of the global-switch changes if the poll is even) Why common control of switches per switching stage (column-control)? When a standard SEMIN and a column-control SEMIN (GSMIN) are operated in forced alternate mode, they become strictly equivalent architectures. Therefore, when a standard SEMIN is operated in forced alternate mode, its performance roughly degrades to that of an alternate operated GSMIN; on the other hand, when a GSMIN is operated in forced alternate mode, the performance remains substantially the same. Multi-permutation interconnection module paradigm Simplicity and Scalability: A whole switching stage needs a unique control signal. Can have a “direct” optical implementation: the switching stage and its adjacent inter-stage permutation can be physically merged into a unique “permutation module" providing two different interconnection patterns (bi-permutationswitching module). A clear improvement in performance is seen in the standard SEMIN for buffer sizes larger than one. This performance may exceed that of the GSMIN, but it seems to increase slowly (with buffer size), so that when the buffer size is equal to five (or four in the case of a 64×64 network), both networks show roughly the same performance. A small mechanical (or optical ) perturbation produces a drastic change of the interconnection pattern from input to output. Two implementation of a column-controlled Multistage Interconnection Network Mechanically reconfigurable multi-stage architecture based on cascading multi-permutation modules Guide-wave based multi-permutation interconnection module “Forced-alternate” selection mode (the state of the global-switch changes periodically) Bi-permutation module (integrates switching and permutation stage) Column-controlled Shuffle-Exchange stage Spanned hypercube using Bi-permutation modules Prototype using interleaved fibers There is no significant change in performance between a buffered GSMIN with "alternate" selection and a buffered GSMIN with "forced alternate" selection. A continual "blind" alternation of switching states performs just as well. Non-integrated bi-permutation module: Individual control of switches as well as arbitration may be unnecessary on a standard SEMIN if the buffer size is larger than four. Also, when the buffer size is equal to three, the 128x128 GSMIN already outperforms a 128x128 full-crossbar. Channels are multi mode fibers: MFD = 9.5  mGrad diam. 125 mNA: 0.1 = Experience with two cascaded modules: Four-dimensional “spanned” hypercube network (16 nodes) using four bi-permutation modules, each providing a cube permutation and the identity. Atotal of 24=16 global permutations for the whole network. “indirect” implementation obtained by electrically joining individual switches (common control lines a, b and c) of a Shuffle-Exchange network (here the Inverse Baseline) A more “direct” Implementation of a column-controlled Multistage Network using multi-permutation switching modules (the GSMIN network) Conclusions and further research Column-control certainly reduces overall interconnection capacity, but if things are properly designed, the MIN architecture can still accommodate communication primitives of most static interconnection networks. The interest of such an arrangement lies in the ease of control and straightforward implementation. We presented here preliminary experimental results demonstrating a simple optical architecture using cascaded fiber-based bi-permutation modules. An electro-mechanical system has been developed providing stage-switching times on the order of milliseconds, making this architecture suitable for reconfigurable, high-bandwidth inter-processor communications. Further research on this topic include the feasibility study of dense guided-wave multi-permutation modules by stacking layers of planar lightwave circuits, as well as the introduction of electro-optical switching. A column-controlled MIN can be an efficient (circuit-switched) permutation network. Displacement pitch for commutation: 125 m Alignment tolerance: 5 m (half peak power). A multistage ”spanned” version of most direct network topologies (hypercube, cube-connected-cycles, etc.) can be implemented as an unbuffered column-controlled MIN architecture. A time-division multiplexing (TDM) technique can be used to select the interconnections at each stage. Inter-module Coupling Efficiency: 1.7dB (no additional optics, matching oil or antireflection coating).  Validation of simple cascaded architecture Electro-mechanical switching device Possible implementation of an electro-optical bi-permutation module using stacked layers and electro-optically controlled normal coupling between the layers. This electro-magnetic actuated device can position the module in both X and Y directions. From the routing point of view, simulations showed that a buffered column-controlled MIN would not require excessive buffer size to achieve respectable performances (under uniform traffic). The path-selection mechanism can be further reduced to the simple alternation of the available stage permutations, without degrading the performance. Under such stage-distributed time-division permutation multiplexing, the SEMIN and GSMIN fabrics become strictly equivalent routing architectures; hence (provided that buffer size is chosen to be larger than three) this analysis-free strategy will provide a very simple arbitration mechanism for standard SEMIN networks. This is an interesting result on its own. The switching speed seems to be limited to the millisecond range (resonant frequency). Micro electro-mechanical actuators (MEMS) may also be an interesting alternative when switching latency in the millisecond range is tolerable. A column-controlled MIN can emulate a full-crossbar by time-multiplexing First two modules of a spanned version of a 4-dimensional hypercube were fabricated using interleaved optical fibers, and the resulting four possible interconnections observed. Despite the reduced permutation capacity of a column-controlled MIN, a permutation network can emulate (by time-multiplexing) a full-crossbar provided that the reconfiguration latency remains small with respect to the interconnection request rate, and that the bandwidth is sufficiently high to accommodate in a single time slot all the data to be transferred. An unbuffered column-controlled MIN may provide enough optical bandwidth - but the reconfiguration time may be still too slow (using thermo-optical switches for instance). This arbitration free mechanism may be very appealing for all-optical networks, if optical buffering functions (e.g. delay lines) could be integrated on the cascaded modules themselves, an issue worth exploring.