The ANET Optical Network Architecture

The ANET Optical Network Architecture

The operation of an ANET region is as follows. Each ANET region contains 64 cores. Each core is assigned a wavelength channel to transmit on and enough receive elements to read all of the data being transmitted from every other core within an ANET region. Data is transmitted at the chip clock frequency, _clock = 1 GHz, so as to avoid costly serialization and deserialization steps. In order to transmit a 4-byte word within a single clock cycle, 32 ANET communication lines are required. An additional 8 lines are added for address information and parity checks bringing the total number of ANET lines within an ANET region to 40. The number of connect cores is N=64 in a standard ANET region.

A more detailed design of the optical network is shown in Figure 4. Figure 4 also summarizes our estimates of optical power consumed by the network. For an ANET with 64 cores and _clock = 1 GHz, the transmit bandwidth is T_BW = 2.6 Tb/s and the receive bandwidth R_BW =161 Tb/s. This simply indicates that because of the broadcast capability, there is a 64X multiplier on the send bandwidth. Our analysis shows that the latency of the ANET network is approximately 3ns. Approximately 0.5ns is due to optical delay and the rest is electrical. The total off-chip power consumption, as shown in Figure 4 is 2.6W. Comparisons of ANET performance with that of an electrical mesh are given in Table 1 (3 pages ahead).

Figure 4: ANET network with memory interface and layout detail for optical power coupling to chip and distribution within the ATAC network. T_BW = MxNx_clock = 2.5Tb/s; R_BW = MxN(N-1)x_clock = 161Tb/s. The power consumption of a 64-core photonic ANET is 2.6W with only 0.6W consumed on-chip.

A key area of innovation in this project is the interface between traditional digital logic and the novel integrated photonic devices. This includes both the low-level details of how a digital signal is translated into an optical signal as well as the higher-level question of how the massive quantity of data received from the optical network should be screened, sorted, and buffered before it is consumed by the processor.

Figure 5 shows the path a bit takes as it is transmitted through the optical network. First, a signal stored in a flip-flop is used to activate a modulator driver (an analog electrical circuit) which, in turn, controls the optical filter/modulator component. The filter/modulator couples light of a specific wavelength from the wideband source waveguide, modulates it, and transfers it to the data waveguide. When the light passes a receiver filter, part of it is drawn off and fed into a photodetector. The photodetector outputs a small electrical signal which is then converted back to a digital bit.

Properly designing the optical components, modulator driver and circuits to receive the signal from the photodetector requires a close collaboration between the architecture and photonics groups. Since these optical components are used only to transmit digital signals, their functional requirements may be significantly different from those needed to transmit analog signals. In addition, the physical characteristics of the optical devices (e.g., size, capacitance, and manufacturing variation) greatly influence the design of the electrical interface circuits.

O
Figure 5: Path of a single bit from the digital domain, through the optical network and back into digital form. Our design does not require the traditional TIA on the receive side.

ur proposed design uses a novel technique to transition from an optical signal to a digital electrical signal.  Photons that are extracted from a data waveguide by a receiver filter are routed to a photodetector whose output is directly connected to a digital logic gate.  The key to making this work is ensuring that the photodetector output is sufficient to charge the input capacitance of a digital gate to the required switching voltage. Traditionally a transimpedance amplifier (TIA) is used as the interface to convert the tiny photodetector output current to digital voltage levels.  However, TIAs are sensitive analog circuits that are difficult to design and consume large amounts of die area (500 μm²) and power (1mW) per receiver. In addition, placing these sensitive analog circuits and noisy digital circuits in close proximity decreases reliability. By using high-efficiency filters and small-footprint, waveguide-integrated photodetectors and carefully managing optical power levels, TIAs can be omitted in future process generations.  As digital circuits shrink, the input capacitances decrease and this technique becomes feasible.  Our research indicates that TIA-free designs will be practical for chips based on 22 nm and smaller process technologies.