Neil Di Spigna

Daniel Schinke
email: djschink@ncsu.edu

Srikant Jayanti
email: sjayant2@ncsu.edu

Veena Misra
email: vmisra@ncsu.edu

Paul Franzon
email: paulf@ncsu.edu

Simulation and Experimental Characterization of a Unified Memory Device with Two Floating-Gates

Keywords: memory, nonvolatile, dynamic, volatile, unified, floating-gate, FLASH, DRAM, high-k dielectric, simulation

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

This chapter is in part based off previously published work on the demonstration of a novel double floating-gate unified memory device [START_REF] Di Spigna | A Novel Double Floating-Gate Device[END_REF]. In this paper, that work is extended through device simulations and additional details on the fabrication, operation, and design of circuits based on such a device. Such a unified memory device could store both volatile (dynamic) and nonvolatile states simultaneously. This could have a dramatic impact on traditional memory hierarchies [START_REF] Park | Novel Double-Gate 1T-DRAM Cell Using Nonvolatile Memory Functionality for High-Performance and Highly Scalable Embedded DRAMs[END_REF][START_REF] Han | Polysilicon Channel TFT With Separated Double-Gate for Unified RAM (URAM)-Unified Function for Nonvolatile SONOS Flash and High-Speed Capacitorless 1T-DRAM[END_REF][START_REF] Schinke | Computing with Novel Floating-Gate Devices[END_REF]. For example, the data stored in the nonvolatile mode of the device when the computer is powered down could quickly be written to the dynamic state when the power is turned on, allowing for instant-on computing. This data transfer could also operate in reverse as dynamic data could be written to nonvolatile states to allow for full or partial hibernation of the memory fabric. Alternatively, writing dynamic data quickly to nonvolatile data could enable fast in-situ checkpointing. Finally, there are a number of novel logic applications for such a device that could impact numerous areas of computation [START_REF] Schinke | Computing with Novel Floating-Gate Devices[END_REF]. Two floating-gates (FGs) have been used previously for enhancing memory operation [START_REF] Afshari | Nonvolatile Memory with Multi-Stack Nanocrystals as Floating Gates[END_REF][START_REF] Lee | Operational and Reliability Comparison of Discrete-Storage Nonvolatile Memories: Advantages of Single-and Double-Layer Metal Nanocrystals[END_REF][START_REF] Singh | Metal Nanocrystal Memory with Pt Single-and Dual-Layer NC With Low-Leakage Al2O3 Blocking Dielectric[END_REF][START_REF] Ohba | Nonvolatile Si Quantum Memory With Self-Aligned Doubly-Stacked Dots[END_REF]. However, these designs typically have been used in an effort to increase the memory window and data retention compared to single FG devices. For example, the size of the nanocrystals in the two FG layers can be engineered to exploit the Coulomb Blockade effect [START_REF] Ohba | Nonvolatile Si Quantum Memory With Self-Aligned Doubly-Stacked Dots[END_REF]. In this research, however, two FGs are used to enable a device which can store both dynamic and nonvolatile states concurrently. The two modes of operation are distinguished by the charge condition of the two FGs. The device is in the dynamic mode when charge is simply redistributed between the two FGs. It is not until charge is drawn up from the substrate that the device enters its nonvolatile mode. Therefore, the operation of the device requires the existence of a window between when charge is merely redistributed between the FGs, and when charge is drawn up from the channel; allowing for the coexistence and selective control of both states. This requires engineering the vertical stack to create and fine tune such a window. The fabrication of the proof-of-concept device is discussed in the next section and the design tradeoffs based on those decisions are considered throughout.

Device Fabrication and Modeling

The double FG MOSCAPs shown in Fig. 1 were fabricated to experimentally demonstrate and confirm the device operation. The process flow is outlined in Fig. 1a. The wafers were cleaned before a SiO 2 gate oxide was grown through thermal oxidation.

For the bottom FG, palladium was deposited through e-beam evaporation and patterned using liftoff. HfO 2 was used as the inter-FG dielectric and deposited through Atomic Layer Deposition (ALD). The top FG was then fabricated once again using palladium. ALD was used to deposit the control dielectric of HfAlO, which has been previously shown to have low leakage that is required of ultra-scaled FLASH memory technologies [START_REF] Jayanti | Ultimate Scalability of TaN Metal Floating Gate with Incorporation of High-K Blocking Dielectrics for Flash Memory Applications[END_REF]. A palladium control gate was deposited and patterned, followed by a backside etch and aluminum deposition. A Transmission Electron Micrograph (TEM) of the cross section of the fabricated device is shown in Fig. 1b. Being a proof-of-concept structure, the layer thicknesses of the vertical stack were not aggressively scaled, which as will be shown later in this paper have led to relatively high operating voltages. Further device scaling combined with engineering the materials has been shown through simulation to lower operating voltages, optimize device performance and achieve high storage density [START_REF] Schinke | Computing with Novel Floating-Gate Devices[END_REF]. Towards these ends, a 65-nm gate length MOSFET, with the properties listed in Table 1, was modeled in Sentaurus TCAD. Bulk Si -This model is used to confirm the characterized device operation and is the link between the proof-of-concept structure and the circuit simulations discussed in Section 6. In addition to more aggressive thickness scaling, some important material distinctions can be made between the fabricated device and the simulated device. In contrast to the fabricated device, different materials were chosen for the two FGs, creating an asymmetry across the inter-FG dielectric, as can be seen in the energy band diagrams for the simulated and fabricated devices shown in Fig. 2. A high work function metal, Pt, was used for the top FG, whereas a low work function metal, Mg, was used for the bottom FG. This results in fast dynamic programming as electrons tunnel from the bottom FG to the top FG relatively easily. Once trapped, the deep potential well of the top FG sustains sufficiently long retention times but comes at the expense of dynamic erasing, as will be shown later in the circuit simulations. The characterization and operation of this device is discussed in the following sections, starting with the dynamic mode.

Dynamic Mode Operation

The mode of the device is determined by the applied voltage envelope. For a relatively small bias and short duration, the device's dynamic mode is programmed/erased. The dynamic operation of the device is illustrated in Fig. 3. The device is swept from a negative voltage, to a positive voltage, and then back to the negative voltage. Initially, as the device has a small negative voltage applied to the control gate, electrons will move to the bottom FG leaving behind a positive charge on the top FG, as pictured on the right-side of Fig. 3. The negative charge on the bottom FG, closer to the substrate, will cause a slight shift of the flat-band voltage to the right, as can be seen in the measured CV characteristic of the fabricated device. As the sweep continues, the voltage applied to the gate becomes positive, and the opposite charge condition results. Electrons now move to the top FG resulting in a positive charge closer to the substrate, as pictured on the left-side of Fig. 3. This positive charge closer to the substrate will cause a slight shift of the flat-band voltage to the left, once again demonstrated by the measured CV characteristic. Thus, for dynamic operation, the hysteresis is counter-clockwise, which would be the opposite direction anticipated for traditional single FG devices. Notice that for dynamic mode operation, the voltage applied to the control gate is insufficient to draw up charge from the substrate, but rather is only strong enough to simply redistribute charge on the FGs across the relatively thin inter-FG dielectric. Thus, there is no net increase in the charge on the FGs. This condition is what distinguishes the mode of operation of the device. The simulations of the dynamic mode operation of the device are shown in Fig. 5. The initial uncharged device characteristics are shown (1). A 5 V pulse is applied to the control gate for 50 ns causing the threshold voltage to shift about -330 mV (2). After about 300 ms, the threshold voltage decays about 220 mV back towards the initial state of the device (3). For these simulations, it is assumed that a 100 mV difference is needed to distinguish between the two distinct states, thus a refresh is required. A 5 V refresh pulse is applied to the control gate. Since the device has not fully decayed back to the initial state, this refresh pulse only needs to be applied for 40 ns, rather than the initial 50 ns applied to redistribute charge in the fresh device. As can be seen in Fig. 5a, this refresh returns the device back to the charged state threshold voltage (4). This volatile cycle continues, requiring a refresh period of about 300 ms to retain the charged state, and thus demonstrating the dynamic mode of the device. The volatile nature illustrated by the device simulations was confirmed in the fabricated device, as shown in the dynamic retention characteristics of Fig. 6. A +10 V pulse was applied to the fabricated device which caused a flat-band voltage shift to the left, as shown in Fig. 6a. This is directly analogous to the simulations shown in Fig. 5b. Once the bias to the control gate was removed, the CV characteristics would decay back to the original curve. This is illustrated in Fig. 6b in which the capacitance at 0.5 V is measured over time. As the charge difference between the two FGs decays, the flat-band voltage shifts to the right and the capacitance at 0.5 V decays from ~8.5 pF to ~6.4 pF. At this point, after ~22 s, the +10 V is reapplied to the control gate, once again refreshing the charge difference between the two FGs and causing the flatband voltage to shift back to the left. This is shown for five cycles in Fig. 6b, successfully demonstrating and confirming the predicted volatile nature of the dynamic mode operation. Finally, the dynamic mode endurance of the fabricated device is shown in Fig. 7. A consistent 300 mV window between the programmed and erased states is maintained as the device was cycled over 10 5 times, though a cycling drift is present. The inter-FG dielectric is critical in ensuring the stable operation over the extensive number of cycles required of DRAM. As this dielectric is further scaled, this will permit the reduction of voltages and fields, and thus the use of lower energy tunneling mechanisms that will reduce the stress on this dielectric. Choosing an appropriate dielectric for this inter-FG is actively being investigated.

Nonvolatile Mode Operation

The nonvolatile mode of the device is entered when a voltage pulse is applied to the control gate that is sufficient enough to draw up a net charge from the substrate to the FGs, as illustrated in Fig. 8. Once again the device is swept from a negative voltage, to a positive voltage, and then back to the negative voltage. However, unlike in the dynamic mode operation, the bias is large enough to draw up charge from the substrate. As the voltage applied to the gate starts out negative, electrons are repelled towards the substrate leaving behind a positive charge on the FGs. A net positive charge on the FGs causes a negative shift in the flat-band voltage, as can be seen in the measured CV characteristic. As the sweep continues, the voltage applied to the gate becomes positive, and electrons are now drawn up from the substrate resulting in a negative charge on the FGs. The voltage is then swept in reverse, and the negative charge on the FGs results in a positive flat-band voltage shift, once again witnessed in the measured CV characteristic. This results in a clockwise hysteresis, which is expected for traditional single FG nonvolatile devices. The nonvolatile program/erase characteristics of the fabricated device are shown in Fig. 10. The asymmetry between the program and erase voltages is expected since, as shown in the band diagram of Fig. 2b for the fabricated device, the electrons can more easily tunnel onto the FGs then they can tunnel back to the substrate. In Fig. 10, both the initial and after-60 second flat-band voltage shifts are plotted, and in every case, the after-60 second shift is more pronounced than the initial shift. This is due to the fact that as the bias is being applied to the control gate, more of the charge is being drawn up to the top FG, relative to the bottom FG. When the external bias is removed, the charge redistributes between the two FGs resulting in an increase in the charge on the bottom FG. Since the bottom FG is closer to the substrate, this charge redistribution leads to a greater flat-band voltage shift over time as the charge settles. This is confirmed by the simulations of the nonvolatile mode of the modeled device shown in Fig. 11. The initial uncharged device characteristics are shown [START_REF] Di Spigna | A Novel Double Floating-Gate Device[END_REF]. A 9 V pulse is applied to the control gate for 30 µs (2). This pulse is large enough to pull up charge from the channel, resulting in a net increase of charge on the FGs. Initially, most of the charge is drawn up to the top FG, limiting the impact on the channel. Thus, only a relatively minor positive threshold voltage shift occurs immediately after the pulse, as shown in Fig. 11a (2). However, after the voltage is removed from the control gate, the charge on the FGs redistributes resulting in a much more pronounced positive threshold voltage shift of about 1.52 V after about 1 s (3). This is the same phenomenon that occurred in the fabricated devices, though not to the same extent. The device does not reach its stable state until after some time passes. The relationship between the initial applied pulse and the charge redistribution settling time is currently being investigated and will have to be accounted for at the circuit level. Finally, as shown in the simulation, a -8.5 V pulse applied for 30 µs (4) followed by a charge settling period of 1 s (5) returns the device approximately back to its uncharged state. To verify the nonvolatile nature of the fabricated device, the retention of the nonvolatile mode is plotted in Fig. 12a. A window of at least 4.5 V is maintained by extrapolating the data out to 10 years. Finally, the nonvolatile endurance of the device was demonstrated through over 10,000 cycles, as shown in Fig. 12b.

Concurrent Mode Operation

The device is not limited to operation in either the dynamic mode or the nonvolatile mode, but rather it can operate in both the dynamic and nonvolatile modes at the same time. The experimental verification of concurrent mode operation is shown in Fig. 13. The device is first programmed into the nonvolatile state using a +17 V sweep, as shown in Fig. 13a, which results in a positive flat-band voltage shift as negative charge is drawn up from the substrate into the FGs. To demonstrate concurrent mode operation, a dynamic state is then embedded on top of the programmed nonvolatile state by the application of a dynamic pulse of +10 V. As a response to this pulse, some of the negative charge on the bottom FG is drawn up to the top FG, leaving behind a less negatively charged bottom FG, resulting in a slightly negative flat-band voltage shift relative to the original charged nonvolatile state, as shown in Fig. 13b. Once this dynamic bias is removed, the charge difference between the two FGs decays, and the flat-band voltage shifts back to the original nonvolatile programmed state. This cycling of the dynamic state embedded on top of the programmed nonvolatile state is repeated five times, as shown in Fig. 13c. Combining this data with that shown in Fig. 6, which represents the dynamic state embedded on top of the nonvolatile erased state, successfully demonstrates that the dynamic state can be embedded on both the programmed and erased nonvolatile states. Thus, concurrent mode operation of the fabricated device is experimentally verified. The concurrent mode simulations are shown in Fig. 14. The device is first programmed into its charged nonvolatile state [START_REF] Di Spigna | A Novel Double Floating-Gate Device[END_REF]. A dynamic pulse of 5 V for 50 ns results in about a -330 mV threshold voltage shift [START_REF] Park | Novel Double-Gate 1T-DRAM Cell Using Nonvolatile Memory Functionality for High-Performance and Highly Scalable Embedded DRAMs[END_REF]. Upon cessation of the bias, the charge redistributes and the threshold voltage starts decaying back to the charged nonvolatile state (3). The CV curve of Fig. 14b is directly analogous to the experimental characterization shown in Fig. 13b. A relatively small dynamic pulse shifts the flat-band voltage in the negative direction, at which point it begins to decay back to the charged nonvolatile state flat-band voltage, as shown in Fig. 13c. Upon a refresh, the flat-band voltage once again shifts to the left (4). Thus, it is shown through simulation that a dynamic mode can be embedded on the charged nonvolatile state. The dynamic, nonvolatile, and concurrent mode operation have been experimentally demonstrated. The characterization of the fabricated devices has been confirmed through device simulation, successfully verifying the operation of this novel unified memory device. A memory array using this device is discussed in the next section.

Circuit Simulations

The memory array shown in Fig. 15 was designed in Cadence Virtuoso 2010 using a BSIM4.0 MOSFET model with a 45-nm gate length and device parameters similar to the simulated device described in Table 1. However, instead of bulk silicon, the substrate was SOI with a thickness of 13 nm; an SiO 2 back gate dielectric of 1.2 nm was used, and the control and back gates were composed of aluminum. The operation of this memory array is described in Table 2. To dynamically program a target device, 3 V is placed on the appropriate WL and -2 V is placed on the appropriate SL. This results in a 5 V bias across its gate stack, which when applied for 50 ns results in the target device being dynamically programmed, as previously described in the device simulation. However, to prevent inadvertent programming of non-target devices on that WL, the non-target SLs need to be biased to 2 V such that there is only a 1 V bias across their gate stack. This represents the dynamic retain condition. As previously discussed, the device was engineered to have a low work function metal for the bottom FG and a high work function metal for the top FG. This resulted in an asymmetric barrier that allowed for fast dynamic programming and increased dynamic retention as charge tunneled easily from the bottom FG into the deeper energy well of the top FG, as shown in Fig. 2a. This resulted in a dynamic retention of 300 ms. However, this came at the expense of the dynamic erase; which as shown in Table 2 takes 10 µs. This is much longer than conventional DRAM. If, on the other hand, the materials are chosen to be symmetric, as was the case for the fabricated device in which palladium was used for both the top and bottom FGs, the dynamic erase time would reduce to 200 ns. Of course with a symmetric barrier, there is no longer the deeper potential well for the charge in the dynamically programmed state and so the retention time would also be reduced. However, for this device the retention time would only reduce from 300 ms to 100 ms, which could prove a wise tradeoff for reducing the dynamic erase time from 10 µs to 200 ns. Certainly further work function engineering can be performed to tailor the device performance towards target applications.

Another advantage of the device is that it operates more like an SRAM than a DRAM, and thus the read operation takes only 2.2 ns, which is much faster than DRAM. The read is also nondestructive, unlike DRAM. The memory array should also have a higher density than DRAM due to the difficulty of scaling the DRAM capacitor and maintaining sufficient charge sharing with the bitline. This device is scalable, in bulk form, to at least the 16-nm node. Through stacking, it has the potential to reach densities equivalent to the 8-nm node.

Overall, the device offers several advantages compared to conventional DRAM [START_REF] Han | Polysilicon Channel TFT With Separated Double-Gate for Unified RAM (URAM)-Unified Function for Nonvolatile SONOS Flash and High-Speed Capacitorless 1T-DRAM[END_REF]. However, such a comparison is ill-conceived. The device may not be wholly superior to DRAM, nor to a similarly scaled single FG nonvolatile device, since it requires an extra FG and the addition of an ultra-thin inter-FG dielectric layer; but the device offers a tremendous advantage that neither of the other devices do singularly; it can store both dynamic (DRAM) and nonvolatile (FLASH) states concurrently. Such a unified memory device has enormous potential to impact next generation memory architectures.

Applications

There are a number of applications for such a unified memory device. For example, the device could be used to enable instant-on computing. The computer could quickly be powered down by simply moving all of the dynamic states into their nonvolatile states. If the entire memory array is written to its nonvolatile state in parallel, this would take only about 30 ms. When the user wants to power the computer back on, the memory controller simply needs to write back all of the nonvolatile data into the dynamic state. Once again, when performed in parallel, this would take only about 14 ms. In theory, the user could power up and power down the computer in only a frac-tion of a second. Beyond user convenience, this could allow for the operating system to power down during moments of inactivity. For example, if the user walked away from their computer to get a drink or take a phone call, the operating system could power down and conserve battery life. When the user returned, the power up penalty would only be a fraction of a second. This device could also enable partial hibernation. For parts of the memory that are not currently being used, those arrays could be written to the nonvolatile state in the background as the user continues to operate their computer. This could enable a flexible memory fabric that could be selectively powered down which could have a significant impact on energy-proportional computing. An example application for this would be Google servers. Recently, a study on their server power usage showed that at utilization workloads that were common (20-30%), the servers operated at less than half their peak energy efficiency performance [START_REF] Barroso | The Case for Energy-Proportional Computing[END_REF]. Given the nature of their utilization, current solutions to transfer to inactive modes are impractical because of both a time latency and energy penalty. The device described in this chapter could make such transitions practical by significantly reducing the wake-up penalties. Alternatively, partial hibernation enabled by this device could be used to further enhance active energy-saving schemes.

Another example application in which these devices could be beneficial is in-situ checkpointing. The device could be running continuously in dynamic mode, and then upon desire for a check-point, the entire memory array could be quickly written to the nonvolatile state in only about 30 ms. This would be much more efficient than writing through narrow channels to disk. Thus, more check-points could be efficiently taken, improving the resiliency of the computer. Upon detection of an error, the correct state could be recovered much faster than traditional memory hierarchies would permit. Instant-on computers, energy-proportional computing, and in-situ checkpointing are just a few examples of the potential that could be realized with a memory array composed of this new unified memory device.

Conclusion

New unified memory devices using two FGs were modeled, simulated, fabricated and characterized. The operation of the devices in dynamic, nonvolatile, and concurrent modes were demonstrated in proof-of-concept MOSCAPs and confirmed through device simulations. The programming, retention, and endurance characteristics were demonstrated for the different modes. A memory array based on these devices was designed and simulated. It was shown that these devices compare favorably to both conventional DRAM and FLASH devices. However, the true potential of these devices is not in their use as either a DRAM or FLASH replacement, but rather as a new unified memory device that can store both dynamic and nonvolatile states concurrently. Applications for such a device were discussed that highlight the significant impact this device could have on next generation memory architectures.

Fig. 1 .

 1 Fig. 1. Device Fabrication. (a) Recipe and (b) TEM cross section

Fig. 2 .

 2 Fig. 2. The energy band diagrams of the (a) simulated and (b) fabricated devices.

Fig. 3 .

 3 Fig. 3. Dynamic Mode Operation

Fig. 4 .

 4 Fig. 4. Dynamic Program/Erase Characteristics

Fig. 5 .

 5 Fig. 5. Dynamic Mode Simulations. The (a) drain current and (b) capacitance vs. control gate voltage.

Fig. 6 .

 6 Fig. 6. Dynamic Mode Retention Characteristics. (a) A +10 V; ~500 ms pulse is applied to the control gate resulting in a negative flat-band voltage shift. (b) The capacitance was measured at 0.5 V over time for 5 cycles showing the charge difference between the two FGs decay back to the original CV curve.

Fig. 7 .

 7 Fig. 7. Dynamic Mode Endurance Characteristics. The device characteristics are shown for 200,000 cycles with a ±7.5 V for ~200 ms pulse.

Fig. 8 .

 8 Fig. 8. Nonvolatile Mode Operation

Fig. 9 .

 9 Fig. 9. Program/Erase Characteristics. The dynamic mode of the curve is enlarged in the inset.

Fig. 10 .Fig. 11 .

 1011 Fig. 10. Nonvolatile Program/Erase Characteristics

Fig. 12 .

 12 Fig. 12. The nonvolatile mode (a) retention and (b) endurance characteristics.

Fig. 13 .

 13 Fig. 13. Concurrent Mode Operation. (a) The device is charged into the programmed nonvolatile mode with a +17 V sweep. (b) A +10 V pulse embeds a dynamic state on top of the programmed nonvolatile state. (c) The capacitance is measured at 3.25 V over 5 cycles demonstrating the retention of the embedded dynamic state. As the dynamic state decays, the device returns to the programmed nonvolatile state, requiring a refresh.

Fig. 14 .

 14 Fig. 14. Concurrent Mode Simulations. The (a) drain current and (b) capacitance vs. control gate voltage.

Fig. 15 .

 15 Fig. 15. Simulated memory array architecture

Table 1 .

 1 Device Model Properties

	Layer	Material	Thickness
	Control Gate	Molybdenum	10 nm
	Control Dielectric	HfO 2	18 nm
	Top FG	Platinum	3 nm
	Inter-FG Dielectric	HfSiO	3.2 nm
	Bottom FG	Magnesium	3 nm
	Gate Dielectric	SiO 2	4 nm
	Substrate

Table 2 . Memory Array Operation Operation Bias Across Gate Stack Duration

 2

	Dynamic Program	5 V	50 ns
	Dynamic Erase	-5 V	10 µs
	Dynamic Refresh	5 V	40 ns
	Nonvolatile Program	9 V	30 µs
	Nonvolatile Erase	-9 V	14 µs
	Low V t Read	1.2 V	2.2 ns
	High V t Read	2.7 V	2.2 ns

Acknowledgements. This article is in part based on works supported by the National Science Foundation under award nos. 0811582 and 1065458. We thank Dr. Eric Rotenberg, Steve Lipa, W. Shepherd Pitts, Shivam Priyadarshi, Vinodh Kotipalli and Narayanan Ramanan for their valuable contributions to aspects of this effort. Thanks to Dr. Dale Batchelor of AIF at NCSU for TEM analysis and to Jonathan Pierce for FIB preparation of the TEM cross sections.


Michael Muehlberghuber

Christoph Keller

Frank K Gürkaynak

Norbert Felber
email: felber@iis.ee.ethz.ch

FPGA-Based High-Speed Authenticated Encryption System

Keywords: Authenticated encryption, High-throughput architecture, FPGA, Pipelining, Serpent, OCB, AES, GCM. Bob Alice Quantum Key Distribution System Encryption Board Optical User Interface Optical User Interface FPGA Client Interface Authenticated Encryption Plaintext Keytransfer Quantum Key Distribution System Secure Channel Encryption Board Optical User Interface Optical User Interface FPGA Client Interface Authenticated Encryption Plaintext Ciphertext Public Channel (100Gbit/s) Keytransfer

The Advanced Encryption Standard (AES) running in the Galois/Counter Mode of Operation represents a de facto standard in the field of hardware-accelerated, block-cipher-based high-speed authenticated encryption (AE) systems. We propose hardware architectures supporting the Ethernet standard IEEE 802.3ba utilizing different cryptographic primitives suitable for AE applications. Our main design goal was to achieve high throughput on FPGA platforms. Compared to previous works aiming at data rates beyond 100 Gbit/s, our design makes use of an alternative block cipher and an alternative mode of operation, namely Serpent and the offset codebook mode of operation, respectively. Using four cipher cores for the encryption part of the AE architecture, we achieve a throughput of 141 Gbit/s on an Altera Stratix IV FPGA. The design requires 39 kALMs and runs at a maximum clock frequency of 275 MHz. This represents, to the best of our knowledge, the fastest full implementation of an AE scheme on FPGAs to date. In order to make the design applicable in a real-world environment, we developed a custom-designed printed circuit board for the Stratix IV FPGA, suitable to process data with up to 100 Gbit/s.

Introduction

Confidentiality and authenticity are two of the most important cryptographic goals. Whereas the former assures that any eavesdropping adversary is unable to decipher a given message-even if she has access to the transmission medium-, the latter refers to the cryptographic service that ensures that the receiver of a message can be sure about its origin, i.e., that an attacker has not impersonated the sender. Authenticated encryption (AE) combines these two services and allows a secure and authentic communication between two parties.

In order to provide high-throughput AE implementations based on block ciphers, so-called combined modes of operation have been designed throughout the last decade. They allow a higher throughput by interleaving the authentication part and the encryption part instead of calculating them consecutively Fig. 1. High-speed authenticated encryption system setup (as traditional AE methods do). The two most widely accepted AE modes of operation are Counter with CBC-MAC (CCM) [START_REF] Whiting | Counter with CBC-MAC (CCM)[END_REF] and Galois/Counter Mode (GCM) [START_REF] Mcgrew | The Galois/Counter Mode of Operation (GCM). NIST Modes Operation Symmetric Key Block Ciphers[END_REF]. Their acceptance is most likely due to the fact that they have been recommended by the National Institute of Standards and Technology (NIST) (cf. [START_REF] Dworkin | Recommendations for Block Cipher Modes of Operation: The CCM Mode for Authentication and Confidentiality[END_REF] and [START_REF] Dworkin | Recommendations for Block Cipher Modes of Operation: Galois/Counter Mode (GCM) and GMAC[END_REF]). Since then, they have been applied to technologies and protocols such as WiFi 802.11 [8] and IPsec [START_REF] Viega | The Use of Galois/Counter Mode (GCM) in IPsec Encapsulating Security Payload (ESP)[END_REF]. Although the specifications of these modes do not determine the underlying block cipher, most applications make use of the Advanced Encryption Standard (AES) [START_REF]NIST: Advanced Encryption Standard (AES) (FIPS PUB 197[END_REF] since it is another algorithm standardized by the NIST.

The present work proposes a block cipher-based hardware architecture for AE, targeting high throughput on field-programmable gate array (FPGA) platforms. Our design has been developed as to fulfill the requirements of the Ethernet standard IEEE 802.3ba [START_REF]IEEE Standard for Information technology-Telecommunications and information exchange between systems-Local and metropolitan area networks-Specific requirements Part 3: Carrier Sense Multiple Access with Collision Detection (CSMA/CD) Access Method and Physical Layer Specifications Amendment 4: Media Access Control Parameters, Physical Layers and Management Parameters for 40 Gb/s and 100 Gb/s Operation[END_REF], which allows for transmission speeds of up to 100 Gbit/s. This work has been designed as part of a system that employs quantum key distribution (QKD) for synchronizing multiple private key exchanges within a single second, and provides authenticated encryption service using conventional cryptographic primitives. Fig. 1 illustrates the overall system setup. The main contributions of our work are related to the Authenticated Encryption part of Fig. 1, i.e., the digital, AE-related parts on the FPGA and have originally been presented in [START_REF] Muehlberghuber | 100 Gbit/s Authenticated Encryption Based on Quantum Key Distribution[END_REF].

So far, our system employed a common GCM-AES-based cryptographic primitive in order to achieve the required throughput. In this work, we examine alternatives for both the block cipher and the mode of operation and compare the performance of these alternatives to the established cryptographic primitives. Besides exploring more efficient hardware implementations, this work is also motivated by providing an alternative AE scheme, in case successful attacks are developed against the existing primitives. We evaluate the Serpent block cipher [START_REF] Biham | Serpent: A New Block Cipher Proposal[END_REF] and the offset codebook (OCB) mode of operation [START_REF] Rogaway | OCB: A Block-Cipher Mode of Operation for Efficient Authenticated Encryption[END_REF] and we provide results of hardware implementations for different mode of operation/block cipher combinations, namely:

-OCB-Serpent -OCB-AES -GCM-Serpent -GCM-AES Our fastest AE implementation is based on an OCB-Serpent architecture and requires 39 kALMs (Adaptive Logic Modules) on an Altera Stratix IV FPGA. It uses four cipher cores for the encryption part and reaches a throughput of 141 Gbit/s, running at 275 MHz.

Moreover, we developed a custom-designed printed circuit board (PCB), which allows us to use the presented designs in real-world applications such as the system illustrated in Fig. 1. So far, two copies of the board have been fabricated and successfully tested in various sample experiments.

The remainder of this work is structured as follows. In the next section, we present an overview of related work on hardware architectures targeting high-throughput AE designs. In Section 3, a description of Serpent and OCB is given. The actual hardware architecture of our design is presented in Section 4. Throughout Section 5, we summarize our results, including a brief discussion. Finally, Section 6 provides a description of the custom-designed PCB including some of its major features, before we conclude our work in Section 7.

Related Work

Due to the standardization by the NIST, GCM-AES has received significant attention from both the research community and the industry, and several implementations targeting FPGAs can already be found in the literature.

In 2009, Zhou et al. [START_REF] Zhou | Improving Throughput of AES-GCM with Pipelined Karatsuba Multipliers on FPGAs[END_REF] presented a single-core GCM-AES design, which targets a Xilinx Virtex-5 FPGA. They achieved a throughput of 41.5 Gbit/s using the 128-bit version of AES. Henzen and Fichtner [START_REF] Henzen | FPGA Parallel-Pipelined AES-GCM Core for 100G Ethernet Applications[END_REF] showed that it is possible to break the 100 Gbit/s barrier on a Virtex-5 FPGA platform. They made use of four fully unrolled AES cores for the encryption part and used four Karatsuba-Ofman (KO) multipliers in order to realize the authentication part. Their design reaches a throughput of 119.3 Gbit/s.

The most complex operation during the computation of a message digest according to GCM is the multiplication in the binary finite-field GF (2 128 ), which is part of the universal hashing function called GHASH. Therefore, most of the effort in improving GCM implementations has been spent on speeding up this calculation. Wang et al. [START_REF] Wang | High-speed architectures for GHASH based on efficient bit-parallel multipliers[END_REF] presented a GHASH architecture based on four GHASH cores that achieved a throughput of 123.1 Gbit/s on a Virtex-5. Crenne et al. [START_REF] Crenne | Efficient key-dependent message authentication in reconfigurable hardware[END_REF] reached 238.1 Gbit/s by using 8 parallel finite-field multipliers, also targeting a Xilinx Virtex-5 FPGA. Since we aim at a full AE architecture, i.e., a design including both the authenticity and the confidentiality part, we do not consider these GHASH-only implementations for our investigations.

To the best of our knowledge, no hardware architecture based on a block cipher other than AES and targeting a high-throughput AE implementation has been presented so far. Moreover, no AES design has been published to date, which makes use of an operation mode different than GCM in order to achieve throughputs up to 100 Gbit/s.

100 Gbit/s Authenticated Encryption Alternatives

In order to reach throughputs exceeding 100 Gbit/s on commercial FPGA devices, it is necessary to use multiple parallel instances of cryptographic primitives. Although AES running in GCM mode is currently the most widespread option for high-throughput hardware architectures, using these cryptographic primitives is not a requirement. Different block ciphers and modes of operation, like the ones presented in the following sections, can be used for a throughputoriented AE system as well.

Serpent Block Cipher

Serpent was the runner-up of the AES block cipher competition. Although it has not been chosen by the NIST during the competition, it was considered to be a close alternative and is still known to be secure from a cryptographic point of view as the considerably large number of rounds contributes to its security [START_REF] Nechvatal | Report on the Development of the Advanced Encryption Standard (AES)[END_REF]. In the following we will briefly discuss the main components of Serpent, i.e., the key schedule and the cipher itself, using the conventional implementation described in the official proposal [START_REF] Anderson | Serpent: A Proposal for the Advanced Encryption Standard[END_REF]. In order to change from the conventional to the bitslice version of Serpent1 , all instances of the initial and the final permutation have to be omitted.

Cipher. Fig. 2 illustrates the Serpent cipher which consists of an initial permutation (IP ), 32 round transformations, and a final permutation (F P ). The first 31 rounds of the cipher include a key-mixing stage, a substitution stage, and an avalanche stage (i.e., a stage where a linear transformation takes place). In the last round of the cipher, the linear transformation is omitted and replaced by another key-mixing operation. Serpent makes use of eight different S-boxes (S i , i ∈ {0 . . . 7}) which repeat themselves every eighth round as shown in Fig. 2. Note that only a single S-box is used within each round of the cipher.

Key Schedule. The key schedule of Serpent takes a 256-bit cipher key K and expands it to thirty-three 128-bit subkeys denoted by Ki . Cipher keys shorter than 256 bits are padded by appending a single "1", followed by as many "0"s as required in order to reach a length of 256 bits. After padding, K gets expressed using eight 32-bit values, i.e., K = {ω -8 , . . . , ω -1 }, and extended to an intermediate key {ω 0 , . . . , ω 131 } according to

ω i = (ω i-8 ⊕ ω i-5 ⊕ ω i-3 ⊕ ω i-1 ⊕ φ ⊕ i) <<< 11 , i ∈ {0 . . . 131} ,
where <<< i denotes a rotate-left function by i bits and φ = 0x9E3779B9, i.e., the 32-bit value of the fractional part of the golden ratio. The actual subkeys, which are required during the round transformations of the cipher, are finally obtained by

Ki = IP (S (3-i) mod 8 (ω 4i , ω 4i+1 , ω 4i+2 , ω 4i+3 )) , i ∈ {0 . . . 32} ,
where S i refers to one of the eight Serpent S-boxes. Similar to the cipher, the bitslice implementation of the Serpent key schedule can be obtained by removing all instances of the initial permutation IP . For a detailed description of Serpent, including additional information about the initial and the final permutation, we refer the reader to the official Serpent proposal [START_REF] Anderson | Serpent: A Proposal for the Advanced Encryption Standard[END_REF].

Offset CodeBook Mode

The offset codebook (OCB) block cipher mode of operation is a combined AE scheme and has first been published by Rogaway et al. [START_REF] Rogaway | OCB: A Block-Cipher Mode of Operation for Efficient Authenticated Encryption[END_REF] in 2001. It is strongly related to the Integrity Aware Parallelizable Mode (IAPM) by C. Jutla [START_REF] Jutla | Encryption Modes with Almost Free Message Integrity[END_REF] and three different versions have been made public since 2001. Throughout the remainder of this work we solely refer to the third version of it, i.e., OCB3 [START_REF] Krovetz | The Software Performance of Authenticated-Encryption Modes[END_REF].

To start the authenticated encryption scheme according to OCB, a plaintext message, denoted by M , gets split into m different blocks, each of length n and an optional block M * of length smaller than n as follows2 :

M = M 1 , . . . , M m , if |M | = k • n and k ∈ N , M 1 , . . . , M m , M * , else .
Algorithm 1 and Fig. 3 describe the authenticated encryption according to OCB using pseudo-code and a block diagram, respectively. For simplicity, only Algorithm 1 OCB authenticated encryption.

Input: Message M , Message block length n, Cipher key K, Nonce N , Associated data A, Authentication tag length τ , 0 ≤ τ ≤ 128 Output: Ciphertext C, Authentication tag T 1: if |N | ≥ n then return INVALID 2: {M1, . . . , Mm, M * } ← M , with |Mi| = n and |M * | < n 3: Checksum ← 0 128 ; C ← 0 128 4: L * , L $ , L[0] . . . L[ log2(m) ] ← Setup(K, m) 5: ∆ ← Init(N, n, K) 6: for i = 1 to m do 7: ∆ ← ∆ ⊕ L[ntz(i)] Inci(∆) 8: C ← C||EK (Mi ⊕ ∆) ⊕ ∆ 9: Checksum ← Checksum ⊕ Mi 10: end for 11: if M * = ∅ then 12: ∆ ← ∆ ⊕ L * Inc * (∆) 13: P ad ← EK (∆) 14: C ← C||M * ⊕ (P ad ∧ (1 |M * | )) 15: Checksum ← Checksum ⊕ M * 10 * , with M * 10 * = M * 1 0 . . . 0, such that |M * 10 * | = n 16: end if 17: ∆ ← ∆ ⊕ L $ Inc $ (∆) 18: F inal ← EK (Checksum ⊕ ∆) 19: Auth ← HashK (A) 20: T ag ← F inal ⊕ Auth 21: T ← trunc(T ag, τ ) 22: return C||T
the cases for full message blocks, i.e., M * = ∅ is shown in Fig. 3. The cipher starts with a setup and initialization step (cf. line 4 and 5 in Algorithm 1). Thereafter, each message block can be processed independently of each other (line 6 to 16). Finally, the authentication tag T is determined throughout line 17 to 21. The characters ||, ⊕, and ∧ represent the concatenation, bitwise exclusive or, and bitwise and operation. The term ntz(i) describes the number of trailing zeroes of i in binary representation. 0 n and 1 n stand for bit strings of length n containing only zeros and ones, respectively. Furthermore, trunc(X, y) truncates the bit string X to its y least significant bits. We use ∅ to represent an empty set. Appendix A provides listings for the Setup, Init, and Hash K procedures used throughout Algorithm 1.

When using a counter for the nonce N , the calculation of the initial offset ∆ requires a block cipher call only every 64th initialization. This is due to the fact that the least significant six bits of N are set to zero before passing it to the block cipher (cf. line 3 of Algorithm 3). This fact together with the parallelizable processing of the message blocks, makes OCB suitable for highthroughput applications.

E K E K A 1 A 2 E K Auth E K Checksum T τ F inal T ag A p E K E K E K M 1 M 2 M 3 C 1 C 2 C 3 E K M m C m Inc1(∆) Inc3(∆) Incm(∆) Inc1(∆) Inc2(∆) Inc2(∆) Inc3(∆) Incm(∆) Inc1(∆) Inc2(∆) Incp(∆) Inc$(∆)
Fig. 3. Overview of the encryption and the authentication part of OCB

OCB-Serpent Hardware Architecture

For the OCB-Serpent architecture, supporting the IEEE 802.3ba Ethernet standard, we assume the following prerequisites:

-The size of the message block counter i is restricted to 7 bits, since 2 7 message blocks are capable of hosting a full Ethernet frame. -As our target application ensures solely full message blocks, we do not handle short final message blocks separately.

As previously mentioned, to achieve extremely high throughputs, a multicore approach has proven to be the only viable option when implementing AE on commercial FPGA platforms. Similar to GCM, OCB also allows two successive message blocks to be processed independently of each other. We have taken advantage of this fact and decided to use four parallel cipher cores in order to achieve the desired throughput. Fig. 4 illustrates the OCB architecture for authenticated encryption based on four Serpent cores.

Pipelined Four-Core Serpent Architecture

Each of the four Serpent cores handles a single 128-bit message block. Therefore, the overall design can process a 512-bit message at a time. As can be seen from Fig. 4, we fully unrolled the 32 rounds of Serpent. Furthermore, we inserted pipeline stages after each round in order to increase the maximum frequency of the cipher cores. Although the pipelined architecture allows us to clock the Serpent cores at a higher frequency, one problem inherent to all pipeline architectures has to be taken into consideration: When the normal flow of operations has to be suspended, the entire pipeline must be stopped in order to allow the rest of the operation to resume. Such an occurrence is known as a pipeline stall and can, for instance, occur during a key change.

Due to the unrolling of the Serpent rounds, we have to realize 1024 of the 4-bit S-boxes for each core, which requires a considerable amount of resources. The subkeys for the key-mixing stage of all four Serpent cores are provided from a single key schedule, i.e., the cores always operate on the same cipher key.

OCB -Authenticated Encryption

OCB can, in general, be subdivided into three stages: Initialization, encryption/authentication, and finalization. During the initialization phase, two potential pipeline stalls may occur if not handled properly. First, after each key change a cipher call E K (0 128 ) is required in order to compute the table values L[..] (cf. Algorithm 2). Second, each new message needs a fresh nonce N , and thus a new offset value ∆. Since the calculation of the initial offset also requires a cipher call, this may result in another pipeline stall. In order to reduce the number of pipeline stalls to a minimum, we precompute the initial offset values.

The limitation of message lengths to a maximum of 2 7 blocks facilitates the precomputation of the L[..]-values, as it limits the maximum number of trailing zeroes of the block counter i to six. Thus, only L $ , L * , and L 0 . . . L 6 have to be precomputed and stored in registers. In fact, when the result of E K (0 128 ) is available, all nine table values can be computed in a single clock cycle 3 .

When processing a block in authenticated-encryption mode, the message gets XOR-ed with the current offset ∆ i , encrypted, and finally XOR-ed with ∆ i again. As pipeline stages were introduced into the cipher cores, the ∆-values either have to be stored or recalculated. We decided to recompute the offsets as it makes the implementation less dependent on the underlying block cipher and the number of pipeline stages used. Since the multi-core design processes four message blocks in parallel, the offset-calculation needs to be capable of providing four offset values at a time. In fact, the calculation of the four offset values is relatively cheap as the initial offset only has to be XOR-ed with the precomputed table values.

As described in Algorithm 3, a nonce-dependent call to the encryption of the block cipher is required. The result of this operation, further-on called Ktop, then has to be shifted by a 6-bit nonce-dependent value Bottom. First, in order to be able to perform this shift-operation, Bottom has to be buffered until the result of E K (T op) is available. Second, the 6-bit variable shift is done using a 192-bit by 6-bit barrel shifter. Although using a counter for the nonce N could avoid the resource-expensive barrel shifter, we decided to keep it in order to stay independent of the actual structure of the chosen nonce.

Decryption

Authenticated decryption according to OCB is very similar to the encryption process. Exchanging the encryption operation E K of the underlying cipher by the decryption operation D K in line 8 of Algorithm 1, turns OCB into decryption mode. However, the other encryption operations remain. Therefore, the multicore decryption unit contains four block-cipher decryption cores, one encryption core, and a common key schedule. In order to assure authenticity of a provided message, the re-calculated message tag T must be equal to the tag T , received from the opposite communicating party.

A minor drawback of authenticated decryption according to OCB is the fact that a delay, dependent on the number of pipeline stages p between the calculation of the plaintext and the calculation of the authentication tag exists. This delay is caused by the calculation of the authentication tag which requires the Checksum of the plaintext. Therefore, in order to verify the authentication tag of a message, p plaintext blocks have to be buffered, resulting in additional memory requirements.

Results

We coded our architectures in VHDL. For the synthesis and place&route design steps, we used Altera Quartus II version 11.0. Functional correctness was verified with Modelsim 6.6e simulator. Synthesis was conducted using a speed-optimzed setting. Except of M9K block memories, no Altera-specific logic blocks had been used. In order to have some reference implementations regarding AES and GCM on our target platform (Altera Stratix IV), we also synthesized a four-core AES cipher architecture as well as GCM based on both Serpent and AES. The AES architectures were accomplished with an underlying four-core AES similar to the one proposed by Henzen et al. [START_REF] Henzen | FPGA Parallel-Pipelined AES-GCM Core for 100G Ethernet Applications[END_REF]. We used the 128-bit version of AES and fully unrolled the 10 rounds. Furthermore, pipelining registers have been inserted after each round similar to the Serpent cipher core design.

The first two rows of Table 1 contain the place&route results of the multicore encryption architectures (i.e., without running any mode of operation). The subsequent rows present the results for the different combinations of block ciphers and modes of operation. Regarding the block ciphers, the fully unrolled four-core AES design requires less area as it only has to provide 160 8-bit Sboxes for each cipher core, compared to the 1024 4-bit S-boxes needed by the Serpent cores. For the AES cores we utilized the M9K memory blocks in order to implement the 160 S-boxes, whereas for Serpent we implemented them solely in look-up tables. One of the reasons for this design decision was the significant routing overhead, which would have been required for the 1024 Serpent S-boxes being realized in M9K memory blocks. Since the high-throughput universal hash function GHASH of the GCM mode occupies a lot of area, OCB designs result in a smaller footprint than their GCM counterparts.

Regarding the throughput, all architectures met the target of 100 Gbit/s. However, the OCB versions are considerably faster. This is mainly because of the simpler architecture of OCB compared to GCM, which requires the resourceexpensive GHASH function. Table 1 shows that the critical path of the GCMbased architectures is dominated by the authentication part, whereas the OCBbased designs almost reach the maximum frequency of the cipher-only implementations (with minor exceptions which are most likely due to placement and routing disparities). Compared to our results in [START_REF] Muehlberghuber | 100 Gbit/s Authenticated Encryption Based on Quantum Key Distribution[END_REF], we were able to further increase the maximum frequency of both our cipher-only architectures as well as the designs based on OCB. Although our results show that OCB is at least as suitable for high-throughput hardware implementations as GCM, the latter is still the preferred mode of operation in the literature when designing high-speed authenticated encryption hardware architectures based on block ciphers. This might be due to the facts that there are some US patents on OCB and that, in contrast to GCM, it has not been recommended by the NIST. Regarding the patents on OCB, its author has recently eased licensing for a variety of applications [START_REF] Rogaway | OCB Free Licenses[END_REF] what may increase the popularity of OCB in the near future. Table 2 summarizes the properties of the two AE block cipher modes of operation. One benefit of GMC might be that it solely requires the encryption of the underlying block cipher whereas OCB also needs the decryption.

Our designs have been tested on a self-designed printed circuit board (PCB), which has solely been developed for high-speed authenticated encryption architectures running on an Altera Stratix IV FPGA.

100 Gbit/s Authenticated Encryption System Design

The AE core, described in Section 4, was developed as part of a larger FPGAbased system that will be used to encrypt data on IEEE 802.3ba Ethernet connections, allowing data rates of up to 100 Gbit/s. Designing a real system that is able to reliably process such high data-throughputs poses some formidable challenges. We have successfully developed a complete FPGA-based system working at 40 respectively 100 Gbit/s and will present both the digital part and the PCB development throughout the next sections.

FPGA Digital Design

Processing data at a rate of up to 100 Gbit/s by itself is per se rather challenging. However, transporting this amount of data to and from the processing cores is also a significant problem. In this system we have decided to aggregate plaintext data from ten separate 10 Gbit/s Ethernet links into a single 100 Gbit/s data stream. This data is then encrypted and an authentication tag is determined using the AE schemes mentioned earlier. The resulting ciphertext data stream is then transmitted over a single 100 Gbit/s Ethernet link. As illustrated in Fig. 5, the receiving path of the system works similarly in the opposite direction. The system ensures that the AE remains transparent for all 10 Gbit/s clients connected to the system. Transmitting Path. The plaintext, received from the 10 Gbit/s Ethernet clients, arrives as a serial data stream. In the 10 Gbit/s Ethernet Physical Coding Sublayer (PCS) block, this data is parallelized and prepared for further operations. Note that in our implementation we do not require a Media Access Control (MAC) unit. Instead, we directly aggregate and encrypt the received data stream. The TDM Encoder collects the data from each PCS and transfers it to the Authenticated Encryption core, where the data is encrypted and a corresponding authentication tag is generated. The Channel Sync unit encapsulates the encrypted data and its authentication tag into a TDM Ethernet frame. To ensure fast resynchronization after a connection or packet loss, a synchronization frame is inserted every millisecond into the 100 Gbit/s stream. The synchronization frame is also used to transmit parameters such as the current initialization vector for GCM and the nonce for OCB. The generated frames from the Channel Sync block are prepared for transmission and serialized in the 100 Gbit/s PCS.

The PCS uses ten 10 Gbit/s serial data streams to transmit the data. One problem during transmission is the synchronization loss due to various effects such as electrical and optical multiplexing in the optical CFP modules and small differences in the length of electrical traces on the PCB. As a result, the serial streams may arrive at the receiver out of order. Therefore, a mechanism is required to reorder and realign the serial streams. Unique alignment markers for each stream are inserted every 100 µs to enable synchronization at the receiver.

Receiving Path. On the receiving path, the PCS deserializes the incoming 100 Gbit/s Ethernet transmission into ten 10 Gbit/s data streams. These data streams are then reordered and possible delays are compensated by utilizing the alignment markers inserted during the transmission. The system is capable of compensating up to 200 ns of delay in this way. In the next step, the now parallelized 100 Gbit/s data stream is decoded by the receiving Channel Sync. If a synchronization frame is detected, parameters for the AE cores are extracted from this frame and applied for the following data frames. When a TDM Ethernet frame is detected, the payload is extracted and sent to the Authenticated Decryption core. In the TDM Decoder, the decrypted data stream is distributed to the corresponding 10 Gbit/s PCS units. In addition, the calculated authentication tag from the Authenticated Decryption block is compared to the received one. If an authentication failure is detected, an alert flag is set. The system can be configured to react with further measures, such as purging its input data for that channel.

An important problem of the system is clock synchronization. It occurs if the clock on the receiving side of the system differs (slightly) from the transmitting side. According to the Ethernet standard, the maximum allowed clock mismatch is 100 ppm. To be able to compensate these mismatches, the system can enlarge or shrink the gap between two Ethernet frames.

System Configuration. The system on the FPGA can be configured and monitored via a USB connection in the development board. Individual 10 Gbit/s links can be disabled, the encryption can be turned on or off, and secret key sizes can be determined through this interface. In addition, the encryption keys are also submitted via this interface. Moreover, the same connection can be used to monitor the operation. Statistical data such as number of transmitted and received frames, status of the Authenticated En/Decryption blocks, or presence and link activity of SFP+, XFP, CFP, and CXP modules can be observed using the configuration interface.

Performance. Although our development board and the AE cores have been designed to support a 100 Gbit/s communication, real-world experiments have so far only been undertaken using a 40 Gbit/s ciphertext interface due to financial reasons 4 . Nevertheless, measurements of the overall system proved it to be operational at data rates up to 40 Gbit/s with all the features described above. The whole digital system showed a constant latency of 3.5 µs for all 10 Gbit/s Ethernet links when configured with OCB-Serpent. Correct transmission of frames up to a length of 16,000 Bytes was observed. The total power dissipation of the overall development board is 45 W, thereof 14 W are consumed solely by the FPGA. If configured with OCB-Serpent, the FPGA's utilization corresponds to 32 %. Note that when operating in the 40 Gbit/s configuration, only two fully unrolled encryption cores for the transmitting path and two fully unrolled decryption cores for the receiving path are required.

PCB Design

Designing such a complex system in one shot is, in our opinion, not a sound engineering strategy. While reconfiguration of the FPGA does not pose a problem at all, the design of the underlying PCB is "rather statical". Therefore, we have adopted a two-stage design process where we have developed two PCBs. The first system shown in Fig. 6, with its main features listed in Table 3, was used as a development board with all the main components and allowed us to test the basic functionality. The second PCB is the final prototype, and in addition to fixing problems detected in the first design, also adds a number of changes to meet the industrial requirements.

In this section, we will describe the main problems (power distribution and signal quality) we have encountered while designing the development system and will also explain the optimizations we have performed for a follow-up board.

Power Distribution. The first challenge in the system design is establishing the connections to the FPGA which uses a 1932-pin BGA package. While the signal connections offer a formidable challenge in terms of routing, the real problem is in power routing. The system uses in total 14 different power supplies and the main digital power supply of 0.95 V was estimated to consume as much as 48 A. The only practical solution to supply the FPGA with stable power is using several dedicated low-impedance power planes. As a result of these considerations, a 24-layer stack was designed for the development PCB. Even though we had anticipated problems with the 0.95 V supply, measurements showed that the voltage-drop across the power plane was still too high. Therefore, we have added two additional power planes for the final design.

Power considerations have also dictated the organization of the layer stack. In the development board, high currents were concentrated on thicker lowimpedance layers in the uppermost layers, close to the energy hungry components and their blocking capacitors. However, this asymmetric PCB stack could not be used for the second PCB, since a different manufacturer had to be used. As a result, half of the power layers had to be moved to the bottom for the second PCB.

Signal Quality. As expected, routing several 10 Gbit/s high-speed differential lines across a large PCB turned out to be a challenging task. In total, the development board used sixty impedance-matched differential lines and ten differential clock signals in the frequency range of 156 to 644 MHz. In the development board, these signals were routed on dedicated high-speed layers towards the bottom of the layer stack, where signal quality was not further compromised by longer via stubs. Although utmost care was taken in the design of these differential lines, actual measurements on the board revealed that the attenuation on high-speed signals was critical and problems were detected in impedance matching of the vias. We were able to reach the bit error rates specified in the IEEE 802.3ba standard by programming the transceivers on the FPGA side. As mentioned earlier, for the second PCB a new manufacturer had to be used, which necessitated a change in the layer stack. In the new layer stack, the highspeed signals are placed in the center of the stack, sandwiched between ground planes. We created a 3D model of the new layer stack using CST Microwave Studio as seen in Fig. 7. This allowed us to make detailed simulations on the behavior of differential vias and determined the best possible geometry to be used. Fig. 8 gives the result of the S 11 parameter and simulated TDR behavior of the differential via shown in Fig. 7.

Advanced PCB. By applying a two-stage design approach, we achieved the following goals. First, the design constraints for the initial board are relaxed, allowing the board to be manufactured early in the process. Second, the development board is then actively used throughout the development of the AE core and the surrounding system, allowing real measurements on a representative system. These in turn were used to identify problems in the development board and has guided the design of the final PCB. In addition to the weaknesses detected in the first PCB design, it was decided to make the following changes to meet industrial constraints:

-Added two additional layers to improve power distribution.

-Moved the secret key port from USB to PCIe to improve throughput.

-Removed the CXP active cable interface which was deemed to be unnecessary for the application. -Replaced the two XFP modules by two SFP+ modules.

-Added electronic dispersion compensation (EDC) to six of the SFP+ modules.

-Adapted the dimensions of the PCB to better comply with the requirements of the industrial partner.

However, our two-stage approach also had some drawbacks. The design parameters for such complex PCB systems are not standardized, and most of these parameters need to be negotiated with the PCB manufacturer directly. If for some reason, the PCB manufacturer has to be changed, it is likely that the chosen parameters can not be reused, necessitating time-consuming re-design work. In our case, we were forced to change manufacturers as the initial manufacturer filed for bankruptcy. It proved to be quite difficult to find an alternative PCB manufacturer, 12 out of the 14 companies we have contacted refused the design due to its high complexity.

The advanced PCB now has 26 layers and is 3.3 mm thick. This thickness posed additional challenges for drill holes. The maximum practical aspect ratio (thickness/diameter) for drill holes is around 16:1. Therefore, the thickness of the board directly determines the minimum diameter of the vias that can be used. Since smaller vias are required for impedance matching of 10 Gbit/s differential lines, a delicate balancing act is necessary to reconcile the demands of signal quality on one hand and safe manufacturability on the other hand.

As a consequence of both the increased number of layers, and the change of the manufacturer, we were no longer able to use vias that were 0.22 mm in diameter, but had to adjust the minimal via size to 0.25 mm. This change alone required significant re-design effort.

Conclusion

In this work, we described a hardware architecture for high-speed authenticated encryption (AE) using block ciphers on FPGAs, based on alternative cryptographic primitives. Our design operates in the offset codebook (OCB3) mode of operation and contains four parallel Serpent block cipher cores for the encryption part in order to achieve the desired data rates according to IEEE 802.3ba. The OCB3-Serpent architecture reaches a throughput of 141 Gbit/s and thus, outperforms all GCM-AES implementations on FPGAs available to date. Although OCB3 has not (yet) been approved by the NIST, its structure (as well as that of Serpent) is definitely suitable for high-throughput implementations as shown during this work. Moreover, we present a custom-designed printed circuit board for the Stratix IV FPGA, which allows us to use the presented AE schemes in real-world applications processing data with up to 100 Gbit/s.

A OCB Algorithms

Algorithm 2 lists the calculation of the table values L[..] required during the OCB mode of operation. The double-procedure is defined according to: double(X) = (X 1) ⊕ (msb(X) • 0x87),

where msb(X) represents the most significant bit of X using binary representation. The initial offset ∆ is determined according to Algorithm 3. X i denotes a left shift operation of X by i bits.

Algorithm 2 Table value calculations.

Algorithm 4 describes the calculation of Hash K (A). Since the Setup procedure already gets called during the actual encryption process of OCB (cf.

Fig. 4 .

 4 Fig. 4. OCB-Serpent architecture

Fig. 5 .

 5 Fig. 5. Block diagram of the whole encryption system on the FPGA

Fig. 6 .

 6 Fig. 6. Overview of the 100 Gbit/s AE development board

Fig. 7 .

 7 Fig. 7. 3D simulation of a differential via from an impedance-matched wire on top to a wire of layer 14 of the pad stack

Fig. 8 .

 8 Fig. 8. Reflexion coefficient S11 of the simulated differential via in Fig. 7 and simulated, differential time-domain reflectometry (TDR) result

Input:

  Cipher key K, Number of message blocks m Output: Setup(K, m)1: L * ← EK (0 128 ) 2: L $ ← double(L * ) 3: L[0] ← double(L $ ) 4: for i = 1 to log2(m) do 5: L[i] ← double(L[i -1]) 6: end for 7: return L * , L $ , L[0] . . . L[ log2(m) ]

Algorithm 3 1 : 2 :

 312 Initial offset (∆) calculation. Input: Nonce N , Message block length n, Cipher key K Output: Init(N, n, K) Bottom ← N ∧ 1 6 Bottom = Least six LSBs of N 2: N once ← 0 127-|N | ||1||N 3: T op ← (1 122 ||0 6 ) ∧ N once Zeroing out least six LSBs of N once 4: Ktop ← EK (T op) 5: Stretch ← Ktop||(Ktop ⊕ (Ktop 8)) 6: ∆ ← (Stretch Bottom) ∧ 1 n Use first n bits of Stretch Bottom 7: return ∆ Algorithm 1, line 4), line 3 in Algorithm 4 can be omitted as long as the table values L[..] are globally available. Algorithm 4 Authentication hash (Hash K (A)) calculation. Input: Associated data A, Associated data block length q, Cipher key K Output: HashK (A) 1: {A1, . . . , Ap, A * } ← A, with |Ai| = q and |A * | < q Sum ← 0 128 ; ∆ ← 0 128 3: L * , L[0] . . . L[ log2(p) ] ← Setup(K, p) 4: for i = 1 to p do 5: ∆ ← ∆ ⊕ L[ntz(i)] Inci(∆) 6: Sum ← Sum ⊕ EK (Ai ⊕ ∆) 7: end for 8: if A * = ∅ then 9: ∆ ← ∆ ⊕ L * Inc * (∆) 10: Sum ← Sum ⊕ EK (A * 10 * ⊕ ∆), with A * 10 * = A * 1 0 . . . 0, such that |A * 10 * | = q 11: end if 12: return Sum

Table 1 .

 1 Encryption-only and authenticated encryption results targeting an Altera Stratix IV (EP4S100G5F45) platform using four cipher cores

	Block	Mode of	Area		fmax	Throughput
	Cipher	Operation	[ALMs] [M9K Bl.]	[MHz] [Gbit/s] [%]
	Cipher-Only Architectures
	Serpent	cipher-only	28,399	0	281	144 136
	AES	cipher-only	7,661	314	267	137 130
	Authenticated Encryption Architectures
	Serpent	OCB	38,312	0	275	141 133
	AES	OCB	11,948	314	250	128 121
	Serpent	GCM	56,474	0	203	104	99
	AES	GCM	24,313	314	206	105 100

Table 2 .

 2 Block cipher modes of operation comparison

	Property	OCB	GCM
	Patented	Yes	No
	Parallelizable	Yes (Encr. + Auth.)	Yes (Encr. + Auth.)
	Decryption required	Yes	No
	Cipher calls (Initialization)	1	1
	Cipher calls (Encryption)	|M |/n + 1.016 1	|M |/n + 1
	1 Applies as long as the associated data is fixed during a single session and a counter is
	used for the nonce.

Table 3 .

 3 Main components of the development board for the 100 Gbit/s AE system

	Component		Type		Description
	Networking	and	Altera Stratix IV GT	FPGA model EP4S100G5F45 with high-speed
	encryption engine			transceivers, 1932-BGA
	Board controller	Altera MAX II	CPLD EPM2210F256
	Plaintext interfaces	8 SFP+		10 Gbit/s Ethernet interfaces. Short range
					(4 SFP+ prepared for Fibre Channel)
			2 XFP		10 Gbit/s Ethernet interfaces. Medium rage
	Ciphertext	inter-	1 CFP		40 Gbit/s or 100 Gbit/s wavelength multiplexing
	faces				four or ten 10 Gbit/s electrical streams per direc-
					tion
			1 CXP		100 Gbit/s active cable using ten 10 Gbit/s fibres
					per direction. Short range
	USB interface	Cypress	EZ-USB	For configuration, key transfer, and statistics
			FX2LP
	Power system	4 LMT4601		4-phase 0.95 V FPGA core supply. max. 48 A
			6 further switched	Digital supplies and analog pre-supplies
			regulators
			13 linear regulators	Analog and timing block supplies
	Clocking system	7 oscillators		System clocks, Transceiver clocks, 24 ... 644 MHz
	PCB		NELCO	NP400-	24 layers, 387 mm × 220 mm × 3 mm, 1175 com-
			13EP		ponents

We refer to the Serpent proposal[START_REF] Anderson | Serpent: A Proposal for the Advanced Encryption Standard[END_REF] for further information on the bitslice implementation.

We refer to the length of x in bits using the following notation: |x|

The calculation only depends on operations cheap to implement in hardware, i.e., fixed shift and conditional exclusive or operations.

The 100 Gbit/s CFP module is about eight to ten times more expensive than the 40 Gbit/s module.

Acknowledgement

This work is part of the QCRYPT project, which is evaluated by the Swiss National Science Foundation and financed by the Swiss Confederation with funding via Nano-Tera.ch.

The authors would like to thank the entire team from the Microelectronics Design Center at the ETH Zurich for their help during the development of the printed circuit board as well as Christian Pendl from Graz University of Technology for his contributions to the digital part of the system.