For a modern computing center involved in international scientific collaborations , the I/O Capacity is a fundamental factor. The needs for the processing of huge amount of data with a great number of CPUs led CNAF to design a high-performance LAN (Local Area Network) able to provide a data access bandwidth of the order of hundreds Gigabit per second. Each single processing job must be able to support the access data rate required by the main scientific experiments using the Computing Center resources. The minimum data access bandwidth guaranteed to every single job is about 5 MB per second.
Ethernet is the main technology used for the local interconnection and it is present in all the main implementations: 1/10/40/100 GE (Gigabit Ethernet). The Core switches of the TIER1 are redundant modular devices with the capability to host high-density 100GE modules with the scalability of more than 500×100 Gbps ports on each device. The total capacity currently installed on the cores is about 18 Tbps (Terabit per second).
CNAF is connected to the wide area network in many different ways:
- 2×100 Gbps links dedicated to the scientific and research networks LHCOPN (LHC Optical Private Network) connecting all the LHC TIER1s and CERN, LHCONE (LHC Open Network Environment) connecting all LHC TIER1s, TIER2s, BelleII and other main Physics experiments computing centers admitted to this “Overlay Network” (more than 180 sites)
- 2x10Gbps links to General Internet
- 2x10Gbps link dedicated to IT distributed applications and cloud environment.
CNAF implemented one of the first Datacenter Interconnection with CINECA, one of the main scientific High-Performance Computing Center in Europe at a rate of 400 Gbps on a total available bandwidth of 1,2 Tbps (Terabit per second).
The extremely low latency introduced by the two DCI devices (5 microseconds per device) enabled the CNAF LAN extension to CINECA Datacenter (20 Kilometers far) with an RTT (Round Trip Time) lower than 1 ms.
This high bandwidth and low latency connection permit a massive use of computing nodes physically installed in CINECA exactly as they were installed locally at CNAF without any penalty on data access rate.
CNAF is also connected through an L3 VPN the INFN Bari Computing Center (800 Kilometers far) on a 2x10Gbps dedicated link. This interconnection is another DCI example characterized by a lower bandwidth and higher latency (about 8.5 ms).
The WAN connectivity is provided by GARR (The Italian National Research and Educational Network).
In general, all the network connectivity and network security related activities are in charge of the Network group of Tier-1 data center unit.