Beluga Cluster Hardware and Networking Overview

Cluster Nodes

• 13 pcR720 PC nodes (beluga1-beluga13), consisting of:
  • Dell PowerEdge R720 server, 2U height
  • 2x Intel Xeon E5-2680 @ 2.8GHz
    • 10 cores per socket, 2 threads per core: 40 total cores
    • L2 Cache: 256KB
    • L3 Cache: 2.5MB
  • 128GB ECC DDR3 1600 MHz memory
  • 4x Broadcom BCM5720 Gigabit Ethernet interface (onboard, distinguished as bge)
  • 1x Intel X540-T dual-port 10GbE adapter (distinguished as ixgbe)
  • 1x Mellanox MT27500 ConnectX-3 dual-port 40GbE adapter (distinguished as mlx4en)
  • 2x Intel SSDSC2BB160G4T 160GB SATA SSD (sda,sdb)
  • 1x Seagate ST1000NM0033-9ZM173 1TB SATA disk (sdc)

• 6 pcR440 PC nodes (beluga14-beluga19), consisting of:
  • Dell PowerEdge R440 server, 1U height
  • 2x Intel Xeon Silver 4110 CPU @ 2.10GHz
    • 8 cores per socket, 2 threads per core: 32 total cores
    • L2 cache: 1024KB
    • L3 cache: 11MB
  • 128GB DDR4 2666 MHz memory
  • 2x Broadcom NetXtreme BCM5720 Gigabit Ethernet interface (onboard, distinguished as bge)
  • 1x Mellanox MT27700 ConnectX-4 dual-port 100GbE adapter (distinguished as mlx5en)
  • 1x Toshiba MG04ACA100NY 1TB SATA disk (sda)

• 3 pcR230 PC nodes (beluga20-beluga22), consisting of:
  • Dell PowerEdge R230 server, 1U height
  • 1x Intel Xeon CPU E3-1270 v6 @ 3.80GHz
    • 4 cores per socket, 1 thread per core: 4 total cores
    • L2 cache: 256KB
    • L3 cache: 8192KB
  • 64GB DDR4 2666 MHz memory
  • 2x Broadcom NetXtreme BCM5720 Gigabit Ethernet interface (onboard, distinguished as bge)
  • 1x Intel XL710 dual-port 40GbE adapter (distinguished as ixl)
  • 1x Seagate ST2000NM0018-2F3 2TB disk, connected to Avago SAS MegaRAID adapter (sda)

Variations

• beluga3 has 2x single-port Intel XL710 40GbE adapters (distinguished as ixl) instead of the Mellanox ConnectX-3 adapter
• beluga4 and beluga7 each have an additional Netronome Agilio CX Dual-Port 40 Gigabit Ethernet SmartNIC (distinguished as netro)
• beluga12 has an additional Netronome Agilio CX Single-Port 40 Gigabit Ethernet SmartNIC (distinguished as netro)
• beluga16 through beluga19 each have an additional dual-port Intel XL710 40GbE adapter (distinguished as ixl)

High Speed Experiment Networks

• 2x Edgecore Wedge100BF-32X 100 Gigabit Ethernet Switch (tofino1-tofino2)
  • 32x 100GbE QSFP28 ports
  • Barefoot Tofino 3.2T
  • Intel Broadwell DE
  • Dual AC PSU

Experiment Network Connections

The physical cabling of the high-speed network interfaces is frequently reconfigured by users of the cluster for various experimental purposes.

The current configuration of these connections is maintained in this spreadsheet in the form HOST:PORT.

Users are responsible for keeping this document up-to-date as they make changes to the physical cabling of these network interfaces.