SPEC® MPIM2007 Result

Copyright 2006-2010 Standard Performance Evaluation Corporation

AMD, QLogic Corporation, Rackable Systems, IWILL

AMD Emerald Cluster: AMD Opteron CPUs,
QLogic InfiniPath/SilverStorm Interconnect

SPECmpiM_peak2007 = Not Run

MPI2007 license: 0018 Test date: May-2007
Test sponsor: QLogic Corporation Hardware Availability: Nov-2006
Tested by: QLogic Performance Engineering Software Availability: Jul-2007
Benchmark results graph

Results Table

Benchmark Base Peak
Ranks Seconds Ratio Seconds Ratio Seconds Ratio Ranks Seconds Ratio Seconds Ratio Seconds Ratio
Results appear in the order in which they were run. Bold underlined text indicates a median measurement.
104.milc 256 83.0 18.8  85.4 18.3  82.3 19.0 
107.leslie3d 256 316   16.5  351   14.9  361   14.4 
113.GemsFDTD 256 867   7.28 889   7.10 886   7.12
115.fds4 256 84.9 23.0  87.0 22.4  86.3 22.6 
121.pop2 256 339   12.2  373   11.1  375   11.0 
122.tachyon 256 207   13.5  195   14.3  203   13.8 
126.lammps 256 229   12.8  230   12.7  229   12.7 
127.wrf2 256 292   26.7  294   26.5  296   26.3 
128.GAPgeofem 256 93.6 22.1  94.9 21.7  95.2 21.7 
129.tera_tf 256 164   16.9  164   16.9  164   16.9 
130.socorro 256 197   19.4  196   19.5  193   19.8 
132.zeusmp2 256 142   21.9  143   21.7  142   21.8 
137.lu 256 102   36.1  109   33.8  111   33.1 
Hardware Summary
Type of System: Homogenous
Compute Node: Rackable, IWILL, AMD
Interconnects: QLogic InfiniBand HCAs and switches
Broadcom NICs, Force10 switches
File Server Node: Headnode NFS filesystem
Head Node: Rackable, IWILL, AMD
Other Node: Headnode NFS filesystem
Total Compute Nodes: 64
Total Chips: 128
Total Cores: 256
Total Threads: 256
Total Memory: 512 GB
Base Ranks Run: 256
Minimum Peak Ranks: --
Maximum Peak Ranks: --
Software Summary
C Compiler: QLogic PathScale C Compiler 3.0
C++ Compiler: QLogic PathScale C++ Compiler 3.0
Fortran Compiler: QLogic PathScale Fortran Compiler 3.0
Base Pointers: 64-bit
Peak Pointers: 64-bit
MPI Library: QLogic InfiniPath MPI 2.1
Other MPI Info: None
Pre-processors: No
Other Software: None

Node Description: Rackable, IWILL, AMD

Hardware
Number of nodes: 64
Uses of the node: compute, head
Vendor: Rackable Systems, IWILL, AMD
Model: Rackable Systems C1000 chassis, IWILL DK8-HTX
motherboard
CPU Name: AMD Opteron 290
CPU(s) orderable: 1-2 chips
Chips enabled: 2
Cores enabled: 4
Cores per chip: 2
Threads per core: 1
CPU Characteristics: --
CPU MHz: 2800
Primary Cache: 64 KB I + 64 KB D on chip per core
Secondary Cache: 1 MB I+D on chip per core
L3 Cache: None
Other Cache: None
Memory: 8 GB (8 x 1 GB DDR400)
Disk Subsystem: 250 GB, SATA
Other Hardware: Nodes custom-built by Rackable Systems. The
Rackable C1000 chassis is half-depth with 450W,
48 VDC Power Supply. Integrated Gigabit Ethernet
for admin/filesystem.
Adapter: Intel 82541PI Gigabit Ethernet controller
Number of Adapters: 1
Slot Type: integrated on motherboard
Data Rate: 1 Gbps Ethernet
Ports Used: 1
Interconnect Type: Ethernet
Adapter: QLogic InfiniPath QHT7140
Number of Adapters: 1
Slot Type: HTX
Data Rate: InfiniBand 4x SDR
Ports Used: 1
Interconnect Type: InfiniBand
Software
Adapter: Intel 82541PI Gigabit Ethernet controller
Adapter Driver: Part of Linux kernel modules
Adapter Firmware: None
Adapter: QLogic InfiniPath QHT7140
Adapter Driver: InfiniPath 2.1
Adapter Firmware: None
Operating System: ClusterCorp Rocks 4.2.1
(Based on RedHat Enterprise Linux 4.0 Update 4)
Local File System: Linux ext3
Shared File System: NFS
System State: Multi-User
Other Software: Sun Grid Engine 6.0

Node Description: Headnode NFS filesystem

Hardware
Number of nodes: 1
Uses of the node: file server, other
Vendor: Tyan
Model: Thunder K8QSD Pro (S4882) motherboard
CPU Name: AMD Opteron 885
CPU(s) orderable: 1-4 chips
Chips enabled: 4
Cores enabled: 8
Cores per chip: 2
Threads per core: 1
CPU Characteristics: --
CPU MHz: 2600
Primary Cache: 64 KB I + 64 KB D on chip per core
Secondary Cache: 1 MB I+D on chip per core
L3 Cache: None
Other Cache: None
Memory: 16 GB (16 x 1 GB DDR400 dimms)
Disk Subsystem: 250 GB, SATA, 7200 RPM
Other Hardware: None
Adapter: Broadcom BCM5704C
Number of Adapters: 2
Slot Type: integrated on motherboard
Data Rate: 1 Gbps Ethernet
Ports Used: 2
Interconnect Type: Ethernet
Software
Adapter: Broadcom BCM5704C
Adapter Driver: Part of Linux kernel modules
Adapter Firmware: None
Operating System: ClusterCorp Rocks 4.2.1
(Based on RedHat Enterprise Linux 4.0 Update 4)
Local File System: Linux ext3
Shared File System: NFS
System State: Multi-User
Other Software: Sun Grid Engine 6.0

General Notes

"other" purposes of this node: login, compile, job submission
and queuing.
This node assembled with a 2U chassis and 700 watt ATX 12V Power Supply.

Interconnect Description: QLogic InfiniBand HCAs and switches

Hardware
Vendor: QLogic
Model: InfiniPath and Silverstorm
Switch Model: QLogic SilverStorm 9120 Fabric Director
Number of Switches: 1
Number of Ports: 144
Data Rate: InfiniBand 4x SDR and InfiniBand 4x DDR
Firmware: 3.4.0.5.2
Topology: Single switch (star)
Primary Use: MPI traffic

General Notes

The data rate between InifniPath HCAs and SilverStorm switches
is SDR. However, DDR is used for inter-switch links.

Interconnect Description: Broadcom NICs, Force10 switches

Hardware
Vendor: Force10
Model: E300
Switch Model: Force10 E300 Gig-E switch
Number of Switches: 1
Number of Ports: 288
Data Rate: 1 Gbps Ethernet
Firmware: N/A
Topology: Single switch (star)
Primary Use: file system traffic

Base Compiler Invocation

C benchmarks:

 /usr/bin/mpicc -cc=pathcc 

C++ benchmarks:

126.lammps:  /usr/bin/mpicxx -CC=pathCC 

Fortran benchmarks:

107.leslie3d:  /usr/bin/mpif90 -f90=pathf90 
113.GemsFDTD:  /usr/bin/mpif90 -f90=pathf90 
115.fds4:  /usr/bin/mpif90 -f90=pathf90 
129.tera_tf:  /usr/bin/mpif90 -f90=pathf90 
132.zeusmp2:  /usr/bin/mpif90 -f90=pathf90 
137.lu:  /usr/bin/mpif90 -f90=pathf90 

Benchmarks using both Fortran and C (except as noted below):

 /usr/bin/mpicc -cc=pathcc   /usr/bin/mpif90 -f90=pathf90 

Base Portability Flags

104.milc:  -DSPEC_MPI_LP64 
121.pop2:  -DSPEC_MPI_DOUBLE_UNDERSCORE   -DSPEC_MPI_LP64 
122.tachyon:  -DSPEC_MPI_LP64 
127.wrf2:  -DF2CSTYLE   -DSPEC_MPI_DOUBLE_UNDERSCORE   -DSPEC_MPI_LINUX   -DSPEC_MPI_LP64 
128.GAPgeofem:  -DSPEC_MPI_LP64 
130.socorro:  -fno-second-underscore   -DSPEC_MPI_LP64 

Base Optimization Flags

C benchmarks:

 -march=opteron   -Ofast   -OPT:malloc_alg=1 

C++ benchmarks:

126.lammps:  -march=opteron   -O3   -OPT:Ofast   -CG:local_fwd_sched=on 

Fortran benchmarks:

107.leslie3d:  -march=opteron   -O3   -OPT:Ofast   -OPT:malloc_alg=1   -LANG:copyinout=off 
113.GemsFDTD:  -march=opteron   -O3   -OPT:Ofast   -OPT:malloc_alg=1   -LANG:copyinout=off 
115.fds4:  -march=opteron   -O3   -OPT:Ofast   -OPT:malloc_alg=1   -LANG:copyinout=off 
129.tera_tf:  -march=opteron   -O3   -OPT:Ofast   -OPT:malloc_alg=1   -LANG:copyinout=off 
132.zeusmp2:  -march=opteron   -O3   -OPT:Ofast   -OPT:malloc_alg=1   -LANG:copyinout=off 
137.lu:  -march=opteron   -O3   -OPT:Ofast   -OPT:malloc_alg=1   -LANG:copyinout=off 

Benchmarks using both Fortran and C:

121.pop2:  -march=opteron   -Ofast   -OPT:malloc_alg=1   -O3   -OPT:Ofast   -LANG:copyinout=off 
127.wrf2:  Same as 121.pop2 
128.GAPgeofem:  Same as 121.pop2 
130.socorro:  Same as 121.pop2 

Base Other Flags

C benchmarks:

 -IPA:max_jobs=4 

C++ benchmarks:

126.lammps:  -IPA:max_jobs=4 

Fortran benchmarks:

107.leslie3d:  -IPA:max_jobs=4 
113.GemsFDTD:  -IPA:max_jobs=4 
115.fds4:  -IPA:max_jobs=4 
129.tera_tf:  -IPA:max_jobs=4 
132.zeusmp2:  -IPA:max_jobs=4 
137.lu:  -IPA:max_jobs=4 

Benchmarks using both Fortran and C (except as noted below):

 -IPA:max_jobs=4 

The flags file that was used to format this result can be browsed at
http://www.spec.org/mpi2007/flags/MPI2007_flags.20070717.01.html.

You can also download the XML flags source by saving the following link:
http://www.spec.org/mpi2007/flags/MPI2007_flags.20070717.01.xml.