SPECaccel(R)2023 Result NVIDIA Corporation Tesla H100 SXM 80GB DGX-H100 accel2023 License: 9045 Test date: Oct-2023 Test sponsor: NVIDIA Corporation Hardware availability: Jul-2023 Tested by: NVIDIA Corporation Software availability: Nov-2023 Base Base Base Base Peak Peak Peak Peak Benchmarks Model Ref. Run Time Ratio Model Ref. Run Time Ratio -------------- ------ ------ --------- --------- ------ ------ --------- --------- 403.stencil TGT 440 209 2.11 S TGT 440 209 2.11 S 403.stencil TGT 440 209 2.10 * TGT 440 209 2.10 * 404.lbm TGT 455 105 4.32 S TGT 455 105 4.32 S 404.lbm TGT 455 108 4.23 * TGT 455 108 4.23 * 450.md TGT 600 187 3.21 S TGT 600 187 3.21 S 450.md TGT 600 187 3.21 * TGT 600 187 3.21 * 452.ep TGT 415 105 3.94 * TGT 415 105 3.94 * 452.ep TGT 415 105 3.94 S TGT 415 105 3.94 S 453.clvrleaf TGT 1000 491 2.04 * TGT 1000 491 2.04 * 453.clvrleaf TGT 1000 491 2.04 S TGT 1000 491 2.04 S 455.seismic TGT 780 233 3.35 S TGT 780 233 3.35 S 455.seismic TGT 780 233 3.35 * TGT 780 233 3.35 * 456.spF TGT 475 98.9 4.80 * TGT 475 98.9 4.80 * 456.spF TGT 475 98.9 4.80 S TGT 475 98.9 4.80 S 457.spC TGT 540 134 4.02 * TGT 540 134 4.02 * 457.spC TGT 540 134 4.03 S TGT 540 134 4.03 S 459.miniGhost TGT 590 292 2.02 * TGT 590 292 2.02 * 459.miniGhost TGT 590 292 2.02 S TGT 590 292 2.02 S 460.ilbdc TGT 555 156 3.55 * TGT 555 156 3.55 * 460.ilbdc TGT 555 156 3.56 S TGT 555 156 3.56 S 463.swim TGT 440 134 3.27 * TGT 440 134 3.27 * 463.swim TGT 440 132 3.33 S TGT 440 132 3.33 S 470.bt TGT 1055 165 6.41 S TGT 1055 165 6.41 S 470.bt TGT 1055 165 6.41 * TGT 1055 165 6.41 * ============================================================================================ 403.stencil TGT 440 209 2.10 * TGT 440 209 2.10 * 404.lbm TGT 455 108 4.23 * TGT 455 108 4.23 * 450.md TGT 600 187 3.21 * TGT 600 187 3.21 * 452.ep TGT 415 105 3.94 * TGT 415 105 3.94 * 453.clvrleaf TGT 1000 491 2.04 * TGT 1000 491 2.04 * 455.seismic TGT 780 233 3.35 * TGT 780 233 3.35 * 456.spF TGT 475 98.9 4.80 * TGT 475 98.9 4.80 * 457.spC TGT 540 134 4.02 * TGT 540 134 4.02 * 459.miniGhost TGT 590 292 2.02 * TGT 590 292 2.02 * 460.ilbdc TGT 555 156 3.55 * TGT 555 156 3.55 * 463.swim TGT 440 134 3.27 * TGT 440 134 3.27 * 470.bt TGT 1055 165 6.41 * TGT 1055 165 6.41 * SPECaccel 2023_base 3.38 SPECaccel 2023_peak 3.38 HARDWARE -------- CPU Name: Intel Xeon Platinum 8480CL Max MHz.: 3800 Nominal: 2000 Enabled: 112 cores, 2 chips, 2 threads/core Orderable: 2 chips Cache L1: 32 KB I + 48 KB D on chip per core L2: 2 MB I+D on chip per core L3: 105 MB I+D on chip per chip Other: None Memory: 2 TB (32 x 64 GB 2Rx8 PC5-4800B-R) Storage: OS: 2x 1.9TB NVMe M.2 Internal storage: 8x 3.84TB NVMe U.2 Other: None Base Threads Run: 1 Min. Peak Threads: 1 Max. Peak Threads: 1 ACCELERATOR ----------- Accel Model Name: H100 SXM 80GB Accel Vendor: NVIDIA Accel Name: Tesla H100 SXM 80GB Type of Accel: GPU Accel Connection: NVLink 4 Does Accel Use ECC: Yes Accel Description: System contains 8 H100 SXM with GPU 0 used for measurement. GPU contain 80GB of memory. Accel Driver: NVIDIA UNIX Open Kernel Module for x86_64 535.104.05 SOFTWARE -------- OS: Ubuntu 22.04.2 LTS 5.15.0-1025-nvidia Compiler: C/Fortran: Version 23.11 of NVHPC SDK Firmware: 1.1.1 07/31/2023 File System: ext4 System State: Run level 3 (multi-user) Other: None Base Parallel Model: TGT Base Threads Run: 1 Peak Parallel Models: TGT Max. Peak Threads: 1 Min. Peak Threads: 1 Operating System Notes ---------------------- Shell stacksize set to unlimited via "limit stacksize unlimited" General Notes ------------- Environment variables set by runaccel before the start of the run: LD_LIBRARY_PATH = "/var/data0/sandbox/nvuser/SPECACCEL/nv239_libs" Set to the location of the NVHPC compiler runtime libraries. Platform Notes -------------- Sysinfo program /var/data0/sandbox/nvuser/SPECACCEL/bin/sysinfo Rev: r6622 of 2021-04-07 b1a7d5f8f71be5aff70a755cad7211a0 running on VIKING80-DVT-800 Thu Oct 19 16:13:09 2023 SUT (System Under Test) info as seen by some common utilities. For more information on this section, see https://www.spec.org/cpu2017/Docs/config.html#sysinfo From /proc/cpuinfo model name : Intel(R) Xeon(R) Platinum 8480CL 2 "physical id"s (chips) 224 "processors" cores, siblings (Caution: counting these is hw and system dependent. The following excerpts from /proc/cpuinfo might not be reliable. Use with caution.) cpu cores : 56 siblings : 112 physical 0: cores 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 physical 1: cores 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 From lscpu from util-linux 2.37.2: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Address sizes: 52 bits physical, 57 bits virtual Byte Order: Little Endian CPU(s): 224 On-line CPU(s) list: 0-223 Vendor ID: GenuineIntel Model name: Intel(R) Xeon(R) Platinum 8480CL CPU family: 6 Model: 143 Thread(s) per core: 2 Core(s) per socket: 56 Socket(s): 2 Stepping: 7 CPU max MHz: 3800.0000 CPU min MHz: 800.0000 BogoMIPS: 4000.00 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cat_l2 cdp_l3 invpcid_single intel_ppin cdp_l2 ssbd mba ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local split_lock_detect avx_vnni avx512_bf16 wbnoinvd dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req avx512vbmi umip pku ospke waitpkg avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq la57 rdpid bus_lock_detect cldemote movdiri movdir64b enqcmd fsrm md_clear serialize tsxldtrk pconfig arch_lbr amx_bf16 avx512_fp16 amx_tile amx_int8 flush_l1d arch_capabilities L1d cache: 5.3 MiB (112 instances) L1i cache: 3.5 MiB (112 instances) L2 cache: 224 MiB (112 instances) L3 cache: 210 MiB (2 instances) NUMA node(s): 2 NUMA node0 CPU(s): 0-55,112-167 NUMA node1 CPU(s): 56-111,168-223 Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization Vulnerability Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected From lscpu --cache: NAME ONE-SIZE ALL-SIZE WAYS TYPE LEVEL SETS PHY-LINE COHERENCY-SIZE L1d 48K 5.3M 12 Data 1 64 1 64 L1i 32K 3.5M 8 Instruction 1 64 1 64 L2 2M 224M 16 Unified 2 2048 1 64 L3 105M 210M 15 Unified 3 114688 1 64 /proc/cpuinfo cache data cache size : 107520 KB From numactl --hardware WARNING: a numactl 'node' might or might not correspond to a physical chip. available: 2 nodes (0-1) node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 node 0 size: 1031795 MB node 0 free: 845562 MB node 1 cpus: 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 node 1 size: 1032154 MB node 1 free: 950740 MB node distances: node 0 1 0: 10 21 1: 21 10 From /proc/meminfo MemTotal: 2113484880 kB HugePages_Total: 0 Hugepagesize: 2048 kB /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor has performance /usr/bin/lsb_release -d Ubuntu 22.04.2 LTS From /etc/*release* /etc/*version* debian_version: bookworm/sid dgx-release: DGX_NAME="DGX Server" DGX_PRETTY_NAME="NVIDIA DGX Server" DGX_SWBUILD_DATE="2023-05-16-16-18-31" DGX_SWBUILD_VERSION="6.0.11" DGX_COMMIT_ID="d0b730d" DGX_PLATFORM="DGX Server for DGX H100" DGX_SERIAL_NUMBER="1661623000112" os-release: PRETTY_NAME="Ubuntu 22.04.2 LTS" NAME="Ubuntu" VERSION_ID="22.04" VERSION="22.04.2 LTS (Jammy Jellyfish)" VERSION_CODENAME=jammy ID=ubuntu ID_LIKE=debian HOME_URL="https://www.ubuntu.com/" uname -a: Linux VIKING80-DVT-800 5.15.0-1025-nvidia #25-Ubuntu SMP Fri Apr 21 21:46:57 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux Kernel self-reported vulnerability status: CVE-2018-12207 (iTLB Multihit): Not affected CVE-2018-3620 (L1 Terminal Fault): Not affected Microarchitectural Data Sampling: Not affected CVE-2017-5754 (Meltdown): Not affected mmio_stale_data: Not affected retbleed: Not affected CVE-2018-3639 (Speculative Store Bypass): Mitigation: Speculative Store Bypass disabled via prctl and seccomp CVE-2017-5753 (Spectre variant 1): Mitigation: usercopy/swapgs barriers and __user pointer sanitization CVE-2017-5715 (Spectre variant 2): Mitigation: Enhanced IBRS, IBPB: conditional, RSB filling, PBRSB-eIBRS: SW sequence CVE-2020-0543 (Special Register Buffer Data Sampling): Not affected CVE-2019-11135 (TSX Asynchronous Abort): Not affected run-level 3 Oct 19 05:44 SPEC is set to: /var/data0/sandbox/nvuser/SPECACCEL Filesystem Type Size Used Avail Use% Mounted on /dev/md127 ext4 28T 11T 16T 41% /raid From /sys/devices/virtual/dmi/id Vendor: NVIDIA Product: DGXH100 Product Family: DGX Cannot run dmidecode; consider saying (as root) chmod +s /usr/sbin/dmidecode BIOS: BIOS Vendor: NVIDIA BIOS Version: 1.1.1 BIOS Date: 07/31/2023 (End of data from sysinfo program) Compiler Version Notes ---------------------- ============================================================================== C | 457.spC(base) ------------------------------------------------------------------------------ /usr/lib64/crt1.o: In function `_start': (.text+0x24): undefined reference to `main' pgacclnk: child process exit status 1: /usr/bin/ld nvc Rel Dev-r239081 64-bit target on x86-64 Linux -tp icelake-server NVIDIA Compilers and Tools Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. ------------------------------------------------------------------------------ ============================================================================== C | 403.stencil(base) 404.lbm(base) 452.ep(base) 470.bt(base) ------------------------------------------------------------------------------ nvc Rel Dev-r239081 64-bit target on x86-64 Linux -tp icelake-server NVIDIA Compilers and Tools Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. ------------------------------------------------------------------------------ ============================================================================== C | 457.spC(base) ------------------------------------------------------------------------------ /usr/lib64/crt1.o: In function `_start': (.text+0x24): undefined reference to `main' pgacclnk: child process exit status 1: /usr/bin/ld nvc Rel Dev-r239081 64-bit target on x86-64 Linux -tp icelake-server NVIDIA Compilers and Tools Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. ------------------------------------------------------------------------------ ============================================================================== C | 403.stencil(base) 404.lbm(base) 452.ep(base) 470.bt(base) ------------------------------------------------------------------------------ nvc Rel Dev-r239081 64-bit target on x86-64 Linux -tp icelake-server NVIDIA Compilers and Tools Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. ------------------------------------------------------------------------------ ============================================================================== Fortran | 450.md(base) 455.seismic(base) 456.spF(base) 460.ilbdc(base) | 463.swim(base) ------------------------------------------------------------------------------ nvfortran Rel Dev-r239081 64-bit target on x86-64 Linux -tp icelake-server NVIDIA Compilers and Tools Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. ------------------------------------------------------------------------------ ============================================================================== Fortran, C | 453.clvrleaf(base) 459.miniGhost(base) ------------------------------------------------------------------------------ nvfortran Rel Dev-r239081 64-bit target on x86-64 Linux -tp icelake-server NVIDIA Compilers and Tools Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. nvc Rel Dev-r239081 64-bit target on x86-64 Linux -tp icelake-server NVIDIA Compilers and Tools Copyright (c) 2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved. ------------------------------------------------------------------------------ Base Compiler Invocation ------------------------ C benchmarks: nvc Fortran benchmarks: nvfortran Benchmarks using both Fortran and C: nvfortran nvc Base Portability Flags ---------------------- 403.stencil: -DSPEC_NO_NOTHING 457.spC: -mcmodel=medium -Wl,--no-relax Base Optimization Flags ----------------------- C benchmarks: -Ofast -mp=gpu -Mfprelaxed -Mstack_arrays -static-nvidia Fortran benchmarks: -Ofast -mp=gpu -Mfprelaxed -Mstack_arrays -static-nvidia Benchmarks using both Fortran and C: 453.clvrleaf: -Ofast -mp=gpu -Mfprelaxed -Mstack_arrays -static-nvidia 459.miniGhost: -Mnomain -Ofast -mp=gpu -Mfprelaxed -Mstack_arrays -static-nvidia Peak Optimization Flags ----------------------- C benchmarks: 403.stencil: basepeak = yes 404.lbm: basepeak = yes 452.ep: basepeak = yes 457.spC: basepeak = yes 470.bt: basepeak = yes Fortran benchmarks: 450.md: basepeak = yes 455.seismic: basepeak = yes 456.spF: basepeak = yes 460.ilbdc: basepeak = yes 463.swim: basepeak = yes Benchmarks using both Fortran and C: 453.clvrleaf: basepeak = yes 459.miniGhost: basepeak = yes The flags file that was used to format this result can be browsed at http://www.spec.org/accel2023/flags/nv2023_flags_v2.html You can also download the XML flags source by saving the following link: http://www.spec.org/accel2023/flags/nv2023_flags_v2.xml SPECaccel is a registered trademark of the Standard Performance Evaluation Corporation. All other brand and product names appearing in this result are trademarks or registered trademarks of their respective holders. ------------------------------------------------------------------------------------------------------------ For questions about this result, please contact the tester. For other inquiries, please contact info@spec.org. Copyright 2023 Standard Performance Evaluation Corporation Tested with SPECaccel2023 v2.0.17 on 2023-10-19 19:13:09-0400. Report generated on 2023-12-06 13:07:14 by accel2023 ASCII formatter v112. Originally published on 2023-11-08.