SPEC CPU2006/SPEC CPU2017 Platform Settings for HPE Compute Scale-up Server 3200

Operating System Tuning Parameters

OS Tuning

ulimit:

Used to set user limits of system-wide resources. Provides control over resources available to the shell and processes started by it. Some common ulimit commands may include:

• ulimit -s [n | unlimited]: Set the stack size to n kbytes, or unlimited to allow the stack size to grow without limit.
• ulimit -l (number): Set the maximum size that can be locked into memory.

Disabling Linux services:

Certain Linux services may be disabled to minimize tasks that may consume CPU cycles.

irqbalance:

Disabled through "service irqbalance stop". Depending on the workload involved, the irqbalance service reassigns various IRQ's to system CPUs. Though this service might help in some situations, disabling it can also help environments which need to minimize or eliminate latency to more quickly respond to events.

Performance Governors (Linux):

In-kernel CPU frequency governors are pre-configured power schemes for the CPU. The CPUfreq governors use P-states to change frequencies and lower power consumption. The dynamic governors can switch between CPU frequencies, based on CPU utilization to allow for power savings while not sacrificing performance.

Other options beside a generic performance governor can be set, such as the perf-bias:

--perf-bias, -b

On supported Intel processors, this option sets a register which allows the cpupower utility (or other software/firmware) to set a policy that controls the relative importance of performance versus energy savings to the processor. The range of valid numbers is 0-15, where 0 is maximum performance and 15 is maximum energy efficiency.

The processor uses this information in model-specific ways when it must select trade-offs between performance and energy efficiency. This policy hint does not supersede Processor Performance states (P-states) or CPU Idle power states (C-states), but allows software to have influence where it would otherwise be unable to express a preference.

On many Linux systems one can set the perf-bias for all CPUs through the cpupower utility with one of the following commands:

• "cpupower -c all set -b 0"
• "cpupower -c all set --perf-bias 0"
• "cpupower set -b 0"

Tuning Kernel parameters:

The following Linux Kernel parameters were tuned to better optimize performance of some areas of the system:

• dirty_background_ratio: Set through "echo 40 > /proc/sys/vm/dirty_background_ratio". This setting can help Linux disk caching and performance by setting the percentage of system memory that can be filled with dirty pages.
• dirty_ratio: Set through "echo 40 > /proc/sys/vm/dirty_ratio". This setting is the absolute maximum amount of system memory that can be filled with dirty pages before everything must get committed to disk.
• swappiness: The swappiness value can range from 1 to 100. A value of 100 will cause the kernel to swap out inactive processes frequently in favor of file system performance, resulting in large disk cache sizes. A value of 1 tells the kernel to only swap processes to disk if absolutely necessary. This can be set through a command like "echo 1 > /proc/sys/vm/swappiness"
• ksm/sleep_millisecs: Set through "echo 200 > /sys/kernel/mm/ksm/sleep_millisecs". This setting controls how many milliseconds the ksmd (KSM daeomn) should sleep before the next scan.
• khugepaged/scan_sleep_millisecs: Set through "echo 50000 > /sys/kernel/mm/transparent_hugepage/khugepaged/scan_sleep_millisecs". This setting controls how many milliseconds to wait in khugepaged is there is a hugepage allocation failure to throttle the next allocation attempt.
• numa_balancing: Disabled through "echo 0 > /proc/sys/kernel/numa_balancing". This feature will automatically migrate data on demand so memory nodes are aligned to the local CPU that is accessing data. Depending on the workload involved, enabling this can boost the performance if the workload performs well on NUMA hardware. If the workload is statically set to balance between nodes, then this service may not provide a benefit.
• Zone Reclaim Mode: Zone reclaim allows the reclaiming of pages from a zone if the number of free pages falls below a watermark even if other zones still have enough pages available. Reclaiming a page can be more beneficial than taking the performance penalties that are associated with allocating a page on a remote zone, especially for NUMA machines. To tell the kernel to free local node memory rather than grabbing free memory from remote nodes, use a command like "echo 1 > /proc/sys/vm/zone_reclaim_mode"
• max_map_count-n: The maximum number of memory map areas a process may have. Memory map areas are used as a side-effect of calling malloc, directly by mmap and mprotect, and also when loading shared libraries.

tuned-adm:

The tuned-adm tool is a commandline interface for switching between different tuning profiles available to the tuned tuning daeomn available in supported Linux distros. The default configuration file is located in /etc/tuned.conf and the supported profiles can be found in /etc/tune-profiles.

Some profiles that may be available by default include: default, desktop-powersave, server-powersave, laptop-ac-powersave, laptop-battery-powersave, spindown-disk, throughput-performance, latency-performance, enterprise-storage

To set a profile, one can issue the command "tuned-adm profile (profile_name)". Here are details about relevant profiles.

• throughput-performance: Server profile for typical throughput tuning. This profile disables tuned and ktune power saving features, enables sysctl settings that may improve disk and network IO throughphut performance, switches to the deadline scheduler, and sets the CPU governor to performance.
• latency-performance: Server profile for typical latency tuning. This profile disables tuned and ktune power saving features, enables the deadline IO scheduler, and sets the CPU governor to performance.
• enterprise-storage: Server profile to high disk throughput tuning. This profile disables tuned and ktune power saving features, enables the deadline IO scheduler, enables hugepages and disables disk barriers, increases disk readahead values, and sets the CPU governor to performance

Transparent Huge Pages (THP):

THP is an abstraction layer that automates most aspects of creating, managing, and using huge pages. THP is designed to hide much of the complexity in using huge pages from system administrators and developers, as normal huge pages must be assigned at boot time, can be difficult to manage manually, and often require significant changes to code in order to be used effectively. Transparent Hugepages increase the memory page size from 4 kilobytes to 2 megabytes. Transparent Hugepages provide significant performance advantages on systems with highly contended resources and large memory workloads. If memory utilization is too high or memory is badly fragmented which prevents hugepages being allocated, the kernel will assign smaller 4k pages instead. Most recent Linux OS releases have THP enabled by default.

Linux Huge Page settings:

If you need finer control and manually set the Huge Pages you can follow the below steps:

• Create a mount point for the huge pages: "mkdir /mnt/hugepages"
• The huge page file system needs to be mounted when the systems reboots. Add the following to a system boot configuration file before any services are started: "mount -t hugetlbfs nodev /mnt/hugepages"
• Set vm/nr_hugepages=N in your /etc/sysctl.conf file where N is the maximum number of pages the system may allocate.
• Reboot to have the changes take effect.

Note that further information about huge pages may be found in your Linux documentation file: /usr/src/linux/Documentation/vm/hugetlbpage.txt

Firmware / BIOS / Microcode Settings

Firmware Settings

One or more of the following settings may have been set. If so, the "Platform Notes" section of the report will say so; and you can read below to find out more about what these settings mean.

Intel Hyper-Threading (Default = Enabled):

This feature allows enabling or disabling of logical processor cores on processors supporting Intel Hyper-Threading (HT). When Enabled, each physical processor core operates as two logical processor cores. When disabled, each physical core operates as only one logical processor core. Enabling this option can improve overall performance for applications that benefit from a higher processor core count.

Intel Virtualization Technology (Intel VT, VT-x) (Default = Enabled):

When Enabled, a hypervisor or operating system supporting this option can use hardware capabilities provided by Intel VT. Some hypervisors require that you enable Intel VT. You can leave this set to enabled even if you are not using a hypervisor or an operating system that uses this option. With default BIOS settings as shipped with most systems, the default state for this setting is Enabled. However, this setting can change it's default setting depending on the Workload Profile that is selected, or what Workload Profile is default for a certain system.

SR-IOV (Default = Enabled):

If Enabled, SR-IOV support enables a hypervisor to create virtual instances of PCI-express device, potentially increasing performance. If enabled, the BIOS allocates additional resources to PCI-express devices. You can leave this option set to Enabled even if you are not using a hypervisor. With default BIOS settings as shipped with most systems, the default state for this setting is Enabled. However, this setting can change it's default setting depending on the Workload Profile that is selected, or what Workload Profile is default for a certain system.

Last Level Cache (LLC) Dead Line Allocation (Default = Enabled):

In the Xeon Scalable processor cache scheme, mid-level cache (MLC) evictions are filled into the last level cache (LLC). If a line is evicted from the MLC to the LLC, the core can flag the evicted MLC lines as "dead". This means that the lines are not likely to be read again. This option allows dead lines to be dropped and never fill the LLC if the option is disabled. Values for this BIOS option can be:

• Disabled: Disabling this option can save space in the LLC by never filling dead lines into the LLC. This can prevent useful data from being evicted.
• Enabled: Opportunistically fill dead lines in LLC, if space is available.

Enhanced Processor Performance Profile (Default = Disabled):

Use this option to select a pre-defined enhanced processor performance profiles. Based upon the selection, this feature will adjust the processor settings for improved performance, but may result in higher power consumption. Values for this BIOS option can be:

• Disabled: No adjustments are made to the system for performance with no change in power consumption.
• Conservative: This profile will apply minimal adjustments, but still allow for some improved performance and higher power consumption.
• Moderate: This profile will apply more adjustments than Conservative profile for improving performance, but may result in higher power consumption than Conservative profile.
• Aggressive: The aggressive profile can provide the maximum improvement in performance, but may result in higher power consumption as compared to Moderate and Conservative profiles.

Last Level Cache (LLC) Prefetch (Default = Disabled):

This option configures the processor Last Level Cache (LLC) prefetch feature as a result of the non-inclusive cache architecture. The LLC prefetcher exists on top of other prefetchers that that can prefetch data in the core data cache unit (DCU) and mid-level cache(MLC). In some cases, setting this option to disabled can improve performance. Typically, setting this option to Enabled provides better performance. Values for this BIOS option can be:

• Disabled: Disabling this option can forces data to fill the MLC before prefetching data to the LLC.
• Enabled: Giving the core prefetcher the ability to prefetch data directly to the LLC without filling the MLC.

NUMA Group Size Optimization (Default = Clustered):

This feature allows the user to configure how the BIOS reports the size of a NUMA node (number of logical processors), which assists the Operating System in grouping processors for application use (referred to as Kgroups). Values for this BIOS option can be:

• Clustered: Might provide better performance for some workloads due to optimizing the resulting groups along NUMA boundaries.
• Flat: Might provide better performance for some workloads that cannot take advantage of processors spanning multiple groups. This setting would be necessary to help this class of applications utilize more logical processors.

DCU Stream Prefetcher (Default = Enabled):

In most environments, leave the option enabled for optimal performance. With certain workloads, disabling it might provide a performance benefit. Do so only after performing application benchmarking to verify improved performance in a particular environment. Values for this BIOS option can be:

• Enabled: Once Enabled, the DCU streamer prefetcher will detect multiple loads from the same cache line within a time limit and prefetches data into L1 cache.
• Disabled: Disabling the DCU streamer prefetcher will not fetch data from L2 or main memory into L1 cache to speed up data access.

Uncore Frequency Scaling (Default = Auto):

This option controls the frequency scaling of the processor`s internal buses (the uncore). Values for this BIOS option can be:

• Auto: Enabled the processor to dynamically change frequencies based on the workload.
• Maximum: Enables tuning for latency.
• Minimum: Enables tuning for power consumption.

Workload Profile (Default = Mission Critical):

This option allows a user to choose one workload profile that best fits the user`s needs. The workload profiles control many power and performance settings that are relevant to general workload areas. Values for this BIOS option can be:

• Mission Critical, In Memory Database, Virtualization, Custom.
• Setting the Workload Profile to any option not named Custom allows the server to automatically configure various BIOS settings. These BIOS settings control many power and performance settings that are relevant to general workload areas that fit the profile name.
• Setting the Workload Profile to Custom allows a user to set any BIOS setting to any supported setting. Choosing Custom after selecting an initial profile does not change the settings controlled by the profile previously selected without user intervention.

Power Regulator (Default = OS Control Mode):

This option can only be configured if the Workload Profile is set to Custom. This feature allows the user to select the following Power Regulator support:

• Dynamic Power Savings Mode: This mode allows the automatic variation of the processor speed and power usage based on processor utilization, resulting a reduction in overall power consumption with little or no impact on performance. It does not require OS support.
• OS Control without Legacy Support Mode
• Static High Performance Mode: This mode allows the processors to run in their maximum power/performance state at all times, regardless of the OS power management policy.
• OS Control Mode: This mode allows the processors to run in their maximum power/performance state at all times unless the OS enables a power management policy.

Minimum Processor Idle Power Core C-State (Default = C6 State):

This option can only be configured if the Workload Profile is set to Custom, or this option is not a dependent value for the Workload Profile. This feature selects the processor's lowest idle power state (C-state) that the operating system uses. The higher the C-state, the lower the power usage of that idle state (C6 is the lowest power idle state supported by the processor). Values for this setting can be:

• C6 State: While in C6, the core PLLs are turned off, the core caches are flushed and the core state is saved to the Last Level Cache. Power Gates are used to reduce power consumption to close to zero. C6 is considered for an inactive core.
• C1E State: C1E is defined as the enhanced halt state. While in C1E no instructions are being executed. C1E is considered for an active core.
• No C-states: No C-states is defined as C0, which is defined as the active state. While in C0, instructions are being executed by the core.

Energy/Performance Bias (Default = Balanced Performance):

This option can only be configured if the Workload Profile is set to Custom, or this option is not a dependent value for the Workload Profile. This option configures several processor subsystems to optimize the processor's performance and power usage. Values for this BIOS setting can be:

• Balanced Performance: Provides optimum performance efficiency and is recommended for most environments.
• Maximum Performance: Should be used for environments that require the highest performance and lowest latency but are not sensitive to power consumption.
• Balanced Power: Similar to Balanced Performance but this option prioritizes more power savings at the sacrifice of performance.
• Power Savings Mode: Should only be used in environments that are power sensitive and are willing to accept reduced performance.

Energy Efficient Turbo (Default = Enabled):

This option controls whether the processor uses an energy efficiency based policy when engaging turbo range frequencies. This option is only applicable when Turbo Mode is enabled. Values for this BIOS setting can be: Enabled or Disabled.

Memory Patrol Scrubbing (Default = Enabled):

This option allows for correction of soft memory errors. Over the length of system runtime, the risk of producing multi-bit and uncorrected errors is reduced with this option. Values for this BIOS setting can be:

• Enabled: Correction of soft memory errors can occur during runtime.
• Disabled: Soft memory error correction is turned off during runtime.

HW Prefetcher (Default = Enabled):

Use this option to disable the processor HW Prefetch feature. In some cases, setting this option to disabled can improve performance. Typically, setting this option to enabled provides better performance. Only disable this option after performing application benchmarking to verify improved performance in the environment. The HW Prefetcher fetches streams of data and instruction from the memory into the second-level (L2) cache if it determines this data is likely to be required in the near future. The prefetcher is capable of handling multiple streams in either the forward or backward direction. The HW Prefetcher is triggered when successive cache misses occur in the last-level cache and a stride in the access pattern is detected, such as in the case of loop iterations that access array elements. The prefetching occurs up to a page boundary. This option can reduce the latency associated with memory reads. Values for this BIOS setting can be Enabled or disabled.

Adjacent Sector Prefetch (Default = Enabled):

Use this option to disable the processor Adjacent Sector Prefetch feature. In some cases, setting this option to disabled can improve performance. Typically, setting this option to Enabled provides better performance. Only disable this option after performing application benchmarking to verify improved performance in the environment. The Adjacent Sector Prefetch retrieves both sectors of a cache line when it requires data that isn't currently in the cache. When disabled, the processor will only fetch the sector of the cache line that includes the requested data. Values for this BIOS setting can be Enabled or disabled.

Advanced Memory Protection (Default = Adaptive Double Device Data correction):

Use this option to configure additional memory protection with ECC (Error Checking and Correcting). Options and support vary per system and configuration. Values for this BIOS setting can be:

• Advanced ECC Support: Keeps all installed memory available for use while still protecting the system against all single-bit failures and certain multi-bit failures.
• Adaptive Double Device Data correction: Enables the system to correct memory errors and continue to operate in cases of multiple DRAM device failures on a DIMM. This provides protection against uncorrectable memory errors beyond what is available with Advanced ECC.

Last modified Jan 9, 2024.