Introduction

The SPEC CPU2000 benchmarks are intended to exercise the CPU itself, the memory hierarcy, and the compilers. How much memory do they actually use?

The data collected here show that SPEC met its goals for memory footprint: most benchmarks are larger than common cache sizes, many are larger than 100MB, and none are larger than 200MB.

It is useful to have benchmarks that are larger than common caches, because SPEC would like to differentiate its benchmarks from "toy benchmarks" that are too easy to run or that simply reflect MHz.
It is useful to keep the benchmarks under 200MB so that the suite leaves a reasonable margin on a 256MB machine. The other 56MB are available for the operating system, graphics system, network daemons, etc, without using 'single user mode' on Unix systems, or killing processes on NT systems. (Such measures may not be representative of how most people use their systems.)

The SPEC CPU2000 benchmarks are derived from real applications, and they exercise more of the system than just the CPU chip.

Testbed and methods

Benchmark sizes were measured using:

The hardware and software from the submission of the Compaq AlphaServer 4100 5/533 - see http://www.spec.org/cpu2000/results/res1999q4/cpu2000-19991130-00015.asc http://www.spec.org/cpu2000/results/res1999q4/cpu2000-19991130-00014.asc
SPEC CPU2000 Kit 92 (the kit used in the submission)
The tuning from the submission, except that -non_shared was added. This was added because a related set of experiments needed non_shared linking in order to include library effects (e.g. libc and libm). It is likely that the actual submission images consumed about the same physical memory as the images reported here; and slightly MORE virtual memory, since they mapped all of libc and libm, not just the portions actually needed by the benchmarks.

Resident size (rsz) and virtual size (vsz) were collected using
ps -o rsz,vsz,command
in a loop with a sleep of 5 seconds between each observation.

Results in table form

In the table that follows, the column num obs indicates the total number of observations, so that number times 5 should roughly equal the times reported at www.spec.org (see above for the exact URLs). The column num unchanged indicates the number of observations that reported the same value prior to the conclusion of the benchmark. Thus a zero in that column indicates that the last observation was different from the second last. The final column contains the word "stable" if the number unchanged is at least 90% of the number of observations; it is intended to indicate a benchmark that grows quickly to its target size and then stays there.

All observations on this web page are expressed in megabytes.

            max     max     num  num
            rsz     vsz     obs  unchanged  stable?
            -----   -----   ---  ---------  -------
gzip        180.0   199.0   181      68
vpr          50.0    53.6   151       6
gcc         154.0   156.0   134       0
mcf         190.0   190.0   232     230     stable
crafty        2.0     2.6   107     106     stable
parser       37.0    66.8   263     254     stable
eon           0.6     1.5   130       0
perlbmk     146.0   158.0   186       0
gap         192.0   194.0   149     148     stable
vortex       72.0    79.4   162       0
bzip2       185.0   199.0   153       6
twolf         3.4     4.0   273       0

wupwise     176.0   177.0   185     181     stable
swim        191.0   192.0   322     320     stable
mgrid        56.0    56.7   281     279     stable
applu       181.0   191.0   371     369     stable
mesa          9.4    23.1   132     131     stable
galgel       63.0   155.0   287      59
art           3.7     4.3   157      37
equake       49.0    49.4   218     216     stable
facerec      16.0    18.5   182     173     stable
ammp         26.0    28.4   277     269     stable
lucas       142.0   143.0   181     179     stable
fma3d       103.0   105.0   268     249     stable
sixtrack     26.0    59.8   148     141     stable
apsi        191.0   192.0   271     270     stable

Results in graphic form

The following benchmarks grow quickly to their target sizes (expressed in megabytes) and then stay there:

ammp, applu, apsi, crafty, equake, facerec, fma3d, gap, lucas, mcf, mesa, mgrid, parser, sixtrack, swim, wupwise

The following benchmarks change size over time:

The following benchmarks are shown rescaled:

This page was last updated 29-Aug-2001 3:04 PM.

The results on this page were generated in late 1999 and early 2000 and have been previously presented at SPEC and ACM seminars. At that time, the author worked for Compaq Computer Corporation, whose permission is gratefully acknowledged for the re-publication at the SPEC web site.

The author can be reached at j.henning@computer.org.