SPECopc Overview

SPECopc Project Rules

Q. What is SPECglperf™?

A. SPECglperf is an executable toolkit that measures the performance of OpenGL 2D and 3D graphics operations. These operations are low-level primitives (points, lines, triangles, pixels, etc.) rather than entire models such as those used in the SPECviewperf benchmark. Because of the time commitment needed to conduct SPECglperf testing and the complexity of presenting the results, the SPECopc project group decided in summer 2000 to discontinue reporting for this benchmark on the web site. SPECglperf will continue to be offered, however, as a free downloadable toolkit that is valuable for internal performance evaluation for vendors and users of graphics systems.

Q. What is the purpose of SPECglperf?

A. The purpose of SPECglperf is two-fold: to standardize the method by which vendors measure OpenGL primitive performance on their systems, and to use interactive graphs and other data analysis tools to present a much fuller performance picture than that provided by isolated primitive numbers.

Q. Who has developed SPECglperf and who will maintain it?

A. SPECglperf was developed by the OpenGL Performance Characterization (SPECopc^SM) project group, a graphics benchmarking organization that operates under the auspices of the Graphics Performance Characterization (SPEC/GPC) Group.

Q. What is a SPECglperf script?

A. A SPECglperf script describes the graphics primitives that will be included in performance tests. One script tests performance, for example, based on a 50-pixel triangle strip. A script allows many disparate primitives to be tested during one SPECglperf run and allows easy changes and comparisons between different tests.

Q. What scripts are used within the SPECglperf toolkit?

A. There are 13 SPECglperf scripts, split into 10 RGB scripts and three color index scripts. The scripts are further divided by functionality: the OPClist scripts (RGB and color index) contain a number of tests for a variety of graphics primitives and other operations (such as window-clears). These tests are probably the closest parallel to primitive-level results available from most vendors today. Other scripts feature specific graphics operations, such as CopyPixl.rgb, DrawPixl.rgb, ReadPixl.rgb, TexImage. rgb measure glCopyPixels, glDrawPixels, glReadPixels and glTexImage2D RGB operations. DrawPixl.ndx and ReadPixl.ndx are the color index analogs to DrawPixl.rgb and ReadPixl.rgb.

Remaining scripts address underlying graphics concepts that affect OpenGL performance -- BgnEnd.rgb measures performance as it varies with the number of primitives batched together (in a glBegin/glEnd pair), FillRate.rgb measures how fast rasterization operations are performed (how many pixels are drawn per second), Light.rgb measures the effect of the number of enabled light sources on drawing a particular primitive, and LineFill.rgb and TriFill.rgb measure the effect of increasing primitive size on the drawing rates of lines and triangles, respectively.

Q. Why are there so many numbers, instead of just one?

A. One number is certainly preferable for marketing purposes. It does not serve justice, however, to boil down meaningful results into one meaningless number. Take the analogy of buying a car: one must compare many aspects of each car to arrive at the best buying decision. Most car shoppers do not take totally unrelated metrics and composite them for comparison purposes. They don't, for example, take an average of the acceleration, top speed, trunk space, fuel efficiency, leg room, gas tank capacity and inverted braking distance. So why would one insist on doing so for something as complex as 3D graphics performance? Rather, one should choose the metrics most related to one's needs and compare using those figures. A CAD user, for example, might not care particularly how fast a system can transfer an image from memory to the screen, but this is vitally important to others, such as those in the film and video industry.

Q. What is the difference between SPECviewperf™ and SPECglperf?

A. Both SPECviewperf and SPECglperf measure the graphics performance of a system through the OpenGL API. They were designed, however, with different goals in mind. While SPECviewperf draws an entire model with differing sizes of primitives (as you would see in an actual application), SPECglperf artificially assigns a specific size to every primitive drawn within a test. While SPECviewperf attempts to emulate what an application would do graphically and measure it, SPECglperf makes no such attempt. Instead, SPECglperf provides a more controlled environment within which to extract and measure the highest performance or "upper bound" of a particular system.

Another difference is that SPECviewperf reports results in frames drawn per second, whereas SPECglperf measures its results in primitives drawn per second, whether the primitive is pixels, points, lines, triangles or some other object. To give an analogy to the automotive world, SPECglperf would be the equivalent of a speedometer measuring top speed, while SPECviewperf would be a stopwatch measuring the average speed through a slalom course.

Q. How do I look at all these numbers and make sense of them?

A. A substantial amount of work went into making the visualization and analysis of SPECglperf data as simple as possible. This goal is realized in the form of Java applets that read the SPECglperf performance results from the server's database, and allow tabular and graphical comparisons on the client's web browser. At the top level, the SPECglperf visualization tools are broken into two branches: the OPClist and the other scripts. Going down the OPClist branch, one can construct a performance comparison by selecting the desired primitives from one list, and the desired systems on another list. The other branch allows users to choose from the different SPECglperf scripts to display the data that most interests them. Each choice enables a table to be displayed with graphs on one axis and system configurations on the other. By selecting different cells, different sets of data will be displayed. Also, the axes can be linear or a log scale, allowing the most flexible plotting of data.

Q. What systems are capable of running SPECglperf?

A. Those systems supporting OpenGL drivers on OS/2, Windows 95/98, Windows NT and UNIX derivatives.
 
 

This document was prepared by John Spitzer, Jspitzer@nvidia.com, former chair of the SPECopc  project group.

If your browser refuses to load the following links, use [Shift] [left click] to save the file to disk.Your zip decoder may offer "Install" as an option. Use it to run the install program.

• InstallShield for Windows, Intel Processor

By downloading the following, you acknowledge you have read, understand, and agree to abide by the terms of the License Agreement.

• .tgz for UNIX / Linux (compile it yourself). Also requires GLscrip8.tgz
• Precompiled excutable for Linux (Red Hat, Mandrake, similar, Intel Processor architecture; see notes in glperf.tgz archive above)