SPEC/OSG Fair Use Rule
Consistency and fairness are guiding principles for SPEC. To assure these
principles are sustained, the following guidelines have been created with
the intent that they serve as specific guidance for any organization (or
individual) who chooses to make public comparisons using SPEC benchmark
results.
When any organization or individual makes public claims using SPEC
benchmark results, SPEC requires that the following guidelines be observed:
|
[1] Reference is made to the SPEC trademark. Such
reference may be included in a notes section with other trademark
references (see http://www.spec.org/spec/trademarks.html
for all SPEC trademarks and service marks).
[2] The SPEC web site (http://www.spec.org) or a suitable sub
page is noted as the source for more information.
[3] If competitive comparisons are made the
following rules apply:
|
a. the results compared must utilize SPEC
metrics and be compliant with that SPEC benchmark's run and
reporting rules,
b. the basis for comparison must be stated,
c. the source of the competitive data must be
stated, and the licensee (tester) must be identified or be
clearly identifiable from the source,
d. the date competitive data was retrieved
must be stated,
e. All data used in comparisons must be
publicly available (from SPEC or elsewhere)
|
[4] Comparisons with or between non-compliant test
results can only be made within academic or research documents where
the deviations from the rules for any non-compliant results have been
disclosed.
[5] For benchmarks that allow estimates, it is acceptable to
compare estimates to rule-compliant results only if the
estimates are clearly identified as estimates. Comparisons,
including competitive comparisons, are allowed as an exception
to rules [3a] and [4] above. Details regarding clear
identification of estimates are provided in the run rules for
benchmarks that allow them.
|
The following paragraph(s) is an example of acceptable language when
publicly using SPEC benchmarks for competitive comparisons:
Example:
SPEC® and the benchmark name
SPECweb® are registered trademarks of the
Standard Performance Evaluation Corporation. Competitive benchmark results
stated above reflect results published on www.spec.org as of Jan 12, 2001.
The comparison presented above is based on the best performing 4-cpu
servers currently shipping by Vendor 1, Vendor 2 and Vendor 3. For the
latest SPECweb99 benchmark results, visit http://www.spec.org/web99.
The following is the list of approved SPEC OSG metrics and submetrics that
may be used for competitive comparisons within the guidelines of the Fair
Use Rule:
CPU2006:
-
The bottom line metrics: SPECint_base2006, SPECint2006, SPECfp_base2006,
SPECfp2006, SPECint_rate_base2006, SPECint_rate2006,
SPECfp_rate_base2006, SPECfp_rate2006
-
Individual benchmark SPECratios
-
Median run times of the individual benchmarks
-
Clearly identified estimates of bottom line metrics may be used.
CPU2000:
-
The bottom line metrics: SPECint_base2000, SPECint2000, SPECfp_base2000,
SPECfp2000, SPECint_rate_base2000, SPECint_rate2000,
SPECfp_rate_base2000, SPECfp_rate2000
-
Individual benchmark SPECratios
-
Median run times of the individual benchmarks
-
Clearly identified estimates of bottom line metrics may be used.
jAppServer2001:
-
metrics: BOPS@Category, Price/BOPS@Category
-
Estimates may not be used.
-
Conversions of the Price/BOPS@Category to other currencies for
purpose of competitive comparison when intended for public view or use is
strictly prohibited.
jAppServer2002:
-
metrics: TOPS@Category, Price/TOPS@Category
-
Estimates may not be used.
-
Conversions of the Price/TOPS@Category to other currencies for
purpose of competitive comparison when intended for public view or use is
strictly prohibited.
jAppServer2004:
-
metric: SPECjAppServer2004 JOPS@Category
-
Estimates may not be used.
JBB2000:
-
metric: SPECjbb2000 only
-
Estimates may not be used.
JBB2005:
-
metrics: SPECjbb2005 bops, SPECjbb2005 bops/JVM
-
Estimates may not be used.
JMS2007:
-
metric: SPECjms2007@Category
-
Estimates may not be used.
JVM2008:
- metrics: SPECjvm2008 Base ops/m and SPECjvm2008 Peak ops/m
- Estimates may not be used.
JVM98:
-
metrics: SPECjvm98 and SPECjvm_base98.
-
Clearly identified estimates of bottom line metrics may be used.
SPECmail2008:
-
metric: SPECmail_MSEnt2008
-
Estimates may not be used.
MAIL2001:
-
metrics: SPECmail2001 and SPECmail2001_users
-
Estimates may not be used.
SPECpower_ssj2008:
-
metrics: SPECpower_ssj2008 overall ssj_ops/watt
-
Estimates may not be used.
SFS2008:
-
metrics: SPECsfs2008_nfs.v3 and corresponding Overall Response Time;
SPECsfs2008_cifs and corresponding Overall Response Time
-
Estimates may not be used.
SFS97_R1:
-
metrics: SPECsfs97_R1.v2 and corresponding Overall Response Time;
SPECsfs97_R1.v3 and corresponding Overall Response Time
-
Estimates may not be used.
WEB99:
-
metric: SPECweb99
-
Estimates may not be used.
WEB99_SSL:
-
metric: SPECweb99_SSL
-
Estimates may not be used.
WEB2005:
-
metric: SPECweb2005
-
Estimates may not be used.
