SPEC MPI2007 Flag Descriptions for QLogic InfiniPath MPI and the PathScale Compiler

Specify the basic level of optimization desired.
The options can be one of the following:

0    Turn off all optimizations.

1    Turn on local optimizations that can be done quickly. Do peephole optimizations and instruction scheduling.

2    Turn on extensive optimization. This is the default.
The optimizations at this level are generally conservative, in the sense that they are virtually always beneficial and avoid changes which affect such things as floating point accuracy. In addition to the level 1 optimizations, do inner loop unrolling, if-conversion, two passes of instruction scheduling, global register allocation, dead store elimination, instruction scheduling across basic blocks, and partial redundancy elimination.

3    Turn on aggressive optimization.
The optimizations at this level are distinguished from -O2 by their aggressiveness, generally seeking highest-quality generated code even if it requires extensive compile time. They may include optimizations that are generally beneficial but may hurt performance.
This includes but is not limited to turning on the Loop Nest Optimizer, -LNO:opt=1, and setting -OPT:roundoff=1:IEEE_arithmetic=2:Olimit=9000:reorg_common=ON.

s    Specify that code size is to be given priority in tradeoffs with execution time.

Equivalent to -O3 -ipa -OPT:Ofast -fno-math-errno -ffast-math.
Use optimizations selected to maximize performance. Although the optimizations are generally safe, they may affect floating point accuracy due to rearrangement of computations.

NOTE: -Ofast enables -ipa (inter-procedural analysis), which places limitations on how libraries and .o files are built.

-apo <path>
This auto-parallelizing option signals the compiler to automatically convert sequential code into parallel code where it is safe and beneficial to do so.

The default number of threads used at run-time is the number of CPUs available in the machine. This number of threads can also be controlled by setting the OMP_NUM_THREADS environment variable.

-fb_create <path>
Used to specify that an instrumented executable program is to be generated. Such an executable is suitable for producing feedback data files with the specified prefix for use in feedback-directed optimization (FDO). The commonly used prefix is "fbdata".
This is OFF by default.

During the training run, the instrumented executable produces information regarding execution paths and data values, but does not generate information by using hardware performance counters.

-fb_opt <prefix for feedback data files>
Used to specify feedback-directed optimization (FDO) by extracting feedback data from files with the specified prefix, which were previously generated using -fb-create. The commonly used prefix is "fbdata". The same optimization flags should be used for both the -fb-create and fb_opt compile steps. Feedback data files created from executables compiled with different optimization flags may give checksum errors.
FDO is OFF by default.

During the -fb_opt compilation phase, information regarding execution paths and data values are used to improve the information available to the optimizer. FDO enables some optimizations which are only performed when the feedback data file is available. The safety of optimizations performed under FDO is consistent with the level of safety implied by the other optimization flags (outside of fb_create and fb_opt) specified on the compile and link lines.

Disable the use of SSE2/SSE3 instructions. SSE2 cannot be disabled under -m64 and will result in a warning.

Compiler will optimize code for selected cpu type: opteron, athlon, athlon64, athlon64fx, em64t, pentium4, xeon, core, anyx86, auto. auto means to optimize for the platform that the compiler is running on, which the compiler determines by reading /proc/cpuinfo. anyx86 means a generic x86 processor. Under 32-bit ABI, anyx86 is a processor without SSE2/SSE3/3DNow! support; under 64-bit ABI it is a processor with SSE2 but without SSE3/3DNow!. Core refers to the Intel Core Microarchitecture, used by 64-bit CPUs such as Woodcrest. The default is opteron.

(For C++ only) -fexceptions enables exception handling. This is the default. -fno-exceptions disables exception handling.

-ffast-math improves FP speed by relaxing ANSI & IEEE rules. -fno-fast-math tells the compiler to conform to ANSI and IEEE math rules at the expense of speed. -ffast- math implies -OPT:IEEE_arithmetic=2 -fno-math-errno. -fno-fast-math implies -OPT:IEEE_arithmetic=1 -fmath-errno.

Do not set ERRNO after calling math functions that are executed with a single instruction, e.g. sqrt. A program that relies on IEEE exceptions for math error handling may want to use this flag for speed while maintaining IEEE arithmetic compatibility. This is implied by -Ofast. The default is -fmath-errno.

Invoke inter-procedural analysis (IPA). Specifying this option is identical to specifying -IPA or -IPA:. Default settings for the individual IPA suboptions are used.

The Code Generation option group -CG: controls the optimizations and transformations of the instruction-level code generator.

-CG:flow : OFF disables control flow optimization in the code generation. Default is ON.

-CG:gcm : Specifying OFF disables the instruction-level global code motion optimization phase. The default is ON.

-CG:load_exe=N : Specify the threshold for subsuming a memory load operation into the operand of an arithmetic instruction. The value of 0 turns off this subsumption optimization. If N is 1, this subsumption is performed only when the result of the load has only one use. This subsumption is not performed if the number of times the result of the load is used exceeds the value N, a non-negative integer.
If the ABI is 64-bit and the language is Fortran, the default for N is 2, otherwise the default is 1.

-CG:local_fwd_sched : Change the instruction scheduling algorithm to work forward instead of backward for the instructions in each basic block. The default is OFF for 64-bit ABI, and ON for 32-bit ABI.

-CG:movnti=N : Convert ordinary stores to non-temporal stores when writing memory blocks of size larger than N KB. When N is set to 0, this transformation is avoided. The default value is 120 (KB).

-CG:prefetch : Suppress any generation of prefetch instructions in the code generator. The default is ON.

-CG:sse_cse_regs=N : When performing common subexpression elimination during code generation, assume there are N extra SSE registers available over the number provided by the CPU. N can be positive, zero, or negative. The default is positive infinity.

-INLINE:aggressive : Tell the compiler to be more aggressive about inlining. The default is -INLINE:aggressive=OFF.

The inter-procedural analyzer option group -IPA: controls application of inter-procedural analysis and optimization.

-IPA:callee_limit=N : Functions whose size exceeds this limit will never be automatically inlined by the compiler. The default is 500.

-IPA:linear : Controls conversion of a multi-dimensional array to a single dimensional (linear) array that covers the same block of memory. When inlining Fortran subroutines, IPA tries to map formal array parameters to the shape of the actual parameter. In the case that it cannot map the parameter, it linearizes the array reference. By default, IPA will not inline such callsites because they may cause performance problems. The default is OFF.

-IPA:plimit=N : This option stops inlining into a specific subprogram once it reaches size N in the intermediate representation. Default is 2500.

-IPA:pu_reorder=N : Control re-ordering the layout of program units based on their invocation patterns in feedback compilation to minimize instruction cache misses. This option is ignored unless under feedback compilation.

0    Disable procedure reordering. This is the default for non-C++ programs.

1    Reorder based on the frequency in which different procedures are invoked. This is the default for C++ programs.

2    Reorder based on caller-callee relationship.

-IPA:space=N : Inline until a program expansion of N % is reached. For example, -IPA:space=20 limits code expansion due to inlining to approximately 20 %. Default is no limit.

-LANG:copyinout : When an array section is passed as the actual argument in a call, the compiler sometimes copies the array section to a temporary array and passes the temporary array, thus promoting locality in the accesses to the array argu- ment. This optimization is relevant only to Fortran, and this flag controls the aggressiveness of this optimization. The default is ON for -O2 or higher and OFF otherwise.

Specify options and transformations performed on loop nests by the Loop Nest Optimizer (LNO). The -LNO options are enabled only if -O3 is also specified on the pathf95 command line.

-LNO:blocking : Enable or disable the cache blocking transformation. The default is ON.

-LNO:cs2 : This option specifies the secondary cache size to be assumed for cache blocking optimizations. N can be 0 or a positive integer followed by one of the following letters: k, K, m, or M. These letters specify the cache size in Kbytes or Mbytes. Specifying 0 indicates there is no cache at level 2.

-LNO:full_unroll,fu=N : Fully unroll loops with trip_count <= N inside LNO. N can be any integer between 0 and 100. The default value for N is 5. Setting this flag to 0 disables full unrolling of small trip count loops inside LNO.

-LNO:full_unroll_outer=(on|off|0|1) : Control the full unrolling of loops with known trip count that do not contain a loop and are not contained in a loop. The conditions implied by both the full_unroll and the full_unroll_size options must be satisfied for the loop to be fully unrolled. The default is OFF.

-LNO:full_unroll_size=N : Fully unroll loops with unrolled loop size <= N inside LNO. N can be any integer between 0 and 10000. The conditions implied by the full_unroll option must also be satisfied for the loop to be fully unrolled. The default value for N is 2000.

-LNO:fission=N : Perform loop fission. N can be one of the following:
0 = Disable loop fission (default)
1 = Perform normal loop fission as necessary
2 = Specify that fission be tried before fusion

Because -LNO:fusion is on by default, turning on fission without turning off fusion may result in their effects being nullified. Ordinarily, fusion is applied before fission. Specifying -LNO:fission=2 will turn on fission and cause it to be applied before fusion.

-LNO:fusion=N : Perform loop fusion. N can be one of the following:
0 = Loop fusion is off
1 = Perform conservative loop fusion
2 = Perform aggressive loop fusion
The default is 1.

-LNO:ignore_feedback=(on|off|0|1) : If the flag is ON then feedback information from the loop annotations will be ignored in LNO transformations. The default is OFF.

-LNO:interchange=(on|off|0|1) : Disable the loop interchange transformation in the loop nest optimizer. Default is ON.

Enable or disable moving loop-invariant expressions out of loops. The default is ON.

This option controls the LNO optimization level. The options can be one of the following:
0 = Disable nearly all loop nest optimizations.
1 = Perform full loop nest transformations. This is the default.

-LNO:ou_prod_max=N : This option indicates that the product of unrolling of the various outer loops in a given loop nest is not to exceed N, where N is a positive integer. The default is 16.

-LNO:prefetch_ahead=N : Prefetch N cache line(s) ahead. The default is 2.

-LNO:prefetch=(0|1|2|3) : This option specifies the level of prefetching.

0 = Prefetch disabled.

1 = Prefetch is done only for arrays that are always referenced in each iteration of a loop.

2 = Prefetch is done without the above restriction. This is the default.

3 = Most aggressive.

Turn ON or OFF the optimization that replaces an array by a scalar variable. The default is ON.

-LNO:simd=(0|1|2) : This option enables or disables inner loop vectorization.

0 = Turn off the vectorizer.

1 = (Default) Vectorize only if the compiler can determine that there is no undesirable performance impact due to sub-optimal alignment. Vectorize only if vectorization does not introduce accuracy problems with floating-point operations.

2 = Vectorize without any constraints (most aggressive).

-LNO:vintr=(0|1|2) : This flag controls loop vectorization to make use of vector intrinsic routines (Note: a vector intrinsic routine is called once to compute a math intrinsic for the entire vector). -LNO:vintr=1 is the default. -LNO:vintr=0 turns off the vintr optimization. Under -LNO:vintr=2 the compiler will do aggressive optimization for all vector intrinsic rou- tines. Note that -LNO:vintr=2 could be unsafe in that some of these routines could have accuracy problems.

Compile for 32-bit ABI, also known as x86 or IA32.

Compile for 64-bit ABI, also known as AMD64, x86_64, or IA32e. On a 32-bit host, the default is 32-bit ABI. On a 64-bit host, the default is 64-bit ABI if the target platform (-march/-mcpu/-mtune) is 64-bit; otherwise the default is 32-bit.

The -OPT: option group controls miscellaneous optimizations. These options override defaults based on the main optimization level.

-OPT:alias=<name>
Specify the pointer aliasing model to be used. By specifying one or more of the following for <name>, the compiler is able to make assumptions throughout the compilation:

typed
Assume that the code adheres to the ANSI/ISO C standard which states that two pointers of different types cannot point to the same location in memory. This is ON by default when -OPT:Ofast is specified.

restrict
Specify that distinct pointers are assumed to point to distinct, non-overlapping objects. This is OFF by default.

disjoint
Specify that any two pointer expressions are assumed to point to distinct, non-overlapping objects. This is OFF by default.

-OPT:div_split=(ON|OFF)
Enable or disable changing x/y into x*(recip(y)). This is OFF by default, but enabled by -OPT:Ofast or -OPT:IEEE_arithmetic=3. This transformation generates fairly accurate code.

-OPT:fast_complex
Setting fast_complex=ON enables fast calculations for values declared to be of the type complex. When this is set to ON, complex absolute value (norm) and complex division use fast algorithms that overflow for an operand (the divisor, in the case of division) that has an absolute value that is larger than the square root of the largest representable floating-point number. This would also apply to an underflow for a value that is smaller than the square root of the smallest representable floating point number.
OFF is the default.
fast_complex=ON is enabled if -OPT:roundoff=3 is in effect.

-OPT:goto
Disable or enable the conversion of GOTOs into higher-level structures like FOR loops. The default is ON for -O2 or higher.

-OPT:IEEE_arithmetic,IEEE_arith,IEEE_a=(1|2|3)
Specify the level of conformance to IEEE 754 floating pointing roundoff/overflow behavior. The options can be one of the following:

1 Adhere to IEEE accuracy. This is the default when optimization levels -O0, -O1 and -O2 are in effect.

2 May produce inexact result not conforming to IEEE 754. This is the default when -O3 is in effect.

3 All mathematically valid transformations are allowed.

-OPT:malloc_alg=(0|1)
Select an alternate malloc algorithm which may improve speed. The compiler adds setup code in the C/C++/Fortran "main" function to enable the chosen algorithm. The default is 0.

-OPT:Ofast
Use optimizations selected to maximize performance. Although the optimizations are generally safe, they may affect floating point accuracy due to rearrangement of computations. This effectively turns on the following optimizations: -OPT:ro=2:Olimit=0:div_split=ON:alias=typed.

-OPT:Olimit=N
Disable optimization when size of program unit is > N. When N is 0, program unit size is ignored and optimization process will not be disabled due to compile time limit. The default is 0 when -OPT:Ofast is specified, 9000 when -O3 is specified; otherwise the default is 6000.

-OPT:roundoff,ro=(0|1|2|3)
Specify the level of acceptable departure from source language floating-point, round-off, and overflow semantics. The options can be one of the following:

0 = Inhibit optimizations that might affect the floating-point behavior. This is the default when optimization levels -O0, -O1, and -O2 are in effect.

1 = Allow simple transformations that might cause limited round-off or overflow differences. Compounding such transformations could have more extensive effects. This is the default when -O3 is in effect.

2 = Allow more extensive transformations, such as the reordering of reduction loops. This is the default level when -OPT:Ofast is specified.

3 = Enable any mathematically valid transformation.

-OPT:rsqrt=(0|1|2)
This option specifies if the RSQRT machine instruction should be used to calculate reciprocal square root. RSQRT is faster but potentially less accurate than the regular square root operation.
0 means not to use RSQRT.
1 means to use RSQRT followed by instructions to refine the result.
2 means to use RSQRT by itself.
Default is 1 when -OPT:roundoff=2 or greater, else the default is 0.

-OPT:unroll_size=N
Set the ceiling of maximum number of instructions for an unrolled inner loop. If N=0, the ceiling is disregarded. The default is 40.

-OPT:unroll_times_max=N
Unroll inner loops by a maximum of N. The default is 4.

-L -lsmartheap ,
when used as an EXTRA_CLIB or EXTRA_CXXLIB variable, results in linking with MicroQuill's SmartHeap 8 (32-bit) library for Linux. This is a library that optimizes calls to new, delete, malloc and free.

-L -lacml ,
when used as an EXTRA_LIBS variable, results in linking with AMD Core Math Library (64-bit) library, compiled with the 64-bit PathScale Compiler for Linux. By setting "RM_SOURCES= specblas.F90 specbessel.c", the calls to LAPACK and BLAS functions in the rest of 130.socorro are resolved by optimized versions in ACML.

-L -lmkl ,
when used as an EXTRA_LIBS variable, results in linking with Intel Math Kernel Library (em64t) library, for Linux. By setting "RM_SOURCES= specblas.F90 specbessel.c", the calls to LAPACK and BLAS functions in the rest of 130.socorro are resolved by optimized versions in MKL.

-static
Suppress dynamic linking at runtime for shared libraries; use static linking instead.

The -WOPT: Specifies options that affect the global optimizer. The options are enabled at -O2 or above.

-WOPT:aggstr=N
This controls the aggressiveness of the strength reduction optimization performed by the scalar optimizer, in which induction expressions within a loop are replaced by temporaries that are incremented together with the loop variable. When strength reduction is overdone, the additional temporaries increase register pressure, resulting in excessive register spills that decrease performance. The value specified must be a positive integer value, which specifies the maximum number of induction expressions that will be strength-reduced across an index variable increment. When set at 0, strength reduction is only performed for non-trivial induction expressions. The default is 11.

-WOPT:mem_opnds=(ON|OFF)
Makes the scalar optimizer preserve any memory operands of arithmetic operations so as to help bring about subsumption of memory loads into the operands of arithmetic operations. Load subsumption is the combining of an arithmetic instruction and a memory load into one instruction. Default is OFF.

-WOPT:retype_expr=(ON|OFF)
Enables the optimization in the compiler that converts 64-bit address computation to use 32-bit arithmetic as much as possible. Default is OFF.

CFP2006:

If -funderscoring is in effect, and the original Fortran external identifier contained an underscore, -fsecond-underscore appends a second underscore to the one added by -funderscoring. -fno-second-underscore does not append a second underscore. The default is both -funderscoring and -fsecond-underscore, the same defaults as g77 uses. -fno-second-underscore corresponds to the default policies of PGI Fortran and Intel Fortran.

SPEC_MPI_LC_TRAILING_DOUBLE_UNDERSCORE: lowercase the name and append two underscores when defining the spec_rand and spec_srand C functions.

mpicc is the MPI wrapper script for a C compiler. The wrapper script provides all the needed include directories and libraries needed to compile programs for MPI. The -cc= flag is to specify the C compiler command name.

Invoke the PathScale C compiler.
Also used to invoke linker for C programs.

MPI wrapper script for a C++ compiler. The wrapper script provides all the needed include directories and libraries needed to compile programs for MPI. The -CC= flag is to specify the C++ compiler command name.

Invoke the PathScale C++ compiler.
Also used to invoke linker for C++ programs.

MPI wrapper script for a FORTRAN 90/95 compiler. The wrapper script provides all the needed include directories and libraries needed to compile FORTRAN programs for MPI. The -f90= flag is to specify the FORTRAN compiler command name.

Invoke the PathScale Fortran 77, 90 and 95 compilers.
Also used to invoke linker for Fortran programs and for mixed C / Fortran. pathf90 and pathf95 are synonymous.

-IPA:max_jobs=N : This option limits the maximum parallelism when invoking the compiler after IPA to (at most) N compilations running at once. The option can take the following values:

0 = The parallelism chosen is equal to either the number of CPUs, the number of cores, or the number of hyperthreading units in the compiling system, whichever is greatest.

1 = Disable parallelization during compilation (default)

>1 = Specifically set the degree of parallelism

The mpirun script lets you start your parallel MPI program on a set of nodes in a cluster. It starts, monitors, and terminates the node programs. The general syntax is:

$ mpirun [mpirun_options...] program-name [program options]

-m %{hosts} specifies the host file which consists of the list of node names, one per line. Each line may optionally be suffixed by a colon followed by the number of processes to be assigned to that node.

-np $ranks specifies the number of ranks to run.

-disable-mpi-progress-check disables MPI communication progress check.