Environment Variables

This page lists every SAREK_* and SPOC_* environment variable that the libraries and the PPX read, with the values each one accepts. Check here before relying on a value such as true or 0: the variables do not share one parsing rule, and a value a variable does not recognise is usually ignored without a message.

Test and benchmark programs read further variables of their own (for example SAREK_REQUIRE_PTX, SPOC_BENCH_*); they are not part of the libraries’ interface and are not listed.

Value conventions

So SPOC_DISABLE_GPU=true does nothing.

“Read at” says when the value is taken: PPX is compile time (the environment of the dune build that expands [%kernel ...]), start is when the library or plugin module is loaded at program start, and launch or code generation is later, while the program runs.

Backends and devices

Device selection is explained in Device and backend selection; the Metal batch in Memory and transfers; the Vulkan batch in Launch window and fusion; the fence bound in the Vulkan README.

Optimisation policy: restrict and explain

SPOC applies some optimisations on its own (the launch window, epilogue fusion, the matrix-unit rewrite, the exact-narrowing proof, loop unrolling), and some only after the program has accepted a deviation from the interpreter’s result (a grant, see Matrix units and Launch window and fusion). The environment can only restrict: no variable grants precision, turns an optimisation on that the program or the backend does not allow, or accepts a deviation. Both variables are read once, when the program starts, and a restriction is one-way for the process.

SAREK_OPT_DISABLE

Items are separated by commas, compared case-insensitively, and surrounding spaces and empty items are ignored (Window, ,Unroll works). An unknown item prints one line on stderr naming the item and the valid items, and is ignored; the other items apply.

There is no all. Other variables that used to switch these off are gone (see CHANGES.md): SAREK_LAUNCH_WINDOW, SAREK_LAUNCH_EPILOGUE, SAREK_MATRIX_REWRITE, SAREK_AUTO_UNROLL and SAREK_MATRIX_REPORT are no longer read, and setting them does nothing.

The generated source and the launch memo follow the policy: a restriction cannot make an earlier check stale.

SAREK_EXPLAIN

One line per distinct (kernel, target, optimisation, decision): a decision that comes back after another one is not printed again (a window switched off and on per prompt chunk prints two lines, not one per switch). The memory of what was printed is bounded (4096 distinct decisions, then forgotten). Fields are separated by single spaces; spaces inside a kernel name are written _, and an epilogue pair p + c is written p+c.

The target says where the decision was taken, in one scheme:

sarek-explain <kernel> <target> <optimisation> applied proof=<text>
sarek-explain <kernel> <target> <optimisation> applied granted=<grant>[,<grant>] reason="<reason>" [lowering="<description>"]
sarek-explain <kernel> <target> <optimisation> applied exact
sarek-explain <kernel> <target> <optimisation> refused <reason>
sarek-explain <kernel> <target> coopmat-gate applied|refused ...
sarek-explain <kernel> - policy refused grant-on-generated-name <grant>
sarek-explain <kernel> - policy refused grant-revoked <grant>

<optimisation> is window, epilogue, matrix, narrowing-proof or unroll; <grant> is one of the grant names above. reason="..." holds only what the program gave when it granted (each reason once, in order, joined with ; ). lowering="..." is the matrix rewrite’s own description (lowering, variant, fragment configuration), present on the float rewrites. coopmat-gate is an event kind of its own, not an optimisation: it reports the accuracy gate of the device backends for a kernel the program wrote with Sarek_gemm.Coopmat (the interpreter’s gate is not reported), SAREK_OPT_DISABLE has no name for it, and nothing restricts it. The last two lines say that a grant the program made has no effect: a Kernel grant on a generated name (a fused, derived or static-fusion kernel) is never counted, and a revoked grant is not obeyed; each is printed once, at the next report for that kernel, also when its decisions settled before. A reporter set with Sarek_policy.set_reporter is called outside the policy’s lock, possibly from several domains: lines of different domains may interleave. Sample lines from a run on an AMD RX 7900 XTX (HIP) are in CHANGES.md.

Revoking an accuracy grant has a visible effect. accumulation-order is the closest equivalent of the old SAREK_MATRIX_REWRITE=exact for float paths: it refuses every float matrix-unit path (the rewrite and Sarek_gemm.Coopmat) and keeps int32. An explicit cooperative-matrix kernel the program wrote then fails at launch with a capability refusal that says accumulation-order is revoked by SAREK_OPT_DISABLE. Revoking operand-rounding-bf16 or -bf16x3 does the same for bfloat16 kernels. Revoking one operand rounding can make the rewrite pick another precision the program also granted (for example float16 instead of bfloat16), which is less accurate; revoke all of them to rule that out.

Launch window and fusion

The launch window and epilogue fusion have no variable of their own. SAREK_OPT_DISABLE=window and =epilogue take them away; they are explained in Launch window and fusion.

Matrix units

The rewrite is restricted with SAREK_OPT_DISABLE=matrix or =automatic-float-rewrite and reported with SAREK_EXPLAIN=1. See Matrix units.

Code generation

SAREK_OPT_DISABLE=unroll switches off the automatic unroll hint on small constant-trip for loops in the CUDA, HIP, OpenCL, Metal and GLSL generators. See Kernel language notes.

CPU backends

Compile cache

Vulkan (SPIR-V) and OpenCL (program binaries) keep compiled kernels on disk so that a new process does not compile them again. The files are in the spoc directory under $XDG_CACHE_HOME, or under $HOME/.cache when that is unset; the parent directory must already exist. The cache is used only when that directory is owned by the current user and not writable by group or others (a directory Sarek creates is mode 0700), and never from the shared temporary directory: without XDG_CACHE_HOME and HOME it is off. OpenCL entries are native device binaries that the driver loads, so a directory others can write to would let them substitute code. A refused directory prints one warning and the program compiles as if the cache were off.

An OpenCL entry is keyed by the platform name and version, the device name and version, the driver version, the exact build-option string and the source, so a driver update or a different option set compiles again and writes a new entry. A cached binary that the driver refuses, or that fails to build, is ignored: the kernel is compiled from source and the entry is overwritten. No error reaches the program. A driver that returns no binary for a program (size 0) is never cached. An entry is a checksum followed by the binary; a truncated or damaged file fails the checksum and is treated the same way. Entries are never evicted; delete the spoc directory to clear them. Set SAREK_DEBUG=device to see each lookup ([Cache] lines).

The key cannot see a driver rebuilt from new sources under the same version string: OpenCL uses CL_DRIVER_VERSION, so entries made by the old build are reused. Vulkan keys on the device name and the Vulkan API version, not on the shaderc version, so a new shaderc leaves old SPIR-V in use. After rebuilding or replacing such a component, delete the spoc cache directory or run once with SAREK_DISK_CACHE=0.

Diagnostics

SAREK_DEBUG is read in three places with three meanings; the rows above are the three. The runtime list is described in the sarek.core README.