Pending (sarek-metal.Sarek_metal.Metal

Pending GPU work

Launches are asynchronous: Kernel.execute appends a dispatch to its queue's open command buffer and returns. At each launch the open command buffer is committed if the queue's previous one has completed (the GPU would otherwise idle) or once it holds SPOC_METAL_BATCH dispatches; otherwise it stays open until the next launch, wait, event record or synchronisation commits it. This table remembers every open or committed command buffer that nobody has waited for yet, per command queue, with the MTLBuffers it binds, so that every point where the host can observe or modify device memory waits for exactly the work that touches it:

a host read or write of a buffer (whole or ranged copy, read-back) waits for the pending command buffers that bind that buffer (wait_buffers);
a launch on one queue first waits for pending work on OTHER queues that binds any of its buffers (wait_buffers_on_other_queues). Within one queue Metal orders command buffers itself: buffers are allocated with the default tracked hazard mode, so a command buffer reading what an earlier one on the same queue wrote waits for it on the GPU;
device synchronisation waits for everything (wait_all), stream synchronisation for its queue (wait_queue).

The two waits run before, not atomically with, the copy or launch they guard: two domains that launch on different queues, or copy and launch, on the same buffers at the same moment must order themselves, as on every backend.

Freeing an MTLBuffer needs no wait: a command buffer retains the buffers it binds until it completes, so a release only drops SPOC's reference.

A command buffer that ends in MTLCommandBufferStatusError raises kernel_launch_failed at the first wait that observes it, naming every kernel the command buffer ran: the command buffer, not the dispatch, is the unit of failure, and that wait may be for an unrelated buffer of the same batch. Failures still pending at exit are waited for and reported on stderr (at_exit).

type entry = {

cmd : unit Ctypes.ptr;
(*
Retained by the table until removed.
*)
names : string list;
(*
Kernels it runs (first few), for errors.
*)
buffers : nativeint array;
(*
MTLBuffer handles its dispatches bind.
*)

}

A committed command buffer nobody has waited for yet.

type batch = {

bcmd : unit Ctypes.ptr;
(*
Retained.
*)
encoder : unit Ctypes.ptr;
(*
Retained.
*)
mutable bnames : string list;
mutable bbufs : nativeint list;
mutable count : int;

}

The open (not yet committed) command buffer of a queue, with its compute encoder; launches append dispatches to it.

type queue_state = {

mutable entries : entry list;
(*
Newest first.
*)
mutable open_batch : batch option;

}

val lock : Stdlib.Mutex.t

val queues : (nativeint, queue_state) Stdlib.Hashtbl.t

val key : unit Ctypes.ptr -> nativeint

val batch_limit : int

Most dispatches one command buffer collects before it is committed even though the GPU is still busy (SPOC_METAL_BATCH, default 32).

val prune_threshold : int

val mtl_status_completed : int

val state_locked : nativeint -> queue_state

val add_name : 'a -> 'a list -> 'a list

val commits : int Stdlib.Atomic.t

Command buffers committed so far (for tests: batching keeps it well below the number of launches when the GPU is the bottleneck).

val commit_locked : queue_state -> unit

val open_batch_locked : 
  unit Ctypes_static.ptr ->
  queue_state ->
  name:string ->
  batch

val encode : 
  unit Ctypes.ptr ->
  name:string ->
  buffers:unit Ctypes.ptr list ->
  (unit Ctypes.ptr -> unit) ->
  unit

Append one dispatch to queue q's open command buffer (encode receives the compute encoder). The batch is committed when it holds batch_limit dispatches, or at once when the queue's previous command buffer has completed (the GPU would otherwise sit idle).

val wait_selected : 
  ?commit_all:bool ->
  ?cmd:unit Ctypes.ptr ->
  (nativeint -> nativeint array -> bool) ->
  unit

val touches : 'a list -> 'a array -> bool

val wait_all : unit -> unit

Wait for all pending work on every queue.

val wait_queue : unit Ctypes.ptr -> unit

Wait for all pending work on queue q.

val wait_buffers : unit Ctypes.ptr list -> unit

Wait for pending work that binds any of the MTLBuffers hs.

val wait_cmd : unit Ctypes.ptr -> unit

Wait for the pending command buffer c (an event's), reporting its failure through the table so it is reported once. No-op when c is no longer pending (someone already waited for it and reported it).

val wait_buffers_on_other_queues : 
  unit Ctypes.ptr ->
  unit Ctypes.ptr list ->
  unit

Wait for pending work on queues other than q that binds any of hs.

val last_on_queue : unit Ctypes.ptr -> unit Ctypes.ptr option

Commit queue q's open batch, then return its newest command buffer, retained for the caller (who releases it), or None when the queue has no pending work.

val forget_queue : unit Ctypes.ptr -> unit

Forget queue q (after waiting for it): its handle may be reused.

val queues_with_pending : unit -> int

Number of queues with pending (open or unwaited) command buffers. For tests: a launch that returned before completing leaves one.