A loop that has to prove every step.
Our agents work inside a convergence engine we call Mosaicist. It takes a reference kernel, builds a candidate in another kernel language or for another accelerator, and edits that candidate until its runtime and accuracy match the original. This page explains how, and why each rule is there.
You cannot make similarity the goal.
The obvious way to port a kernel is to make the new one look like the old one. It fails. Two kernels can share nearly every instruction and run at different speeds, and a candidate tuned to resemble its reference will resemble it without running like it.
So the roles are fixed in advance. Runtime is the goal. Numerics decide pass or fail. The machine-code comparison has one job: to tell the agent what to change next.
Runtime is the objective; numerics are a gate
A candidate is kept only if it passes the numerics suite and runs faster.
Diff fingerprints, not text
Two compilers never emit comparable text. Both kernels are lifted into a structural fingerprint first.
Converge in dependency order
Coarse structure before fine detail. Instruction order means nothing while the tile shape still differs.
One discrepancy per edit; knobs before rewrites
Every speedup is attributable to one specific change.
Stop at the noise floor
The run ends when the candidate is indistinguishable from the reference within its own measured noise. Anything still different is listed by name.
Capture, propose, verify, converge
One capture of the reference, then many candidates. A failing candidate goes straight back for repair and is never compared or timed.
Capture
The reference is compiled in its own process and everything is kept as a bundle: the virtual instruction listing (PTX), the machine code (SASS), the assembler's resource log, the launch record, per-run device timings and the outputs.
Why a bundle
Bundles are complete, so the comparison tooling still runs against them after the hardware has been released.
What we learned
Grid and block dimensions are not in the code listing. In one port, only the launch record revealed that the reference was persistent and warp-specialized.
Propose
A first, naive translation establishes correctness. After that each edit targets a single ranked discrepancy between the two fingerprints. Parameter changes are tried first because they are cheap and deterministic; structural rewrites come after, one at a time.
The catalog
A construct catalog maps each concept, such as a memory transfer or a pipeline stage, to its expression in both kernel languages, and flags any with no direct analogue.
What we learned
A setting that cannot be built is recorded and stepped over. It should never end the run.
Verify
Outputs are checked against a float64 oracle with guard bands suited to the number format. Only then is the candidate timed, using device time from profiler activity records, not wall clock.
Why device time
These kernels finish well inside a host round trip. A wall-clock loop reports the same floor for all of them and can rank a fused path as faster than one of its own parts.
What we learned
Several changes that looked like improvements did not survive repetition. They are reported as null results.
Converge
A candidate replaces the incumbent only if it is faster by more than the reference's own measured run-to-run noise. A small beam keeps the best few, so a structural step that is briefly slower is not lost.
How it ends
Inside the reference's noise floor, with a report of every edit that was accepted and every one that was not.
What we learned
Candidates that get closer in fingerprint and slower in runtime are logged. They usually reveal a hidden dependency between changes.
Each accepted edit becomes the new incumbent; each rejection is kept as evidence.
Six layers, compared in order
A textual diff of two compilers' output is useless: registers are renumbered, address arithmetic moves, prologues differ. Each kernel is parsed into basic blocks, a control-flow graph and natural loops, normalised, and lifted into six layers.
Counts are taken per loop body and per warp role, because whole-kernel counts mostly measure unrolling. Each layer yields a distance and a list of discrepancy rows. The distance is only a tiebreaker. The rows are what matter, because each one points to an edit.
A caveat we learned the hard way: after one real improvement the overall distance got slightly worse, because the change reordered a loop body. The distance is a search heuristic, not a scoreboard.
Grid, block and cluster shape; warpgroups and their roles; shared memory; registers per thread; occupancy.
Matrix-multiply shape, types and flags; memory-transfer form; barrier operations; access widths.
Loop nest, pipeline depth, warp-role branches, persistent tile loop, the form of the epilogue.
Sequence alignment of the critical operations inside each steady-state loop body.
Rounding and approximation: contraction, approximate exponentials, reciprocal versus divide, conversion modes.
Hot-loop length in real machine code, register spills, stall reasons, tensor-pipe utilisation, memory throughput.
One rule decides what is kept
Everything above reduces to a single predicate. A candidate c replaces the best so far only if it passes the numerics gate and is faster by more than the noise, or is within the noise and structurally closer to the reference.
The noise term is measured from the reference's own repeated runs. It is never assumed.
accept(c) ⇔ numerics_pass(c) and ( t(c) < t(best) − noise or ( |t(c) − t(best)| ≤ noise and D(c) < D(best) ) ) t measured device time noise run-to-run spread of the reference D fingerprint distance, a tiebreaker only
Why you can believe a kernel that an agent wrote
Automated kernel optimization has a public history of results that fell apart because the optimizer learned to satisfy the checker. Our answer is to make the checks independent of the thing being optimized, and to layer them.
An oracle, not a peer
Candidates are compared with a high-precision reference model of the computation, not only with another kernel's output.
Bit-exact where it is possible
For formats whose products are exactly representable, the multiply is required to match exactly. That makes speed comparisons unambiguous.
Race detection before hardware
Warp-specialized pipelines run first in an interpreter that checks the barrier protocol for data races.
Lowering checks anywhere
Every configuration is lowered for the target architecture, with memory budgets validated, on machines with no accelerator at all.
Native tests and fuzzing
On real devices, idle accelerators run a randomized fuzzer over shapes, masks and types, for outputs and gradients.
Null results are results
Leads that looked decisive and were not are written down, with the measurement that ruled them out.
What a kernel boundary costs
Frameworks run a model by calling one library kernel after another. Each is tuned by experts and knows nothing about its neighbours. At every boundary there is a launch, and often an intermediate written to memory only to be read straight back. A megakernel removes the boundaries for one application. The model below is arithmetic, not a benchmark: set the sliders to what you measure on your own stack.
Hover or tap a segment for its value.
Show the numbers as a table
| Component | Kernel library | Megakernel |
|---|
What the model leaves out, on purpose: it holds compute constant, so it shows only what removing boundaries buys. Published launch costs are a few microseconds per kernel; memory traffic is usually the larger term. Real candidates are measured on the device, not predicted.
What this is not
- Not a compiler. It uses compilers. The difference is the search: agents propose, and measurement on the real device corrects them, one application at a time.
- Not tied to one language. The work so far targets Pallas Mosaic GPU and CuTe DSL. The loop only needs a reference, a target and a way to capture both.
- Not a one-off. When the model or the chip changes, the loop runs again. The loop is the product of the work; a kernel is one of its outputs.
What do the agents actually do?
Nearly all of it: they write the kernels, run the experiments, read the machine code, and write the reports, under human direction. The harness supplies what an agent cannot be trusted to supply for itself: the oracle, the timing, the noise floor and the acceptance rule.
What is a reference, and do I need one?
A reference is a kernel that already computes the right thing somewhere: on another accelerator, in another kernel language, or as a plain dense implementation. Convergence needs something to converge on, and correctness needs something to be checked against.
Which hardware and languages?
So far: NVIDIA Hopper and Blackwell, with Pallas Mosaic GPU, CuTe DSL and CUTLASS C++ as source or target, and JAX's TPU kernels as a source. The method is not specific to any of them.
Where are the numbers?
We publish figures when they are ready to be defended in public. The case studies describe the work qualitatively for now.
See it applied.
Two ports on Blackwell, what was verified, and what each one taught us.