Hazel reports · ^report(29) · performance · started 2026-10-22 · work in progress · about 10 minutes to read

Where the Gap Comes From

Matthew, on ^report(27)'s Figure 1: seems to say that Hazel and Fumola are less than 3x apart in this situation. How would you attribute the difference in performance? Is it Wasm vs JS -- or more than that? Both use a small-step evaluation system, for a language with purely-functional, higher order data. Let's draft a report WIP --- looking more into the contrast and how it shows up with actual analysis of each system, while running the same programs. Eight small programs, written the same way in both languages, timed, step-counted and profiled (^ticket(545)).

The programs#copy link

Each written the way the other language would, as near as the two allow: recursion over numbers, and lists built from pairs (Hazel's ::, Fumola's ?(x, rest), since Fumola's [...] are arrays). No type annotations in either; Hazel infers and inserts its own. A held list is built once, outside the call, and captured: in Hazel by a closure, in Fumola by a top-level binding. Every call's answer is checked.

let count = fun n -> if n == 0 then 0 else count(n - 1) in
let fib = fun n -> if n < 2 then n else fib(n - 1) + fib(n - 2) in
let build = fun (n, acc) -> if n == 0 then acc else build(n - 1, n :: acc) in
let sum = fun l -> case l | [] => 0 | x :: t => x + sum(t) end in
let map = fun (f, l) -> case l | [] => [] | x :: t => f(x) :: map(f, t) end in
let len = fun l -> case l | [] => 0 | _ :: t => 1 + len(t) end in
let sumbuild = fun n -> sum(build(n, [])) in
let mapinc = fun n -> len(map((fun x -> x + 1), build(n, []))) in
let skip = fun l -> 0 in
let k_id = fun k -> k in
let hold = fun n ->
  let held = build(n, []) in
  (h_id = (fun k -> k),
   h_ref = (fun k -> let u = held in k),
   h_walk = (fun k -> len(held))) in
(count = count, fib = fib, sumbuild = sumbuild, mapinc = mapinc, skip = skip, k_id = k_id, hold = hold)

func count(n) { if (n == 0) { 0 } else { count(n - 1) } };
func fib(n) { if (n < 2) { n } else { fib(n - 1) + fib(n - 2) } };
func build(n, acc) { if (n == 0) { acc } else { build(n - 1, ?(n, acc)) } };
func sum(l) { switch (l) { case null { 0 }; case (?(x, t)) { x + sum(t) } } };
func map(f, l) { switch (l) { case null { null }; case (?(x, t)) { ?(f(x), map(f, t)) } } };
func len(l) { switch (l) { case null { 0 }; case (?(_, t)) { 1 + len(t) } } };
func sumbuild(n) { sum(build(n, null)) };
func mapinc(n) { len(map(func (x) { x + 1 }, build(n, null))) };
func skip(l) { 0 };
func k_id(k) { k };

let held = build(n, null); func h_id(k) { k }; func h_ref(k) { let u = held; k }; func h_walk(k) { len(held) }; 0

Steps, and what each costs#copy link

Fumola counts its steps in every answer. Hazel's count came from a measuring build (trial/r25-stats: the best build, with a counter in the evaluator) that tallies every pass of the evaluator by the kind of step it took, or none when the term was already final. Times are from the best build without the counter, on the laptop.

A Hazel pass is one trip through the evaluator's stages for one term; a step is a pass that rewrote the term (the rest found a value). Fumola's step is one transition of its machine. Hazel's steps for count(1000): 9,007, nine an iteration (a lookup, an application, a comparison, a subtraction, ...); Fumola's: 27,025, twenty-seven an iteration, since its machine writes down every frame it pushes and pops.

Read across a row. Where the passes grow with n and the µs a pass stays put (count, fib, h_walk), the gap is steps times the cost of each: a constant factor, about 12x to 10x. Where the µs a pass climbs with n (sumbuild: 3.9, then 17.6, then 49.3 µs), some pass is doing work that grows with the data. And where the passes themselves grow with a list the program never walks (h_ref: 3,008 passes for one reference to 3,000 numbers), the evaluator is visiting it anyway.

What a Hazel step costs#copy link

Hazel's evaluator is small-step in what it records, but it runs as a recursive descent: evaluate asks for each subterm's value by calling itself, on a trampoline so deep recursion does not overflow JS's stack. Every call goes through the same stack of stages (src/language/dynamics/Evaluator.re):

1. eval_5_state_merge: at the top level, a fresh state, merged back after.
2. eval_4_reuse: has this term's value been kept from a previous run? (the incremental evaluation of live programming)
3. eval_3_record_probe_sample: is this term probed? Open and close a sample.
4. eval_2_until_final: step until the term is final.
5. eval_1_effects: run the step's effects on the state (stack frames, pattern matches for probes), update the reuse map.
6. eval_0_main: the rule itself (Transition.re), with the stages above as its way of evaluating subterms.

The stages are what make Hazel's live features work: probes, the stepper, results kept across edits. They run for every term, including numbers and lists that are already values. Profiles of the calls alone (set-up excluded), on a build that keeps OCaml's names (embed-520's dev build, 2026-10-21; its evaluator is the same shape as the best build's):

• The loop (count): garbage collection is the largest part. Each pass allocates: a trampoline closure per request, a new state record, a fresh term for each result. The rule is 7%.
• The list loop (sumbuild): 46% minting fresh ids, and most of the rest pushing types and collecting the garbage. The next section is why.
• One reference to a held list, from JS (h_ref): all of it in the stages and their trampolines: 3,008 passes, one for the list and one for each number in it, each finding a value and taking no step (why: the next section).

Why lists cost more in Hazel#copy link

A function returned to JS re-evaluates what it holds#copy link

Using a list that a function holds costs a pass for every part of the list, when the function is called from JS: let u = held in k, (held, 1), case held | _ => k end, (fun x -> k)(held) all cost n + 8 passes for a list of n, twice that for two references, and three passes a pair for a list of pairs. It does not matter how the list was made (built in Hazel, passed in from JS, annotated). A reference in a branch not taken costs nothing.

A trace of every pass (a copy of the dev build, patched to log the term each pass visits) shows why. The function's body, as the library calls it, is not let u = held in k but let u = [1, 2, 3, 4] in k: the held list has become a literal in the body. Evaluating a literal list evaluates each element, as it must for [1 + 1, f(x)].

> Closure(Let(...))                 the body, after the call's FunAp
  > Let(...)
    > ListLit[4]                    held, written in as a literal
      > Atom  < Atom Final (no step)   ... once for each element
    < ListLit[4] Final (no step)
  > Closure(Var(k))  < Final

The literal is written in by the library, not the evaluator. Evaluator.evaluate finishes every result with finish = e => e |> Substitution.in_exp(env) |> Exp.replace_all_ids, which turns a closure (a function and its environment) into a closed term by substituting the environment's values into the body. HazelLibCore.to_js then wraps that closed term as a JS function. Inside Hazel nothing is substituted: a closure keeps its environment, and a variable's lookup is one step (Transition.re marks a looked-up list final and returns it as it is). Measured with the counter build, the same held function called 100 times by a Hazel loop against called once from JS:

So this cost is the library's, and the fix is the library's: hand JS the closure as the evaluator left it, environment and all, rather than its finished form; the call then applies the closure as Hazel would. That is the same move as ^ticket(539)'s change for a call's argument (bind it in the environment, do not write it in), made for what comes back. Fumola never has this problem: its values are not syntax, so there is nothing to write back into a body.

A loop re-types its list#copy link

Hazel is gradually typed: it inserts type ascriptions where it infers types, and pushes them through values as it runs. In build(n - 1, n :: acc) the accumulator meets [Int] once an iteration (the counter build shows one Ascription step per iteration), and the push walks the whole list, with a fresh term and a fresh id for each element (src/language/dynamics/transition/Ascriptions.re):

| (ListLit(ds), List(ty)) =>
  let ds' = List.map(d => recur(Asc(d, ty) |> DHExp.fresh), ds);
  unchanged(ds, ds')
    ? Some(e)
    : Some(IdTagged.fast_copy(DHExp.rep_id(e), ListLit(ds') |> DHExp.fresh));

In the profile of sumbuild(1000), 60% of the time is spent inside Ascriptions.re, the ids it mints included. When every element already has the type, unchanged gives back the list as it was, so the result is right and small; but the n wrappers and their n ids were made first and are thrown away. Over a loop of n iterations that is about n²/2 ids. #2681's memo (^report(26)) skips this push for a value it has seen at the top of an annotated call; an accumulator is new at every iteration, so it is never seen.

Two changes would help, neither tried yet: check whether a part needs the push before wrapping it (an element of type Int that is an Int needs nothing), and mint ids only for parts that change.

And a fresh id is dear under js#copy link

Hazel's ids are UUIDs (src/util/Id.re). Minting one is already cheap in OCaml terms (a counter written into 16 bytes, not a random draw), but Uuidm.v4 copies the bytes into a string, and under js_of_ocaml bytes are an array and a string is a JS string, so the copy is a loop over 16 characters building a JS string. In the list loop's profile, 46% of the time is under Id.mk, and three quarters of that is js_of_ocaml's caml_subarray_to_jsbytes and caml_convert_bytes_to_array. This is the one place where the target clearly costs Hazel: natively it would be a 16-byte copy.

What a Fumola step is#copy link

Fumola's machine (crates/fumola_semantics/src/lib/vm_step.rs) is a continuation machine: its state is the current continuation (an expression to evaluate or a value to return), a stack of frames, and an environment, a persistent hash map (im_rc) from names to values. Values are their own type, apart from syntax, so a list held in the environment is a pointer that is never looked into again. A step is one transition: push a frame, pop one, look a name up. Steps are many and small (twenty-seven an iteration of count) and cost about 0.13 µs each in wasm. Fumola's incremental machinery (Adapton's graph) is paid only when a program uses it; Hazel's stages run for every term. do big { ... } runs a region without writing its intermediate states down, about twice as fast again (Adapton/fumola#128, not used here).

One thing found on the way: a deep Fumola answer crashes the wasm instance (memory access out of bounds), and the instance is unusable after (^ticket(549)). Holding a deep value is fine; sending it out is not. It is not the printer, for a list: data answers go out through a structural translation (translate and nested in crates/fumola_wasm/src/lib.rs, calling each other once a nested value, into a serde_json tree), and that recursion runs out of wasm's stack. The printer (format_one_line, used for values with no structural translation, such as a function) recurses too, and runs out sooner. Measured, a fresh instance for each (drivers/deep_answer.mjs):

An option list is two nested values an element (the option and the pair), so the structural limit is the same for both shapes: somewhere between 800 and 1,500 nested values. The kernels end their set-up text with 0 to stay clear of both.

Wasm against native#copy link

On one of GitHub's runners (AMD EPYC 7763 64-Core Processor), Fumola built twice from the same commit (2ec634b) runs every kernel through the same entry point, fumola_eval_top: once compiled to the machine, once to wasm in node. Hazel's best build runs beside them. The mean of 3 rounds, ms per call at the largest size:

Wasm costs Fumola 1.2x to 1.5x on the programs that run many steps (the median of each program's ratio at its largest size: 1.4x). That is all the compilation target does to Fumola, and it is small beside the gap: against native Fumola, Hazel is 12x slower on the loop and 8.1x on calls; against Fumola in wasm, 8.4x and 5.4x. Hazel's own js penalty cannot be measured the same way: its core does not build natively (its utility library depends on a browser UI library, bonsai.web), so what js costs it shows only where a profile names a js-only cost, as with ids above.

One more difference this table shows: a call's floor. A Fumola call that does nothing (h_id) costs 0.325 ms natively and 0.425 ms in wasm, the price of parsing the call's text and writing its answer as JSON, natively too. Hazel's is 0.029 ms: a JS function called with a JS value. For a game that asks a rule once a frame, Fumola's floor is about 2% of a 60 fps frame; Hazel's is nothing.

^report(27)'s Figure 1, read again#copy link

At 64x64 (4,096 cells), the World rule:

• Passed in: Hazel 341 ms, Fumola 148 ms (2.3x). Close, for different reasons. Fumola spends most of its time parsing the map, written out as 281,427 characters of source text, every call (195,144 steps). Hazel spends 80.3 ms bringing the map over from JS (a rule that ignores it, given the same map: ^report(25)'s idU), records and lists of text being dearer to convert than the numbers of skip, and the rest, 261 ms, walking the list, about 64 µs a cell, a few dozen passes each. The finder's annotation costs one look-up a call, not a walk: #2681's memo knows every tail of a list it has checked.
• Held as a list and walked: Hazel 256 ms, Fumola 28.7 ms (8.9x). Steps times their cost, as in h_walk, plus, in Hazel, the held world re-evaluated as a literal at every call, since the rule is a function the library returned to JS.
• Neighbours only: about 1 ms each. The boundary is all there is, and the two boundaries cost about the same for a handful of cells.

So the Figure's under-3x lines are the ones where something other than stepping dominates. Where stepping dominates, the gap is 12x to 10x; where Hazel's per-use walks and re-typing apply, it grows without bound.

First fixes, on trial branches#copy link

Three of the findings above, each fixed on a trial branch of the counter build (trial/r25-stats), built by GitHub, measured on the laptop as before against the counter build itself (so both sides carry the counter). Not PRs yet.

• trial/r25-closures (^ticket(547)): the library hands JS functions unfinished, as closures over their environments (Evaluator.evaluate_unfinished), in run and in every call.
• trial/r25-asc (^ticket(548)): a list push stops at a tail #2681's memo already knows; the memo stops remembering at one; and the evaluator's Asc rule no longer asks a throwaway stepwise push first.
• trial/r25-handles (^ticket(551), on the closures branch): Hazel.value(x) converts once and returns a handle a call takes in x's place.

• ^ticket(547), closures kept whole: from JS, a call that refers to a held list now takes 9 evaluator passes at every size measured (100, 1,000, 3,000), where it took n + 8: one reference to a list of 3,000 is 77x faster. A call from JS now costs fewer passes than the same call from inside Hazel (27, which includes the loop calling it). Walking the list (h_walk) still costs a pass per element, as it should.
• ^ticket(548), list loops: linear now. Building and summing 3,000 numbers, 8.9x faster; three times the list costs three times as much.
• With 547, the World rule's world held as a tree is O(log N) a move: about a millisecond at every size from 8x8 to 64x64 (the table below), the same as passing only the neighbours, and the goal ^ticket(540) set. Building the tree, once a world, is still slow (seconds at 64x64): it passes lists on as it builds, the re-typing 548 fixes; the two are on separate branches and not yet measured together.
• ^ticket(551), handles: the map crosses the boundary in 0.025 ms a call at any size, where converting it costs 49 ms at 64x64; a rule given the handle still walks the list it holds.
• Noise: count, which no fix touches, moved by up to 1.2x either way between runs (another job shared the laptop's two cores during them): read differences under about 1.3x as noise. Pass counts are exact.

The World rule, the world held#copy link

^report(25)'s held list (walked) and ^ticket(540)'s balanced tree (looked up), each built once and held in a closure, ms a move, before and after ^ticket(547):

Fumola: values in as JSON#copy link

Fumola's wasm build took only source text, so a passed map was written out and parsed, then stepped back into a value, every call. Adapton/fumola#148 adds fumola_define_json and fumola_eval_top_with: JSON read straight into Fumola values and bound by name (^ticket(555)). The World rule with the map passed, wasm in node on one of GitHub's runners, three rounds, JSON.stringify counted in, ms a call:

What is left is the rule walking the list of cells; holding the world in a HashMap remains the way to a flat move (about 0.6 ms).

The World rule, the world passed#copy link

The map passed at every call, as JS (converted each time) and as a handle (converted once, Hazel.value), ms a call; ignore is a rule that does not look at it, so the boundary alone; U and A the unannotated and annotated finder:

Second fixes, on trial branches#copy link

Two more, measured on GitHub's runners, never the laptop: four jobs each, the two builds alternating on one runner, so a time is compared only with the other build on the same machine; the median over the jobs. Not PRs yet.

The evaluator's fast path for values#copy link

Every term the evaluator visits goes through five stages before its rule, values included (What next, step 5). trial/r25-fastpath (^ticket(559)) returns a term already final at once, when nothing is probed, nothing reused and nothing published (the library's calls, an app's update; the editor always has statics, so it never takes this path): an atom, a constructor, a function's closure, or a list or tuple of these. The step count goes up by the passes the stages would have made, so it reads the same, and the evaluator's tests pass unchanged (CI green on both trial branches). Passes from the counter build and the fast path on it, exact; times from trial/r25-best and the same fast path on it (trial/r25-fastpath-best):

A third fewer passes on a loop buys about a tenth of its time: the stages are only part of a pass's cost. The held list a function refers to (h_ref), written into its body as a literal by the library's finish, is now skipped whole: 7 passes at every size, without ^ticket(547). The long list loops do not move: their time is the re-typing ^ticket(548) fixes. Noise is the timing build's own spread over its two rounds in a job. Runners: INTEL(R) XEON(R) PLATINUM 8573C v22.23.3; AMD EPYC 9V45 96-Core Processor v22.23.3; AMD EPYC 7763 64-Core Processor v22.23.3; AMD EPYC 7763 64-Core Processor v22.23.3.

The game's own map#copy link

The World rule with the world's squares held in a map, in the game's own rule texts: kids-choice-games' branch hazel-map-553 (^ticket(553)) adds to 📚 Common a balanced search tree of the squares, ordered by row and column (Common.squares, square, set_square), and to 🗺️ World with_squares(m), the same rule looking one square up instead of walking the list. Checked first in every run: on a world of random kinds and ways, its cells shuffled, the map rule and the shipped one agree for every square and all eight directions (392 cases). Library: trial/r25-553, trial/r25-best with 547, 548 and 551 together. ms a move, the map made once:

The map rule is flat, about a millisecond a move at every size: the goal ^ticket(540) set. It needs 547: on trial/r25-best alone the held map is written into the closure and walked at every call, slower than passing the list. Making the map, once a world, is still about 1.5 s at 64x64. Runners: AMD EPYC 9V45 96-Core Processor v22.23.3; AMD EPYC 7763 64-Core Processor v22.23.3; AMD EPYC 9V45 96-Core Processor v22.23.3; AMD EPYC 9V74 80-Core Processor v22.23.3.

Against Fumola's HashMap (^report(27)'s driver, wasm built from 667a34d), on the same runners, alternating, ms a move and the map's set-up:

Both are flat; Hazel's lookup is within about 1.3x to 3.5x of Fumola's. Its set-up is not: about 6.5x at 64x64. Runners: AMD EPYC 9V74 80-Core Processor v22.23.3; AMD EPYC 9V74 80-Core Processor v22.23.3; AMD EPYC 7763 64-Core Processor v22.23.3; AMD EPYC 7763 64-Core Processor v22.23.3.

Where Hazel's set-up goes: Common.squares whole, then each of its parts called alone (the order check, the count, the balanced build), and a call that ignores the cells (the conversion from JS alone), ms; then the whole with the map's helpers unannotated:

Annotations make no difference here. Each part is a walk of the list at Hazel's cost per pass, about 60 to 70 µs a cell even for the bare count: the set-up is the per-pass cost three times over, not the map. Runners: AMD EPYC 9V45 96-Core Processor v22.23.3; INTEL(R) XEON(R) PLATINUM 8573C v22.23.3; INTEL(R) XEON(R) PLATINUM 8573C v22.23.3; AMD EPYC 9V74 80-Core Processor v22.23.3.

What next#copy link

1. Keep closures whole across the library (^ticket(547)): functions returned to JS as the evaluator left them, not finished, on a trial branch stacked with ^ticket(539)'s, and these programs measured again: h_ref should be flat. Then ^ticket(540)'s balanced tree for the World rule can pay off: a held world, O(log N) a move.
2. Push types without copying (^ticket(548)): check before wrapping in Ascriptions.re, and measure sumbuild again: it should become linear.
3. Cheaper ids under js: an id that is a JS string from the start, or an integer, and what that does to the loop's 4%, the list loop's 46%.
4. Fumola's deep answers (^ticket(549)): send answers out without recursion, both the structural JSON and the printer, or stop at a depth with a clear error (an issue for Adapton/fumola, not filed yet).
5. The per-pass cost of the stages when nothing is probed and nothing reused: a fast path for terms that are already values would skip all five.

How this was put together#copy link

Node v22 on the Ubuntu laptop (Intel i7), one process at a time, pinned to two threads and niced. drivers/kernels.mjs runs the programs: Hazel through its JS library (Hazel.run once, then each program a JS function, called with n), Fumola through its wasm build (fumola_eval_top of the call's text, pinned build 2ec634b5df). Three warm-up calls, then up to 60 timed (stopping after 2.5 s and at least 5), the median. Hazel's library: trial/r25-best (all the performance PRs, ^report(25)) for times, trial/r25-stats (the same with a counter, Hazel.stepCounts()) for passes. Profiles: drivers/prof_ref.mjs records only the calls, through node's inspector, on a dev build with OCaml's names and source positions; drivers/profiles.py splits them by source file. GitHub: Adapton/fumola branch bench/545-contrast builds Fumola natively (crates/fumola_wasm/examples/kernels.rs, calling the same fumola_eval_top) and as wasm, and runs both and Hazel's best build, three rounds interleaved. One round on the laptop so far: read single differences under about 10% as noise.

The data#copy link

• The numbers, for programs too: every table in this report, and what its numbers measure, is also in this page as JSON, in its <script type="application/json" id="report-data"> element, so a reader or an agent that fetches the page alone has the dataset. The same, with any raw files that survive, is in 029/data/.