Hazel reports · ^report(35) · Hazel's evaluator · 2026-10-26 · work in progress · after ^report(29)
Paying Only for What Is Watched
^report(29) found that a Hazel step costs about 15 Fumola steps, and that most of the difference is the machinery Hazel's evaluator runs around each rule for its live features, paid even where no feature is in use. What could be done about it, what would each option cost those features, and which should come first? Ideas for Cyrus and Matthew to weigh.
Short answers.
Where the time goes: on a loop, the rule itself is 7% of a call. The five stages that wrap every pass and their trampolines are 37%, garbage collection 36%. For a reference to a held list it is 66% stages and trampolines, and for a list loop 46% minting fresh ids (before the fix in
^report(29)'s trials). Probes, the stepper and reuse across edits need that machinery. The library's calls and a game's rules use none of them, yet pay it all (Figure 1).A loop's profile in four parts; the rule is the small green one. All three programs in Figure 1. The cheapest option is built and measured: a fast path that returns a term already final without its five stages, when nothing is probed, reused or published (idea 1). A third fewer passes on a loop (14,009 to 9,007), about 8% less time, and a held list of 3,000 referred to 32x faster (3.9 to 0.122 ms). Every evaluator test passes unchanged, and it costs no feature anything.
Before (grey) and after (green) the fast path; ^report(29)'s runs on GitHub, median of four jobs.Next cheapest, not yet tried: when a step is taken, take the next one in place instead of going back through all five stages (idea 2). Today every pass that steps re-enters the whole stack, and on the library's path nothing in the stack can use it. About 30 lines, under the same conditions as idea 1. Its saving is an estimate, at most the stages' and trampolines' share of what the fast path leaves.
The question for Cyrus: beyond these two, the options split. Either the library gets a second, plain driver for the same rules (idea 3: no stages at all, guarded by running every evaluator test through both), or the one driver learns to pay per term only for what watches it (idea 6: helps the editor too, but every live feature must be kept working through the rewrite). Values as a type of their own, as Fumola has them (idea 8), removes the cause, but it is a research-sized change.
Suggested order: land idea 1, try idea 2, then measure what is left before choosing between 3 and 6. Ids and allocation (ideas 4 and 5) are worth a trial each, since garbage collection is the largest single share on a loop.
Where the time goes#
Hazel's evaluator asks for each subterm's value by calling itself, on a trampoline so that deep recursion does not overflow JavaScript's stack. Every call passes through the same stages before the rule for the term is applied (src/language/dynamics/Evaluator.re, read here at baa35eab57):
- eval_5, state merge: at the top level, a fresh state record for the term, appended to its parent's after.
- eval_4, reuse: was this term's value kept from the previous run? On a hit, the kept value and state are spliced in; on a miss at the top level, an entry is made for the next run.
- eval_3, probe sample: is this term probed? If so, open an observation span before and close it, with the value, after.
- eval_2, until final: take one step; if the result is not final, evaluate it again from the top, through all five stages.
- eval_1, effects: apply the step's effects to the state (stack frames, tests, probe data): a new state record each pass, with the step count one higher. Then the reuse map is updated.
- eval_0: the rule itself (
Transition.re), with the stages above as its way of evaluating subterms.
Each stage binds its result with a let.trampoline, so one pass makes about five trampoline closures and continuations, plus the state records. These stages are what make probes, the stepper and reuse across edits work. They run on every pass, including passes over numbers and lists that are already values, and including programs that use none of those features.
^report(29)'s profiles of the calls alone, set-up excluded, on a build that keeps OCaml's names (embed-520's dev build, 2026-10-21, whose evaluator is the same shape as the timing build's). Profiled times run long, so read the shares, not the milliseconds. Under the bars: which of the ideas below aims at each part.| part of the time | count(5000) | sumbuild(1000) | h_ref(3,000) |
|---|---|---|---|
| the evaluator's stages (Evaluator.re, EvaluatorState.re, IncrEval.re) | 25.0% | 5.8% | 43.6% |
| trampolines (Trampoline.re) | 11.7% | 3.5% | 22.4% |
| garbage collection | 36.4% | 15.8% | 13.7% |
| fresh ids (Id.re) | 4.0% | 46.2% | 0.1% |
| pushing types through values (Ascriptions.re) | 0.0% | 9.3% | 0.0% |
| terms and their ids (IdTagged.re, DHExp.re, Exp.re, ...) | 1.2% | 7.3% | 1.5% |
| environments (Environment.re) | 7.2% | 2.3% | 0.2% |
| the step's rule (Transition.re, PatternMatch.re, Unboxing.re) | 7.0% | 3.0% | 6.7% |
| everything else | 7.3% | 6.8% | 11.8% |
| total, profiled | 2,836 ms | 2,508 ms | 1,238 ms |
The list loop's ids and re-typing are a separate story, already fixed on a trial branch (^report(29), #re-type: sumbuild of 3,000 went from 6.1 s to 0.69 s). This report is about the rest: the cost every pass pays for features that are not in use.
The ideas#
For each: what it removes, what is known about the saving (measured, or an estimate from Figure 1's shares, as an upper bound), what it would cost the live features, and how much code it touches. Line counts are from the files at baa35eab57.
| idea | removes | saving | costs the features | touches |
|---|---|---|---|---|
| 1. fast path for values | the five stages, for a term already final | measured: a third of a loop's passes, ~8% of its time; a held list's reference 32x | nothing: taken only when no feature is in use | ~90 lines, Evaluator.re |
| 2. step in place | re-entering the stages after each step | estimate: at most ~37% of a loop (stages and trampolines), likely much less | nothing: the same conditions as 1 | ~30 lines, Evaluator.re |
| 3. a plain driver | all stages and the state record, for the library's calls | estimate: at most ~37% of a loop, plus some of the garbage | nothing in the editor; a second driver to keep in agreement | ~150 new lines; Transition.re shared, unchanged |
| 4. fewer allocations a pass | a state record and ~5 closures each pass | unknown: garbage collection is 36% of a loop | nothing; internal | EvaluatorState.re (300), Evaluator.re (768) |
| 5. cheaper ids | a UUID made a JS string per fresh id | estimate: 4% of a loop; less than 46% of the list loop since the re-typing fix | none, if saved files still read | Id.re, used everywhere; saved files hold ids |
| 6. pay per watched term | the stages' layers, where a term is not watched | estimate: most of 2's and 3's, in the editor too | all features must keep working through a rewrite | the stages, ~350 lines of Evaluator.re |
| 7. remember finals | visiting a value again, by identity | estimate: the editor's share of what 1 saves | none known; memory for the table | ~60 lines, Evaluator.re |
| 8. values as their own type | the cause: values are expressions, so they are visited | estimate: all re-visits, by construction | the stepper shows values as expressions; probes sample them | Transition.re (1,412), Unboxing, PatternMatch, Substitution, the stepper |
1. A fast path for values (built, measured)#
When an evaluation has no probe targets, no previous run to reuse and no outbox to publish to (the library's calls and an app's update; the editor always has statics, so it never takes this path), a term that is already final is returned at once: an atom, a constructor, a function's closure, or a list or tuple of these, followed 16 levels deep. The step count goes up by the passes the stages would have made, so it reads the same. On trial branches trial/r25-fastpath (with pass counters) and trial/r25-fastpath-best (baa35eab57, for times), CI green on both. ^report(29)'s table has every program. The loop gets about 8% faster for a third fewer passes, because a pass that finds a value is cheaper than one that steps. The reference to a held list drops to 7 passes at every size, since the library writes the held list into the function as a literal and the fast path now skips it whole.
2. Step in place (not yet tried)#
In eval_2_until_final, a step whose result is not final evaluates the result through evaluate, which runs the five stages again from eval_5: a fresh state record and merge at the top level, a reuse check, and a probe lookup, each with its trampoline bind. With no targets, no previous run and no outbox, none of those stages can do anything for the stepped term. So under idea 1's conditions, eval_2 could loop on eval_1 directly until the term is final. On a loop nearly every pass left after idea 1 is a step (9,007 of count(1000)'s), so this is where the remaining stage cost sits. The saving is at most the stages' and trampolines' share (37% of a loop), less what each step must still do. It needs a trial before any number is claimed. With probes present it must not apply: a delegating step re-evaluates the same probed term inside its result, and the stages are what notice.
3. A plain driver for the library#
Transition.re's rules are already written once over an evaluation mode (Transition(EvaluatorEVMode)), with the stages supplied as the mode's way of evaluating subterms. A second mode could evaluate subterms directly, keeping only a step count and the tests' results: no state record per pass, no reuse map, no observation trace. The library's run and its calls would use it, and the editor would keep the full driver. The rules stay shared, so they cannot drift apart. The drivers could, so the guard is a differential test: every evaluator test is run through both drivers, and the values and step counts must agree. The saving is again bounded by the stages' and trampolines' share, plus whatever garbage the state records make. Ideas 1 and 2 recover part of it without a second driver, so measure after them.
4. Fewer allocations a pass#
Each pass copies the state record (about ten fields) to raise the step count by one, in EvaluatorState.update, and makes the stages' trampoline closures. Garbage collection is the largest single share of a loop (36%). Holding the step count in a mutable cell for the duration of an evaluation, and fusing the stages' binds where no stage has work, would cut both. This is invisible to the features, but the state record is shared with the stepper and the trace, so the change is spread out. Try it on a trial branch and read the garbage share again.
5. Cheaper ids#
A fresh id is a random 128-bit UUID, and under js_of_ocaml its bytes are made into a JS string. That was 46% of the list loop before the re-typing fix minted fewer, and is 4% of a loop. An id that is an integer, or a string from the start, would cost less. But ids are keys across the whole editor and are saved in files, so the change would be wide. Measure the list loop again with the re-typing fix first: what is left may not be worth it.
6. Pay per watched term#
Ideas 1 and 2 switch the stages off only when no feature is in use anywhere. The editor always has some feature in use, but it watches few terms: a handful of probes, and reuse only at the top level, outside function bodies (the code already says so: Function bodies are not incremental-cache boundaries
). Folding the five stages into one function that checks once whether anything watches this term, and takes the plain way when nothing does, would bring most of ideas 2 and 3 to the editor as well. This is the larger rewrite: every live feature must keep working through it, and the evaluator's probe, stepper and reuse tests are the guard.
7. Remember finals#
Remember by physical identity every term the evaluator has found final (with Util.IdentityTable, as the has_type memo of #2681 does), and return it at once when asked again, before the stages. For the library's path, idea 1 and keeping closures whole (^report(29)'s first fixes) already cover this, so what remains is the editor's path, where idea 1 does not apply. It has not been measured there.
8. Values as their own type#
Fumola's machine keeps values as a type of their own, beside expressions, so a value is never evaluated again: a large part of why its step is cheap (^report(29), #fumola-step). In Hazel a value is an expression that happens to be final, so the evaluator visits it to find that out, and ideas 1, 2 and 7 each avoid one kind of visit. A separate value type would remove the cause. But the rules (1,412 lines), unboxing, pattern matching and substitution all work on expressions, the stepper shows values as expressions, and probes sample them. That is a research direction, to weigh once the cheaper ideas show how much is left.
What next#
- Open idea 1's fast path as a PR (today on trial branches stacked on
^report(29)'s builds), with its tests unchanged and^report(29)'s table as its evidence. - Try idea 2, stepping in place, on a trial branch over the fast path, and time it on GitHub's runners on
^report(29)'s programs (count,fib,h_walk,sumbuild), both builds alternating on each runner. - Profile the loop again on that build: what share is left in the stages, the trampolines and the garbage. That number decides between a plain driver (idea 3) and paying per watched term (idea 6).
- Try idea 4's step counter in a mutable cell, and read the garbage share before and after.
- Time the list loop with the re-typing fix alone, to see whether cheaper ids (idea 5) still matter.
- Ask Cyrus which shape is welcome in the evaluator: a second driver beside the editor's, or one driver that pays per watched term.
How this was put together#
No new measurements. The profiles and the fast path's numbers are ^report(29)'s: its profiles of the calls alone on embed-520's dev build (2026-10-21; 029/data, prof_*.json), and its fast-path table from GitHub's runners, four jobs with the two builds alternating on each runner, median of the jobs (#fix-fast-path). The description of the stages and the line counts come from reading Evaluator.re, EvaluatorState.re and the files named, at baa35eab57 (trial/r25-fastpath-best). Every saving marked as an estimate is an upper bound read off Figure 1's shares, not a measurement: steps are not time, and these need trials before any number is claimed. Figure 1 is drawn by 035/gen035.py from ^report(29)'s files.
The data#
- The numbers, for programs too: every table in this report, and what its numbers measure, is also in this page as JSON, in its
<script type="application/json" id="report-data">element, so a reader or an agent that fetches the page alone has the dataset. The same, with any raw files that survive, is in035/data/.