Hazel performance analysis · ^report(10) · 2026-10-13

What a slider change touches

When a slider moves, which parts of the picture does it actually change, and which does it leave alone? And can Hazel tell the two apart? ^report(9)'s Figure 5 found that evaluating a slide incrementally after a slider edit is no faster than starting over: the edit's dependents are evaluated again whatever they compute. This report measures what is inside them, on the Livelit Playground's slides, with an oracle that names each call the way Incremental Computation with Names (OOPSLA 2015) names allocations in its interpreter: by where it is.

At a glance

The question

Moving a slider changes one literal in the program, a livelit's model, and keeps its id. Hazel's incremental evaluator reuses the cached result of each top-level binding whose code and free variables are unchanged (^report(9)), so after the edit it evaluates again the binding the slider is in and every binding that uses it, and the picture uses it. Inside the picture, nothing is reused: a call made again is a call evaluated again. ^report(9) measured the result: incremental evaluation no faster than starting over.

That says nothing about how much of the picture the slider actually changes. If it changes most of it, there is nothing to save; if it changes a little, an evaluator that could recognize the rest would save most of the work. Two measurements answer it: the picture compared shape by shape, before and after the edit, and every call in the evaluation compared with its counterpart in the run before.

The picture, shape by shape

Each slide was cut at its picture, as ^report(9) cut the woods: the program up to the line that hands the picture to Drawing.^picture, ending with the picture's expression. It was evaluated before and after the edit, and each shape of the resulting list compared with the shape in the same place by a hash of its id-free form (below). A shape is a Polygon or a Path: a name, points, a stroke and a fill.

Every shape the edit did not change is equal, value for value, to the one before, in the same place. The lists have the same length and order, so "the same in place" and "the same anywhere in the list" agree.

An oracle that names calls

A local build of hazelgrove/hazel#2635 (livelits × modular editors) at c4e64060f1, with a module that watches the evaluator. At each call of a function (the step from f(arg) to f's body), it records:

• a name: the caller's name, the call site's id, and how many times that site has been called in that caller. A call made at the top level takes the id of the top-level expression it is in instead of a caller's name. This is the paper's naming for its interpreter, the program position and the loop count, made from the call stack Hazel's probes already keep;
• its inputs: the values of the variables its body reads, the argument among them, and the body itself, each as a hash of an id-free form: the form the views' call memo uses (CallMemo), in which every id is the same and a function is its code and the values of the variables its code reads;
• its result, the same way, and the steps it took, its callees' included.

Two evaluations, before and after an edit, are then compared call by call. A call is reusable by name when the evaluation before had a call of the same name with the same body and the same inputs. Reusing a call skips its callees too, so what a memo could save is the steps of the outermost reusable calls. For comparison, a call is reusable by content when its body and inputs occurred anywhere before, under any name: the call memo's kind of key, with no limit on how many it keeps. Nothing is reused: the oracle only watches, and it cannot see what checking would cost.

The control

The same program evaluated twice, with no edit, should give every call the same name and the same inputs. At first it did not: about 1,200 calls on Scenes, and a dozen or so on Full Body and Witch, looked changed. Walking the two forms of a differing input side by side found an id where they parted: evaluation mints ids for the items of a module value (ModVal), here the Drawing library's, and the id-free form erased ids only in expressions, patterns and rules. Erasing them in module items too (and in types, which made no difference), the control is clean on every slide: Full Body 3,712 calls, every one reusable, Witch 6,639 calls, every one reusable, Scenes 31,613 calls, every one reusable. The views' call memo does not erase them either, so a call there whose inputs hold a module value made afresh would not be recognized; whether that happens in the views is not measured here. No input or result in any run was a value the hash could not take (a function, or a lazy value not yet forced); had one been, it would have counted as changed.

What a memo by name could skip

The whole slide, evaluated from scratch after the edit, and incrementally against the run before it, as the eval worker does. Steps are the evaluator's own steps, counted as they are taken: today's incremental run adds a reused binding's recorded steps to its count without taking them, so they are counted here directly.

• Most of the work is in calls, 88 to 95% of the steps. The rest is the top level, binding after binding, which a memo inside calls does not touch.
• The saving is in the bigger calls. 34 to 53% of the calls take fewer than 40 steps, the call memo's threshold; skipping only the others keeps 94 to 98% of the saving.
• Names by position barely move. After the edit, Full Body: 43 new names and as many gone, of 3,712 calls, and 36 names now calling a different function; Witch: 53 new names and as many gone, of 6,639 calls, and 47 names now calling a different function; Scenes: 10 new names and as many gone, of 31,613 calls, and 9 names now calling a different function. A slider edit changes values, not which calls are made; an edit that inserts an element in a list would shift every later name, which is why the paper derives names from a list cell's own name instead.

What would still run

The steps not in a reusable call, after today's reuse, by the function they are in (its own steps, not its callees'):

• The changed shapes' own work: drawing the mouth (along_lip, lip_y), and moving the points of the shapes that moved (shift_points, a Drawing helper that moves every point of a shape), on the witch her hat and warts among them.
• Calls whose answer did not change. 123 calls on Full Body, 139 on Witch and 126 on Scenes ran with changed inputs and returned what they returned before. The largest is clear_of_eyes, which works out how far the mouth must drop for its top curve to clear the eyes, recursing over 41 points of the lip: a call of 35,879 steps, its callees' included, that comes to the same answer. A memo by name runs it again, since its inputs changed, though most of its callees are skipped (its own steps are in the table above); the callers that read only its answer then find their own inputs unchanged. That is the paper's early cutoff, here for free, because inputs are compared by value.
• The walks over the picture's lists. On Scenes, the largest calls that run again walk the picture's list of shapes to move it (moved, and map under it), and what differs in their inputs is the list. Each call takes the rest of the list, and a rest that holds a changed eye is itself changed, so the walk runs again from the head of the list down to the last changed eye, while the calls on each unchanged shape are skipped. This is the cascade the paper begins from (its Section 2.2), for a list's spine; its remedy is a name for each cell, forked from the cell before.
• The functions that read the slider's value run again, though most of their callees are skipped: on Scenes, woods, which reads it as glow.

In the paper's terms

The oracle is the paper's inner layer with two simplifications. Its names come from position, as in the paper's interpreter, not from the names of the inputs. And it has no graph of reads: a call's dependencies are the values of every variable its body mentions, where the paper records each read with what it saw. A function that mentions the slider's value anywhere in its body depends on it, even on a path that does not read it, which can only cost a reuse. Comparing inputs by value is what makes the early cutoff free here. A memo that keeps the graph would not compare inputs: it would mark dirty what read the slider, and re-check only that.

What next

1. Time a memo by name. Added later on 2026-10-13: ^report(11) suggests which name: the call site and the argument, a call named for its operand. Build it at the call boundary, where the views' call memo hooks in: look a call up by its name, compare its inputs by hash, reuse its result. Checking costs a hash of each call's inputs; with ^report(9)'s Merkle names kept on the values, a value's hash is already there, for 11 to 31% more evaluation time to keep them. Measure real time against today's incremental run and against a run from scratch, on the three slides and in the browser.
2. Replay what a reused call recorded. A top-level binding's cache entry replays its probe samples and tests; a reused call would have to as well, or the probes inside it would go blank.
3. Names for list cells, so that a walk over a list stops at the changed cells instead of running again down its spine: the paper's remedy, and the largest part of what is left on Scenes. Corrected later on 2026-10-13: not the largest part; the table above already shows the walk's own steps are a fraction of what is left. ^report(11) measures lists as named cells on every Scenes slider: they lower the update cost by 0.0 to 1.5 points of a run from scratch.
4. Reads instead of mentions. Record what each call actually reads, so that a function that mentions the slider on a path it does not take is not counted as reading it: the paper's graph of what was demanded.

How this was put together

• Build: #2635 at c4e64060f1, with measuring code on a local branch, not pushed: the oracle (CallNames), its hook at the evaluator's function call, a count of the steps taken, and ^report(9)'s experiments, switched off.
• Programs: the Livelit Playground's Full Body, Witch and Scenes slides, each parsed once (FastParse, MakeTerm) and analyzed with Statics.mk_unmemoized; the edit replaces one integer literal, the slider's model, keeping its id. The whole-slide runs evaluate the slide with Evaluator.evaluate: once before the edit, again unedited (the control), from scratch after the edit, and incrementally after it against the first run's cache. The picture runs evaluate the slide cut at its picture with Evaluator.evaluate_open, before and after.
• Counts: node, the test build. Steps are counted by the oracle as the evaluator takes them; they are deterministic, the same in every run, so one run of each suffices, and the control is the noise floor: zero.
• Drawing: Figure 1 is drawn from the shapes the evaluation produced, exported as JSON: points, strokes and fills as the Drawing library would hand them to SVG.
• The numbers, for programs too: every table in this report, and what its numbers measure, is also in this page as JSON, in its <script type="application/json" id="report-data"> element, so a reader or an agent that fetches the page alone has the dataset. The same, with any raw files that survive, is in 010/data/.