Hazel reports · ^report(18) · 2026-10-19

#2635 has failed CI on every run since the merge, for three known reasons

Cyrus asked why CI fails on #2635, the branch that merges modular editors (#2469) into the livelits branch (#2597). The answer, from all 97 of #2635's CI runs, the failing tests in 10 of them, and both parents' runs over the same eighteen days.

Short answers. As of 2026-10-19 at 09:55 MDT. Every run is linked below, and its numbers are in this page's data.

It is the merge, and it has been since the first run. #2635 has run CI 97 times since it was made on 2026-10-01, and it has never passed: 43 runs failed and 54 were cancelled by a newer push. Every failing run read (10 of the 43, from the first, on 10-01, to the last, on 10-17) fails the same tests: 8 until 10-17, and 10 once the branch took #2469's reorganized tests that day.
Each parent passes on its own, with the same tests. #2597 is green at f8ae6f2b9c (4,522 tests in Fast checks and 4,974 in Full tests). #2469 is green at c349058154 (4,810 and 5,202), which contains the commit #2635 took on 10-17, with the same Test_DefStaticsParity.re. Over these days #2597 had 26 green runs and #2469 33; #2635 had none. Each failing test belongs to one side: the module-analysis tests are #2469's, and the table test is the livelits branch's.
Three causes, all known since the merge. Two were written into its commit messages on 10-01, and all three have been listed as the branch's known failures since 10-10:
- 8 tests: #2469 analyzes a module one member at a time, using dev's module statics. Modules II, which is on the livelits branch only, replaced those statics. So the merge analyzes a module literal whole, and the tests' counts of what is re-analyzed come out low.
- 1 test: dev's canonical completion closes a stray [ inside a table cell. #2469 has it through dev. #2597 does not: its dev is from the morning of 09-22, hours before canonical completion (#2374) landed. So the livelits table test's input no longer makes the hidden error it checks for.
- 1 test: one livelits document elaborates differently per item than as a whole program. Its cause has not been found. Update, 2026-10-19: found and fixed, below.
The fix for the 8 already exists. Andrew's #2687 (the standard library, into #2469) carries modular editors with Modules II, and on 10-18 he rebuilt the per-member analysis on Modules II (00bc805ca1). Its CI is green, with the same DefStaticsParity cases and the same expected counts (5,298 and 5,709 tests). Taking that into #2635 should clear the 8. It may need adapting, because #2687 has Modules II parts 1 and 2 as they are now, and #2635 has parts 1 to 3 as they were on 09-16. That leaves two: the table test needs a new input, and the one mismatch needs finding. The table test will also fail on #2597 itself when it next merges dev, which is 484 commits ahead. Cyrus's plan for merging into dev puts Andrew's modular editing PRs ahead of the livelits and performance PRs, so the rebuilt analysis reaches the livelits branch through dev in any case; taking it into #2635 now would only get there first.

Update, later on 2026-10-19: #2635 is green. With #2699 merged, #2635 passed Fast checks and Full tests on 90ed4505bf and again on 5961401df2, its first green runs since 2026-10-01. What happened to each of the next steps below, and what is still open (#2597's table test, a new conflict in one file, the merge plan for dev), is in ^report(23).

Update, 2026-10-19 at 12:45 MDT: each of the three causes has a fix. The third cause is found, and it was a real bug in incremental statics: a livelit use was not analyzed again when the livelit it applies changed (1131a457e4). The table test has an input whose error stays hidden under canonical completion (27f100e37c). And #2699, a draft into #2635, brings in Andrew's per-member analysis on Modules II, with one fix of its own (223357135a). On GitHub, #2635's Fast checks at 1131a457e4 now fail only the 8 module tests (5,552 tests run), and #2699's pass; #2699 merges into #2635 once its Full tests pass too. Details in the update below.

Every CI run on the four branches, 2026-10-01 to 2026-10-19, one tick each: green passed, red failed, grey cancelled (a newer push replaced it). The dots above #2635 mark the failing runs read, each with its number of failed tests. The circled ticks are the commits that #2635's last run is compared with. #2687 is Andrew's branch that carries the per-member analysis rebuilt on Modules II. Its red runs, from 10-16 to 10-18, failed three other DefStaticsParity cases (moving a member above or below, and mega-1k), and none since. #2469's own red runs come and go within hours (it is under active work; see below). #2635's have never stopped.

The ten tests that fail on #2635's last run, grouped by cause. For the eight module tests, the numbers are the test's expected and received counts of what is re-analyzed after an edit. The table test checks a flag: expected true, received false. The corpus test reports a mismatch at a livelit application.

Red since the first run

#2635 (integration/livelits-modular-editors) was made on 2026-10-01 from the livelits branch, by merging dev (at 9b3f18323a, modular editors' base) and then #2469's own commits. Since then it has taken the livelits branch's later work as cherry-picks, and on 10-17 it took #2469 again, at 19fef7c9bb. Its CI has run 97 times and has never passed. Of the 43 failing runs, these 10 were read in full: the first and the last, about every sixth in between, and the one just before the 10-17 update.

Run started (UTC)	Head	Run	Failed	Tests run	Failing tests (Fast checks)
2026-10-01 16:15	`1a2f0338fb`	36890639241	8	5,281	DefStaticsParity 1, LabelBench 0, ModRoot 7, ModRoot 9, ModRoot 10, ModRoot 11, TypeDeps 0, Editing.TableDecorations 1
2026-10-01 17:26	`42f1ab0be7`	36899430811	8	5,281	DefStaticsParity 1, LabelBench 0, ModRoot 7, ModRoot 9, ModRoot 10, ModRoot 11, TypeDeps 0, Editing.TableDecorations 1
2026-10-03 19:08	`493adc0761`	37146802267	8	5,291	DefStaticsParity 1, LabelBench 0, ModRoot 7, ModRoot 9, ModRoot 10, ModRoot 11, TypeDeps 0, Editing.TableDecorations 1
2026-10-05 18:08	`109d3557c8`	37353713816	8	5,304	DefStaticsParity 1, LabelBench 0, ModRoot 7, ModRoot 9, ModRoot 10, ModRoot 11, TypeDeps 0, Editing.TableDecorations 1
2026-10-10 13:21	`cefa5c2c5c`	38055383868	8	5,326	DefStaticsParity 1, LabelBench 0, ModRoot 7, ModRoot 9, ModRoot 10, ModRoot 11, TypeDeps 0, Editing.TableDecorations 1
2026-10-11 00:23	`602498d7aa`	38098251585	8	5,343	DefStaticsParity 1, LabelBench 0, ModRoot 7, ModRoot 9, ModRoot 10, ModRoot 11, TypeDeps 0, Editing.TableDecorations 1
2026-10-12 07:18	`126f595ce9`	38216998783	8	5,390	DefStaticsParity 1, LabelBench 0, ModRoot 7, ModRoot 9, ModRoot 10, ModRoot 11, TypeDeps 0, Editing.TableDecorations 1
2026-10-14 12:12	`41897ec3a5`	38562046525	8	5,390	DefStaticsParity 1, LabelBench 0, ModRoot 7, ModRoot 9, ModRoot 10, ModRoot 11, TypeDeps 0, Editing.TableDecorations 1
2026-10-17 03:13	`438812718b`	38997939260	8	5,391	DefStaticsParity 1, LabelBench 0, ModRoot 7, ModRoot 9, ModRoot 10, ModRoot 11, TypeDeps 0, Editing.TableDecorations 1
2026-10-17 12:06	`9e7258b55b`	39031252985	10	5,551	DefStaticsParity 1, DefStaticsParity 2, DefStaticsParity 4, DefStaticsParity 33, DefStaticsParity 38, DefStaticsParity 39, DefStaticsParity 40, DefStaticsParity 41, DefStaticsParity 42, Editing.TableDecorations 1

The list is the same on every run until 10-17. On that day, the update brought #2469's reorganized tests. Andrew had moved the module-analysis checks from ModRoot, TypeDeps and LabelBench into DefStaticsParity and added cases of the same kind. So the names changed and the count went from 8 to 10, but the causes did not. The same update also brought four new failures where #2469's code met things only #2635 has. Those were fixed in 9e7258b55b (keys, renames, hinted tests, do-blocks), and the ten in the last run are what is left.

The three causes

1. Per-member module analysis meets Modules II (8 tests)

#2469's statics work per item: a top-level definition is analyzed on its own, and after an edit only the items it touches are analyzed again. Inside a module literal, #2469 goes further and works per member. Its DefStaticsParity tests count the units analyzed after an edit. For example, "item + 1 member re-analyzed" expects 2: the member and the item around it.

That per-member path reads modules through dev's module statics (value_exports, collect_type_exports, module_actual_type, and the labeled-tuple lowering). Modules II, which is on the livelits branch only, replaced those with signatures, sealing and abstract members. To make the merge build, 43089b065d (2026-10-01) has DefStatics analyze a module literal whole, as one item, and says so: per-member analysis "is off until it is rebuilt on Modules II". The trial merge on 10-17 (8c968efcc4) restates it.

So the counts come out low: the whole module is one unit, re-analyzed all at once, where #2469 would re-analyze one member and the item around it. The results are right. DefStaticsParity's other tests, which compare per-item results with whole-program results, pass on the merge. What is lost is granularity, which on a large module is a cost in time.

The rebuild has been done, on another branch. Andrew's #2687 ("Standard library in Hazel source", stdlib into modular-editors) has merged Modules II parts 1 and 2 (#2493 and #2494) since 10-16. In 00bc805ca1 (2026-10-18) he rebuilt the module part of per-item statics on Modules II. Members are lowered against the signature, the module's type is its signature type, and a signature the members do not meet still analyzes the item whole. His message measures "edits in mega-mod-1k wrapped or sealed in a module: ~100 → 7ms". #2687's CI has passed since 10-18, with every DefStaticsParity case named below present, their expected counts unchanged, and the corpus test still in: 5,298 tests in Fast checks and 5,709 in Full tests at 5cf71f2d30. #2635's Modules II is older, parts 1 to 3 as merged into the livelits branch on 09-16, so the port may need adapting to part 3 (abstract type members, sealing, path types). Under Cyrus's merge plan (What next, step 4), Andrew's modular editing PRs go into dev before the livelits PRs, so the livelits branch will take this rebuild from dev when it next merges dev, and adapt it to part 3 then if it has not been done already.

Test (last run)	What it covers	Check	Expected	Received
`DefStaticsParity 1`	the app's path reuses its slot	item + 1 member re-analyzed	2	1
`DefStaticsParity 4`	a module root's items	5 mod items + the exports tail	6	1
`DefStaticsParity 33`	a member retype	step 1 analyzed	4	2
`DefStaticsParity 38`	a module root	step 1 analyzed	2	1
`DefStaticsParity 39`	a recursive type export	step 1 analyzed	2	1
`DefStaticsParity 40`	mega-1k	step 1 analyzed	2	1
`DefStaticsParity 41`	mega-mod-1k	step 1 analyzed	2	1
`DefStaticsParity 42`	mega-mod-1k in one module	step 1 analyzed	3	1

The same checks on the first run (10-01), under their old names:

Test (first run)	What it covers	Check	Expected	Received
`LabelBench 0`	in-situ incremental calc	incr calc: item + 1 member	2	1
`ModRoot 7`	statics incremental	6 items	6	1
`ModRoot 9`	corpus mega-mod-1k	item + 1 member re-analyzed	2	1
`ModRoot 10`	big module (stage D)	3 analyzed at depth 2	3	1
`ModRoot 11`	recursive type export	item + 1 member re-analyzed	2	1
`TypeDeps 0`	module member retype	member-retype: analyzed	4	2

2. A livelits table test meets dev's canonical completion (1 test)

Editing.TableDecorations 1 ("typed table: hidden row error draws no arms, flags the chrome") puts a stray [ in a table cell. Its error then lands in a row the table hides, and the test checks that the table's chrome flags it. Since 2026-09-22 (#2374), dev's canonical completion closes a stray bracket where it stands. So the error lands in the cell, where it can be seen, and the chrome is right not to flag it: expected true, received false. The test needs an input whose error is really hidden. 11fbc4480d (2026-10-01) left it failing on purpose and says why.

#2469 does not cause this: it carries the completion only because it is based on a newer dev. The livelits branch #2597 is still on dev as of the morning of 2026-09-22 (2f9f2f6424, just before #2374), 484 commits behind. It will meet the same failure when it next merges dev.

3. One livelits document elaborates differently per item (1 test)

DefStaticsParity's corpus test ("the slide corpus through calc_auto") elaborates every program in the slide corpus twice, per item and as a whole program, and compares the two. One document differs: docs/livelits/expansion-errors/macro-result.hz, at a livelit application and at the let around it. A local run when the branch was made, on 10-01, found the per-item elaboration reading a + 2 where the whole-program one reads a + 1. The 2 is a literal whose source appears in no program, which suggests something carried over between documents in the corpus run. It has been listed among the branch's known failures since cefa5c2c5c (2026-10-10), and its cause has not been found. Of the ten failures, this is the one that may be a real bug rather than a known gap.

Update, 2026-10-19: the cause is found: after the corpus test's edit, a livelit use kept the expansion of the edited definition. See the update.

Which side each test comes from

Where each failing test's file exists. The module-analysis tests are on #2469's side only, and the table test only on the livelits side, so neither parent's CI runs the other side's tests.

Commit	Module-analysis tests	TableDecorations test
dev	no	no
#2597 (f8ae6f2b9c)	no	yes
#2469 merged (19fef7c9bb)	`Test_DefStaticsParity.re`, `Test_ModRoot.re`	no
#2635 first run (42f1ab0be7)	`Test_DefStaticsParity.re`, `Test_LabelBench.re`, `Test_ModRoot.re`, `Test_TypeDeps.re`	yes
#2635 now (9e7258b55b)	`Test_DefStaticsParity.re`, `Test_ModRoot.re`	yes

Three things that are not the cause

The 10-17 update. It renamed and added tests, from 8 failures to 10 (above). The same three causes were behind both lists.
#2469 being red today. #2469's head as of this morning, a41b292007 (2026-10-19, 08:12 MDT), fails 7 of its own tests in Fast checks (OpenMember 0, OpenMember 1, OpenMember 2, OpenMember 3, OpenMember 4, OpenMember 5, ItemSpans 6; 5,084 tests run), with Full tests still running. This is work in progress on #2469 itself, which goes red and back to green within hours (the timeline). #2635 has not taken it.
#2635's conflict with its base. GitHub shows #2635 as conflicting with integration/livelits-splicerefs-full (#2597's branch) in 25 files. That is how it has been kept up to date, not a failure: of the livelits branch's 110 commits since 10-01, 79 are on #2635 as cherry-picks, and none as a merge. So git sees each change twice. It needs a merge before its next CI run, but that will not change the tests.

Update, 2026-10-19 at 12:45 MDT: a fix for each cause

The corpus document (cause 3): the corpus test analyzes each document, edits it (a 1 becomes a 2), analyzes it again, and then analyzes the original once more. On macro-result.hz that last analysis kept the edited expansion: ^^livelit(^inc(...)) elaborated to a + 2, where a cold analysis and monolithic statics both give a + 1. A use records the livelit it applies as ^inc among the names it depends on, but DefStatics named a livelit's own entry plain inc, so a changed livelit marked nothing stale. 1131a457e4 names it ^inc. The bug was in incremental statics, not in the test.
The table test (cause 2): 27f100e37c gives the row a wrong label (c, not bc): an error in table syntax that the projector hides, whatever completion does. The state from the original crash report, a stray bracket typed into a cell, is still drawn and checked. #2597 still has the old input, which will fail there at its next merge with dev.
The 8 module tests (cause 1): #2699 ports Andrew's per-member analysis on Modules II into #2635: his a6b5478f52, the form from before spans stage 6, which #2635 does not have (00bc805ca1 re-ported it onto stage 6), fitted to #2635's Modules II part 3. That cleared the 8, but its CI then failed the corpus test on another document, color-fig3.hz. A livelit definition, let ^color = { ... }, binds a variable, so the per-member path took it for a module and analyzed it member by member, skipping what Statics does with a livelit definition. 223357135a keeps a livelit definition whole. With #2635's fixes merged in, #2699 passes all 43 DefStaticsParity cases locally, and its Fast checks pass on GitHub.
The conflicts: cf325d4f61 merges the livelits branch into #2635, keeping #2635's side of every conflict, since each of #2597's commits since the merge base was already there. GitHub now shows #2635 as mergeable.
So, of the steps below: 1 is done through #2699, 3 and 5 are done, and 2 is done on #2635 but not yet on #2597.

What next

Bring Andrew's per-member analysis on Modules II (00bc805ca1, on #2687) into #2635, adapted to Modules II part 3. Until then, skip the 8 cases when Modules II is on, each with a reason that links here, so that a red run means something new. Since the analysis reaches the livelits branch through dev anyway (step 4), a port now is a separate, exploratory draft: it finds out early what part 3 needs.
Give Editing.TableDecorations 1 an input whose error stays hidden under canonical completion, on #2597 too, ahead of its next merge with dev.
Find why macro-result.hz elaborates differently per item: run DefStaticsParity's corpus test on that document alone and read its saved output.
Follow Cyrus's plan for merging into dev, broadly: first the PRs that have been waiting (which ones is not settled yet), then Andrew's modular editing PRs (#2469, with #2687 in it), then the livelits and performance PRs, in an order still to be decided. Agree with Andrew that #2687 goes into #2469 before #2469 goes to dev, so that the per-member analysis on Modules II arrives with it.
Merge the livelits branch into #2635 instead of cherry-picking, to clear the 25 conflicting files before its next CI run.

How this was put together

The runs: every CI run on the four branches since 2026-10-01, listed through GitHub's Actions API on 2026-10-19 between 09:40 and 09:55 MDT. A run counts as green, red or cancelled as GitHub gives its conclusion.
The failures: each run read is its Fast checks job's log (and, for #2635's last run, Full tests too). The log gives alcotest's summary (failures, tests run, time) and each failing test's first failed check, with expected and received values. Each run read is linked in the table, and its job id is in the data.
The causes: from the commit messages that recorded them (43089b065d and 11fbc4480d on 2026-10-01, cefa5c2c5c on 2026-10-10, and 8c968efcc4 and 9e7258b55b on 2026-10-17), checked against the logs. Which branch has which test file, and the ancestry facts, come from git (merge-base, cherry, merge-tree). The a + 2 reading is from a local run on 10-01 and was not repeated for this report.
Not done: no tests were run for this report, and no fix was tried. Every number is from CI's own runs.
The data: in this page's <script type="application/json" id="report-data"> element and in 018/data/ (the runs, the runs read with their failures, the parents, the test files, and the git facts). The collector and this page's generator are in 018/drivers/. The job logs themselves are on GitHub, linked from each run.
#2635 is Matthew Hammer's branch, made and kept with Claude. #2469 and #2687 are Andrew's. Modules II is Alexander's (#2493 and the parts stacked on it), merged into the livelits branch.