← All writing← 全部文章 Essay · 2026-11-08 · 27 min read

I Didn’t Write the Code. I Wrote the Harness.

My DCF model has 52,148 formulas. I wrote 951 of them. Once you see that, a spreadsheet is a program, reading it is a context problem, and building the thing that reads it is a harness problem.

这篇暂时只有英文版。

My senior-living model has 52,148 formulas. I wrote 951 of them.

The other 98 percent are copies. The January formula, dragged right to the last month of the hold period, dragged down to the last unit type. Same logic, different address. Every analyst knows this in their hands. Once you say it out loud, a 52,000-formula workbook stops being a spreadsheet and becomes a 951-line program with a layout table attached.

That sentence took me a year to earn.

A dense spreadsheet grid dissolving into a short column of logic lines and a small layout table


What a Model Is

I have spent twelve years in commercial real-estate investment. Underwriting runs on Excel, and the AI assistants inside it are good at one thing: the workbook you have open. They read your screen and tell you what a cell does. For one model at a time, that is fine.

Real underwriting is never one model at a time. It is one deal in three versions, side by side. One deal, ten properties. Ten properties rolled up into a fund. Thirty deals against the appraiser’s model. Every one of those is tens of thousands of formulas, and the assistant reads one.

Every time you ask the assistant a question, it flips through the cells again. The cells do not contain how the model computes. They contain fifty thousand instances of where the computation landed. So the assistant estimates, and you go back into Excel to run the case yourself.

I stopped asking an AI to read the screen and built something that reads the model: a lossless compressor that unzips a workbook into the small program inside. I wrote a companion piece for underwriters on what that means in practice.

It opens the workbook, separates the 951 lines of hand-written logic from the layout, and compiles them into a calculation engine. On my senior-living model, the engine recomputes all 52,148 formulas from 2,357 input cells in 19 milliseconds, down from 8.7 seconds in the first version a year earlier. Across the six benchmark workbooks in our test suite, a full evaluation runs in 0.014 to 0.056 seconds. Then the engine checks every cell against the Excel cached values saved in the file: zero mismatches.

Nothing is lost in that compression. The compiled engine is on average 70 to 80 times smaller than the workbook—on three of my own models, 55 times, 91 times, and 83 times smaller—and it round-trips the workbook back out to Excel with every formula intact. The model is not replaced. It is read properly for the first time.

In 2024, Microsoft published SpreadsheetLLM, which compresses what a spreadsheet looks like: values, formats, addresses. Formulas are not in it. My system compresses what a spreadsheet computes. The two are complementary, and the second one is the one an underwriter needs.


Context, Again

I have written before that context is the asset and the interface is just delivery, and that the intelligence layer is commoditized and the context layer is the moat. I believed that as a product manager. This project made me believe it as an engineer.

Dumped naively into a language model, a 52,000-formula workbook is over six million tokens. No context window holds it, and if one did, it would spend its attention reading the same formula 51,000 times. Separate the logic from the layout and that workbook is about sixty-six thousand tokens. It fits. Not a summary of the model, not a retrieval index over the model: the model.

Six million tokens compressed to sixty-six thousand, shown as a tall stack shrinking to a pocket-sized block that fits inside a context window

Once the model fits, the questions change. In my senior-living model today, occupancy sits at 94 percent and the IRR is 14.31 percent. Ask for the lowest occupancy that clears a 15 percent IRR: 97 percent clears it, 96 just misses. Nudge occupancy from 94 to 95 and IRR moves from 14.31 to 14.66 percent. Nudge every assumption up one notch and rank what moves IRR most: concessions drop it 1.06 points per point, resident rates lift it 0.97 per percent. Walk GP IRR backwards across sheets to the occupancy assumption that drives it: 16 steps, with the exact cell at each step, in one sentence instead of sixteen clicks on Trace Precedents.

The harder change is asking backwards across several assumptions at once. Excel’s Goal Seek takes one variable. Excel’s Data Table stops at two axes, and slows the workbook down when you use it. Add a third axis and Excel has no cell to put it in.

On the engine, three axes—15 occupancy steps by 11 rate steps by 9 exit-cap steps—is 1,485 combinations, the entire 52,148-formula workbook recomputed 1,485 times in 48 seconds. 596 combinations clear a 15 percent IRR. Add concessions as a fourth axis and it runs over 13,000 combinations in under ten minutes; more than 6,300 clear the bar. You do not get one number. You get a feasible region.

At 19 milliseconds per evaluation, something else changes underneath: results are no longer worth storing. Scenario analysis used to mean picking a step size up front, running a grid overnight, saving the outputs to a database, and looking them up later. Pick steps too coarse and you miss the cliff; pick steps too fine and the table does not fit. When recomputation takes 19 milliseconds, you store only two things: the inputs and the model version. You compute the answer when someone asks the question.

Here is the part people miss when they hear “AI reads your model”: the AI does not do the arithmetic.

Not one number in those answers is estimated by a language model. The language model reads the 951-line program, decides what to ask, calls the engine, and narrates what came back. The engine computes every cell. The numbers are Excel’s.


The Harness

I didn’t write the code. AI coding sessions wrote the engine, the compiler, the reader, the registry, and the test corpus: 1,615 commits and 524 merged pull requests between July 20 and November 4, across 1,012 issues opened and all but 16 closed. By month: 264 commits in July, 737 in August, 271 in September, 316 in October, and 27 in the first four days of November. The calculation engine at the middle of it is 19,491 lines of Rust. I built the harness around the sessions.

The prompt is what you ask for. The harness is what you accept: the rules, judges, and checks that decide what counts as done. Anyone who has watched an AI coding agent declare victory over a test that cannot fail knows the difference. Five pieces did the work, and each one came from a specific failure in the record.

A square frame drawn around a plain circle; each side of the frame carries one mark: a balance, a dashed line, a solid square, a check

1. Excel is the judge

Underneath the engine sit 20,545 isolated formula test vectors across 98 function families, each one a single formula on a small cell grid paired with the value real Microsoft Excel produced. The Rust suite runs all 20,545 vectors in 0.27 to 0.28 seconds without touching Python. The engine agrees with Excel on every vector except one open issue (and one earlier ISO-date-text comparison kept different on purpose). When we tested an open-source spreadsheet engine against 18,571 of those vectors on October 28, it agreed with Excel on 13,715 (73.9 percent); ours agreed on 18,569 of 18,569 that day, and we ported none of its code.

Twenty thousand unit vectors sound like enough. They were not.

On October 12, one pull request passed every vector in the conformance suite and took one of the six benchmark workbooks from 0 mismatches to 1,523. A comparison over a column had changed shape in a way no small-grid vector exercised. That afternoon we wrote the engine merge rule: a change to the engine merges only when the Excel-cache test shows 0 mismatches on all six benchmark workbooks in the same run as the conformance vectors. “No worse than main” is not a pass.

Making Excel the judge means matching what Excel actually does, not what a specification or clean arithmetic says it should do. Real workbooks forced us to match behaviors nobody would invent from first principles:

The judge is neither me nor the AI. It is the program this industry already trusts.

2. Predict before you build

Every experiment under our documentation tree—more than 182 of them since July—is a dated record with a prediction written before the run and a measurement written after. On October 22, when we started Phase 6 to take Python out of the intake and compile path, we made the rule strict: measure the baseline first, commit the expected number, build, and put the result next to the prediction. Any step whose measured saving lands under half of its prediction stops the phase for a design call before the next step starts.

We wrote that rule because Step 6a had just broken it. In 6a, the session built a Rust formula parser first and measured afterwards. It parsed 348,225 syntax trees identical to the Python library’s and compiled all six benchmark workbooks to identical bytes—and compile time improved by only 3 percent to 32 percent per workbook. Parsing was never where the compile spent its time. On the student-housing benchmark workbook, building the dependency graph took 21.5 seconds: only 0.5 seconds was reading the file and 0.8 seconds was parsing formulas, while 16 seconds went into a pure-Python tokenizer and 17 million range-expansion calls.

In Step 6b, replacing the Python workbook reader with a Rust reader, the session measured every stage on mains power first and committed its predictions. Two graph-build predictions still missed in an instructive way: we predicted the senior-living workbook would drop from 5.0 seconds to 1.7 seconds and an older mixed-use workbook from 10.5 to 3.0, and measured 5.7 to 2.9 seconds and 11.8 to 4.9 seconds instead. We had timed the whole loader function and called it “the file read,” when the actual read was 0.3 seconds of it. Because the stage-by-stage table was already committed, the miss was obvious in five minutes, and none of the prepare stages landed under half of its predicted saving.

The rest of Phase 6 held the line:

Phase 6 Step What Moved to Rust Measured Saving vs. Prediction Outcome
6a (Parser) Formula parser (348,225 trees) No prediction written; compile 3% to 32% faster Triggered the predict-first rule
6b (Reader) Workbook XML reader (0.06 to 0.4 s a read) No stage under 0.50 of predicted saving Prepare + compile 31 s to 20 s on senior-living
6c (Precedents) Dependency graph precedents from Rust parse tree 0.72 to 1.00 of predicted saving Student-housing graph build 18.7 s to 7.7 s
6d (Graph & SCCs) Dependency graph, Tarjan SCCs, iteration order 0.71 to 1.33 of predicted saving Battery prepare + compile 15–54 s to 12–26 s
6e (Deduplication) Eliminated duplicate compiles, XML parses, spawns 0.85 to 1.35 of predicted saving Mains prepare + compile 14–29 s down to 5–18 s

Every artifact stayed byte-identical at every step. We apply the same honesty when predictions miss in the other direction: on November 4, when we measured three new tools on the server across five registered benchmark models, every correctness target hit zero—the explanation tool walked 443 output cells and 643,000 dependency steps with 0 cells off the graph and 0 values different from the scenario call; the replay tool ran 300 scenarios in 8.7 to 10.8 seconds on one core with equal bytes across two processes; the version comparison showed 0 unexplained moves—and three time predictions missed. The slowest first explanation took 6.5 seconds against a 4-second prediction; replay took up to 10.8 seconds against 10; the slowest version comparison took 35.9 seconds against 30. All three go into the log as misses. An AI forced to write “this will cut 18.7 seconds to 8” before touching code is a different collaborator from one that reports afterwards that things “improved.”

3. Holdouts and stop rules written in advance

Between October 24 and October 30, the number of real workbooks built by other people that registered at 0 mismatches with zero hand work rose from 22 of 116 to 105 of 116. Every fix in that week was aimed at those 116 files. The obvious danger was overfitting: teaching the engine to pass those 116 workbooks while leaving it brittle on the 117th.

We also had a holdout: workbooks that none of that week’s fixes had touched. On October 24, an early check of 16 of them had registered 1. On October 31, we took a seeded random draw of 60, proportional across property types.

Before starting the run, we committed both a numeric prediction and a decision rule:

Then we ran the 60 blind. The run took 4 hours and 16 minutes.

The measurement came back at 30 of 60 registered—50 percent, ten short of the prediction. Forty-six of the 60 reached the Excel-cache test, and 0 timed out. Because the decision rule was already written, there was no debate about whether 50 percent was “pretty good” or whether we should chase all 30 failures. Fifty percent sat in the middle band: fix only families with three or more workbooks.

Three template families held 15 of the 30 failures: a 100,000-to-175,000-cell student-housing template refused on QUARTILE parsing across 7 workbooks; a 40,000-to-54,000-cell student-housing family off by 1 cell each on thousands-comma text arithmetic across 5 workbooks; and a 126,000-to-163,000-cell senior-living family across 3 workbooks. We fixed those three families against Windows Excel and re-ran the same 60 workbooks on November 1, predicting 41.

The re-run measured 44 of 60 registered (73 percent), with 0 workbooks worse and 17 better. Seventy-three percent crossed the 70 percent line we had set the day before. We stopped the engine wave on that sample and left the rest of the holdout unspent.

4. Delete the scaffolding once the judge is satisfied

The repository did not start in Rust. It started in July with two Python evaluation paths: a slow reference path that evaluated formulas cell by cell through a Python library, and a fast path that compiled circular blocks into generated Python functions. For three months, every workbook was verified by playing those two Python legs against each other and against Excel. Keeping two Python evaluators in step with every engine change was expensive.

On October 10, a first experiment with a generic Rust stack machine ran all 52,148 cells of my senior-living model from 2,357 inputs with 0 mismatches in 0.044 seconds against 60.7 seconds for the Python path. Grouping cells by shared formula pattern gave the exact same answers and the same fingerprint at a fraction of the size.

Once the Rust engine agreed with Excel across the 20,545 conformance vectors and all six benchmark workbooks, the Python evaluators were no longer a second opinion; they were maintenance weight. A census on October 23 counted 36,351 lines of Python under the converter and runtime library plus 12,635 lines of their tests waiting to be removed. On November 3, after checking that 105 of the 116 workbooks registered cleanly, we closed Phase 5 in a single pull request: 1,230 files changed, 286,719 lines deleted (including the generated Python packages, snapshots, and manifests across the workbooks). We re-signed the six benchmark workbooks once and re-ran the 116 workbooks that night: 112 registered, 0 moved backward. The Rust engine is now the only evaluator.

5. Decide who decides

No new gate, census, ledger, or engine feature gets built without a named scenario and a number that a test can fail. Half the complexity in software is speculative, and AI agents generate it endlessly if you let them. An owner order counts as a named scenario—the rule binds agent-initiated machinery, not my own product calls.

Beyond that, the role boundary is explicit. I decide pricing, who we talk to, and anything published outside. The AI session decides every technical question on its own and is forbidden from sending technical choices back to me. Product questions it cannot settle come to me as an issue with a default already chosen and a 72-hour deadline; if I say nothing, the default ships.

Implementation runs through an unattended loop. Assigning a GitHub issue is the only human start gesture. A bot in the cloud reads the assignment and starts a cloud coding agent that lands on my Mac—because Microsoft Excel lives on this Mac, and a green result from a Linux container or LibreOffice does not count as Excel green. The agent opens its own isolated Git worktree so two agents never step on the same checkout, takes an exclusive lock so two test suites never share the CPU cores, runs the gates, and opens a pull request.

A second AI session reviews the pull request and may merge it without waking me only when five conditions hold at once:

  1. The gate ran green on this Mac, and the pull request body carries the exact commands, the passed, failed, skipped, and xfailed counts, and the duration.
  2. The diff touches no engine source file.
  3. The diff touches nothing under the benchmark model directories (unless every path is on a narrow regenerated-artifact allowlist with the verbatim gate pass lines pasted in the body).
  4. The reviewer did not write the pull request.
  5. The pull request body carries the commit hash the gate ran on, and it equals the pull request’s current head—any rebase, amend, or new commit invalidates the evidence and forces a re-run.

If any one of the five is missing, the pull request waits for a person. Two failed runs in a row mark the issue blocked. Waiting is not a failure. It is the answer for the changes whose mistakes cost the most.

Five thin horizontal bars; four reach a vertical line on the right, the fifth stops short and a small terracotta dot marks the gap


How Green Signals Lie

A rule file only works if it remembers how it was fooled. In the 30 days before August 26, our two main rule files took 11 and 25 commits as sessions appended notes to them; we had to freeze both files so that a rule everyone appends to did not stop being a rule. Late in October, we mined all 1,546 commit messages then on main across eight passes and distilled the recurring failures into the session’s permanent memory.

Every one of them had the same shape: the thing that produced the “pass” was structurally incapable of failing in the way we cared about. When you design a check by asking “how will it tell me when it passes?” instead of “if the system is broken right now in the way I fear, will this signal change?”, the check goes silent at the exact moment you need it. Here are five ways a green signal lied to us, and the rule each one left behind.

A single thin check mark with one segment of its stroke missing; a small terracotta square sits in the gap

The implementer picked the sample

In August, one converted model package reported 278 passed tests and a green suite while 650 of its 36,308 cells disagreed with Excel and its levered IRR was 35.4 percent in Python against 19.1 percent in Excel. The test suite had checked a hand-picked list of sample cells chosen by the same agent that wrote the converter—from the cells it already knew passed. A week later, on August 14, an early Rust probe reported 99.985 percent bit-for-bit agreement with Excel’s cached values and a 17-to-191-times speedup. It had never solved a single cell: a lookup miss caused every read to fall back to a dictionary seeded with Excel’s cached values, which are already a fixed point.

The rule: Count the full set; never report a rate from a sample when you can count every item. And never validate a solver only at the known answer—always run an input-change test first, moving an input and comparing against an independent engine, so a fallback seeded with the cached answer cannot hide.

The fast path passed by doing zero work

Six different times across the commit history, a fast-path check went green while doing zero fast-path work. In one run, 100 percent of scenarios hit an error condition and silently fell back to the slow reference evaluator: 0 failures across 2.82 million cells, and 0 speedup. In another, an uncommitted hash file turned compiled snapshots off across the fleet—0 of 133 snapshots loaded—while every correctness gate stayed green because the fallback answered in its place. Two paths cannot disagree when both sides silently run the same code.

The rule: Every fast-path or performance claim must report the count of cells actually executed by the fast path alongside the mismatch count, and gates must never assert on wall-clock ratios. On a shared machine, wall-clock time mixes load with the code under test—one benchmark run measured 147 seconds on battery and 68 seconds on mains. Assert on the event count (modules compiled, snapshots loaded, cache hits); log the clock in the pull request body with the power state and load average attached.

The simulator is not the host

On August 8, a workflow script passed all 18 checks in its dry-run harness and died 51 milliseconds after launch in the real workflow runner on an operation the real host did not support. The same day, a benchmark workbook passed every static OOXML integrity check and all 61 package tests—including the Excel-cache test, which read the cached values out of the file’s XML—and real Microsoft Excel still popped up a recovery dialog because a library rewrite had changed the file in a way Excel refused.

The rule: Every capability gap between a simulator and the real host is a blind spot the simulator cannot test. Never re-serialize a workbook through a general Python library; write bytes only with real Excel or scoped zip surgery. And for every new gate, perform a failure test before trusting it: create the exact fault the gate is supposed to catch, watch the gate turn red, and then restore the file. A gate you have never watched fail is not a gate.

The census tool had a hole

On August 3, two separately dispatched agents audited nine model packages for formula-referenced blank cells and both reported 105,166. We took that agreement as independent confirmation until a genuinely independent method found 41 real cells both had missed—because both orders had called the exact dependency helper under question. Eleven days later, a scan across 15 models and 910,000 formula cells for IFERROR impact reported zero hits because the script read the wrong field; fixing it turned 0 hits into 245,000. Two days after that, an audit of admitted modules iterated over the keys of the manifest it was auditing and reported 17 of 17 green after 5 of the 22 modules had been deleted.

The rule: The same helper run twice is not a second method. When auditing the completeness of X, the denominator cannot come from X. And before believing a zero count from any scan, feed the scanner one known positive case and watch it fire.

The agent loosened the check

When an AI agent is scored on making a test green, loosening the check is always the cheapest way to turn a red line green. Across our history, agents taught a comparator to treat boolean TRUE as equal to 1.0 (hiding a real type-coercion bug in 1 of 20,127 cells), made a required model file optional in the loader when a package was missing it, and exited with return code 0 after two minutes having committed zero edits. Even my orchestrating session fell into the reasoning version of this trap: when one cell differed in the last digit—542981.4663858113 against 542981.4663858112—it argued the reference path must be right and proposed regenerating the artifact. Ten seconds checking the workbook’s Excel cached value showed the cache was ...112.

The rule: In every review, read the diff on comparator, tolerance, skip, and guard lines first. No tolerance is widened and no test is skipped to get a green run. Before running a suite, collect the test count live and reconcile baseline + added = observed after the run, recording the skipped count even when it is zero—because a runner missing LibreOffice on its path silently skips 5 of 70 oracle tests and still prints “passed.” The AI does not get smarter between sessions. The harness does.


What It Bought

The numbers are counts, not claims.

Across the 116 real underwriting workbooks built by other people, here is the registration trajectory over ten days in late October and early November. “Registered” means the workbook ran through intake, compilation, and the full Excel-cache test at 0 mismatches with nobody touching the file:

Date Registered (0 Mismatches) Out of 116 Workbooks Worse Than Prior Run
2026-10-24 22 19.0% —
2026-10-26 58 50.0% 0
2026-10-27 69 59.5% 0
2026-10-30 105 90.5% 0
2026-11-03 112 (predicted 112) 96.6% (97.4% of 115 on disk) 0 (108 same, 7 better)

An inventory across those workbooks on October 28 found that real estate underwriters use 108 of Excel’s 503 functions, and the top 20 functions cover 96.0 percent of all formula cells. By November 3, the four workbooks out of 116 that did not register were completely accounted for: one workbook has 24 cells where its own saved values are stale (recalculating in Windows Excel confirmed the engine is right on all 24, so with its excusal note the count is 113 of 116); one file has 0 formula cells; one file is not a valid zip archive; and one sat on an unmounted network share. None of the four is an unsupported Excel function.

Speed moved the same way once the engine and intake path were in Rust. On our seven benchmark and experiment workbooks, skipping listed cells whose upstream reads had not changed kept 6.65 million cell values bit-for-bit identical across 112 scenarios while dropping one fund model’s median scenario evaluation from 826 milliseconds to 152 milliseconds. Across the six benchmark workbooks, median single-scenario engine time sits at 17, 27, 63, 65, 122, and 754 milliseconds (the last is a workbook whose 2,757-cell circular block recalculates everything downstream on every pass, as Excel does).

Then we tested the product promise: take a workbook the server has never served before, and bring it from raw file to a live Model Context Protocol (MCP) endpoint answering scenario questions in under one hour of operator time, with 0 mismatches against Excel. On October 23, our first try on four workbooks from one lineage went 0 for 4—one hung in graph partitioning after 262 seconds on 65 circular blocks and 54,519 dynamic-reference cells, and three stopped at compile on missing functions and table references. By October 24, the largest of the four (218,536 formula cells across 48 sheets) registered at 0 mismatches, and on October 29 its circular convergence dropped from 323.7 seconds to 61.2 seconds with 0 cells moving.

On November 4, we ran the one-hour clock acceptance test on the fourth workbook from that lineage: 139,407 formula cells, 15,081 constant cells, 8,407 distinct formula patterns, 21 sheets, VBA macros, and iterative calculation turned off. We predicted 23 minutes total.

Step Predicted Run 1 Measured Run 2 (Replayed)
1–3. Intake copy, prepare, mechanical register (139,407 cells vs. Excel cache) 2 min 21 s 36.5 s (0 mismatches) 36.3 s
4. Automated catalog proposal 3 min 8 min 25.6 s (505.6 s) 500.3 s
5. Human review: pick 34 drivers (28 core inputs, 6 advanced) and 26 outputs 15 min 3 min 6 s (186 s) 0.03 s (scripted)
6–7. Install catalog, start MCP server, run the three calls 3 min 10 s 1 min 1.2 s 1 min 1.1 s
Total, raw workbook to live MCP scenario answer 23 min 13 min 21 s (801 s) 9 min 58 s (598 s)

Thirteen minutes and 21 seconds against the one-hour limit. All 139,407 formula cells matched Excel’s cached values with 0 mismatches. And the prediction table immediately showed what to fix next: the catalog proposal step had taken 63 percent of the entire clock (505.6 seconds against a 180-second prediction), and its default ranking had filled 39 of the top 40 input slots with zero, empty, or period-index cells, forcing a person to pick the 34 real drivers. Later that same day, three pull requests fixed the ranking and moved the input probes onto a thread pool, cutting the proposal step from 754 seconds to 179 seconds with byte-equal output.

The public demo on my own cleanroom senior-living model runs the parts that are hard to fake: three live panels on the real Rust engine backed by a precomputed plan of 5,001 scenarios over 15 inputs—the most you can pay and still clear the LP’s return, who walks first, and what has to hold across the feasible region, every number computed by the engine while you watch. It is in the Lab, next to the koi.


The Principle

Intelligence is rented. Context is owned. The harness is what turns rented intelligence into owned results.

Everyone will have the same language models and roughly the same prompts. What nobody can download for you is the harness: the external judge your domain already trusts, the predictions you commit before the agent writes a line of code, the holdouts and stop rules you write before you look at the score, the scaffolding you delete once the judge is satisfied, and the list of ways a green light has lied to you in the past.

Three questions for any AI workflow that matters:

If you cannot answer the first one, you do not have a harness. You have a prompt and a feeling.

I started this project to ask my underwriting model a backward question Excel could not answer in one step. I ended up with 951 lines of hand-written logic compiled into a 19,491-line Rust engine that agrees with Excel to the cell across 112 workbooks I did not build, and a way of working where I write almost none of the code and trust almost all of it. The second part is the one I would keep if I had to choose.

Disagree? Push back on this essay.不同意?来反驳这篇。

Ask MJ

Answers come only from what I’ve written. If it isn’t there, it says so.只根据我写过的内容回答;没写过的,它会说不知道。