Characterization tests before AI refactoring: learn what untested code does

A formatter turns build:cache:hit into a note ending in cache. Another input leaves doubled spaces in the output. The code has no tests telling you whether either behavior was intended. Before asking AI to simplify it, you need a way to notice if those results change.

Suppose this formatter writes dated notes from label:detail strings. Download the complete characterization test example and run it with Node.js 22 or later. The recorded run used 22.22.1, without third-party dependencies.

Observe a few inputs before proposing a better implementation

Unzip the download, enter characterization-tests-before-ai-refactoring, and run:

node observe.mjs original

The command prints twelve observations as JSON, including each input, a supplied instant, a supplied environment, and the output. Start with an ordinary string, then change one detail at a time. These are two entries from that run, shown with only the input and output:

[
  {
    "text": "build:cache:hit",
    "output": "2026-10-13 | BUILD | cache"
  },
  {
    "text": " build : ready ",
    "output": "2026-10-13 | BUILD  |  ready"
  }
]

The second output has two spaces before and after the last |. Keep them visible while investigating. The other observations show that build and build: both end in -; an empty label uses the supplied NOTE_LABEL, or note when that value is missing or empty.

Now open original.mjs:

export function formatNote(text, runtime = systemRuntime) {
  const fields = text.trim().split(':');
  const fallback = runtime.readEnv('NOTE_LABEL') || 'note';
  const label = (fields[0] || fallback).toUpperCase();
  const detail = fields[1] || '-';
  const day = runtime.now().toISOString().slice(0, 10);
  return `${day} | ${label} | ${detail}`;
}

fields[1] explains the lost hit: later fields are never used. The first trim() applies to the entire input, so it leaves the space after build and the space before ready. Rendering adds its own separator spaces beside them.

An observation gives you something to investigate before choosing a fix. Michael Feathers describes a similar progression from calling existing code, to inspecting its response, to saving useful interactions as tests in Unit Conversations.

Make the date and environment explicit

A saved output containing today’s date will fail tomorrow. A developer’s NOTE_LABEL can also change the output of a blank label. Those values belong in the test input.

The example already provides a small replacement point: the optional runtime argument. Its default reads the clock and the named process environment variable. Every documented observation and test command instead supplies this implementation from runtime.mjs:

export function fixedRuntime({ at, env }) {
  return {
    now: () => new Date(at),
    readEnv: (name) => env[name],
  };
}

Most cases use 2026-10-13T12:00:00.000Z and { NOTE_LABEL: 'daily' }. One uses 2026-10-13T23:30:00-07:00; its output starts with 2026-10-14, because toISOString() formats UTC. Missing and empty environment values have their own cases.

A replacement point like this is commonly called a seam. Fowler’s Legacy Seam discusses separating a dependency so tests can choose its behavior. This download starts with that parameter available. If real code has the dependency buried inside it, introduce the smallest suitable seam as a separate, reviewed change before reorganizing the parser; preserve how the default path obtains its values.

Here, no test changes the global clock or edits the process environment. The checks also passed with the child process set to TZ=Pacific/Kiritimati and a different NOTE_LABEL, confirming that the supplied inputs determine these observations. The default adapter’s interaction with a real application still needs its own integration check.

Save what happened, including the parts you question

cases.mjs holds the exploratory inputs. After running and inspecting the original results, we saved them in characterized.json. It records exact strings, including the unexpected spaces and discarded fields.

A characterization test asks whether the selected behavior has changed. It does not prove that the recorded behavior is correct. For example, preserving cache in this fixture leaves open whether the application should actually retain cache:hit.

run.mjs reads the saved JSON and compares each implementation’s result with it:

for (const item of fixtures) {
  test(item.name, () => {
    const actual = versions[mode](item.text, fixedRuntime(item));
    assert.equal(actual, item.output);
  });
}

The runner never calls the original formatter to generate an expected result during the assertion. Changing the original implementation therefore cannot silently redefine the saved answer. observe.mjs only prints; it does not overwrite the fixture either.

Run the original against that record:

node run.mjs original

All twelve checks pass. They cover ordinary input, missing and empty details, later fields, internal spaces, blank inputs, supplied and missing fallback labels, and the UTC date case. These are selected string inputs. Unicode casing, invalid dates, non-string arguments, side-effect order, and the application’s real callers have not been characterized here.

Refactor while keeping the saved results

The behavior-preserving version extracts parsing and rendering into named functions. Its parser deliberately retains the original choice of fields:

function parseFields(text) {
  const [label, detail] = text.trim().split(':');
  return { label, detail };
}

function render(day, label, detail) {
  return `${day} | ${label} | ${detail}`;
}

refactored.mjs uses those functions while keeping the fallback rules, uppercasing and UTC date conversion. Run:

node run.mjs refactored

All twelve saved observations pass, and its complete observation output matches the original byte for byte. That is useful support for this refactor over the recorded inputs, while leaving the untested cases visible.

If you ask AI to help at this stage, give it the source and the characterization cases. Ask it to preserve the existing strings and identify any change it thinks would alter behavior. A request to “clean this up” leaves much more room for an unannounced decision about spaces or delimiters.

Let an attractive cleanup fail visibly

The hypothetical cleanup in accidental.mjs uses a parser that may look more convenient:

function parseFields(text) {
  const [label, ...rest] = text
    .trim()
    .split(':')
    .map((field) => field.trim());
  return { label, detail: rest.join(':') };
}

It trims each field and rejoins the rest. Run the three versions separately:

node run.mjs original
node run.mjs refactored
node run.mjs accidental

Version	Passed	Failed	Exit code
original	12	0	0
refactored	12	0	0
accidental	10	2	1, expected failure

The failing cases are exactly the two surprises observed earlier. build:cache:hit now ends in cache:hit, and the spaced input now produces 2026-10-13 | BUILD | ready. Neither change has been hidden by regenerating the fixture.

Use node observe.mjs accidental to inspect those new outputs. Run commands separately if you want to continue after the expected failure; joining them with && stops at exit code 1. The default node --test run.mjs checks only original.

The red result means that the cleanup changed recorded behavior. Whether either new output is preferable is the next decision.

Decide on one behavior change separately

DECISION.md contains a proposed change for the extra-colon case: retain cache:hit instead of cache. It records the old and proposed outputs and leaves the proposal out of the behavior-preserving refactor.

In a real application, inspect representative inputs and the consumers of this string before accepting that change. The saved observation cannot tell you whether a user expects the discarded portion or relies on the shorter form. If the new rule is accepted, implement it in a separate patch, update the relevant expectation with its reason, and add cases for the newly agreed behavior. Keep the whitespace and date rules unchanged in that patch.

This is also where code change impact analysis becomes useful: the observation identifies a difference; callers and their tests help assess its consequences. When a patch already includes tests, review the assertions changed alongside it to see whether they still check the intended behavior.

Use the observations to guide the next reading session

If you cannot yet find the source of the formatted value, start by following one value through the unfamiliar codebase. Once the formatter is located, keep its current implementation, the recorded cases and the proposed refactor together.

An optional AI prompt is:

Explain how this formatter produces each recorded output. Point to the code for discarded fields, retained spaces, fallback labels and the date. Suggest a refactor that preserves these observations, and list any proposed behavior change separately. Do not infer that a saved output is a requirement.