Characterization and Golden-Master Tests

A characterization test records what an existing system actually does, especially when its requirements are poorly understood. It can warn that a refactoring changed an important behavior before anyone has decided whether that behavior should change.

A golden master, also called an approved output or snapshot, stores a reviewed example of a larger result. A test compares a fresh result with that saved output and shows differences. Approval-testing tools automate that comparison and review workflow. 1

The important word is reviewed. Copying today’s output into a file does not prove that today’s output is correct.

First observe, then record

Suppose an old receipt generator produces this text for Mira buying two items:

CUSTOMER: Mira
ITEMS: 2
TOTAL:  10 coins

There are two spaces after TOTAL:. Perhaps a printer template depends on the alignment. Perhaps nobody does. Before reorganizing the function, we want to notice a change rather than unknowingly tidy it away.

public static class LegacyReceipt
{
    public static string Render(string name, int items)
    {
        return "CUSTOMER: " + name + "\n" +
               "ITEMS: " + items + "\n" +
               "TOTAL:  " + (items * 5) + " coins\n";
    }
}

public static class Tests
{
    public static void ReceiptMatchesRecordedOutput()
    {
        const string approved =
            "CUSTOMER: Mira\nITEMS: 2\nTOTAL:  10 coins\n";
        string actual = LegacyReceipt.Render("Mira", 2);
        if (actual != approved)
            throw new Exception("Receipt differs from the reviewed output.");
    }
}

Tests.ReceiptMatchesRecordedOutput();

def render_legacy_receipt(name, items):
    return (
        "CUSTOMER: " + name + "\n"
        + "ITEMS: " + str(items) + "\n"
        + "TOTAL:  " + str(items * 5) + " coins\n"
    )


def test_receipt_matches_recorded_output():
    approved = "CUSTOMER: Mira\nITEMS: 2\nTOTAL:  10 coins\n"
    actual = render_legacy_receipt("Mira", 2)
    assert actual == approved

test_receipt_matches_recorded_output()

A difference asks for a decision

If a rewrite removes the extra space, inspect whether that change was intended. For a behavior-preserving refactoring, restore the old output. For an approved formatting change, update the expected output deliberately and record the reason.

Do not use “update all snapshots” as an automatic repair for a failing suite. That can approve a missing customer name, an incorrect total, and a valid formatting change together. A smaller test for the total can help distinguish a business-rule failure from a presentation difference.

Characterization also applies to exceptions, ordering, side effects, and accepted inputs—not just printable output. An old parser accepting blank fields or an API returning a particular error may be behavior existing users rely on. Capture the relevant cases before changing them.

Stabilize noise without hiding meaning

Timestamps, random identifiers, temporary paths, and unordered output can make snapshots change on every run. Supply a fixed clock or deterministic input where practical. If comparison normalizes an irrelevant field, state exactly what is removed and why.

A normalizer that deletes every number would also erase evidence that the total changed. A snapshot should preserve the information the test is supposed to protect.

Finally, one receipt is one observed case. It says nothing about zero items, Unicode names, or unusual quantities. Select representative inputs and meaningful boundaries. The goal is a useful safety net around existing behavior, not a museum that makes every legacy bug untouchable.

Sources

1. ApprovalTests, tools and documentation for reviewed-output comparisons.
2. Martin Fowler — Self-Testing Code, on automated regression checks as support for change.