Assertions are just temporarily embarrassed snapshots
Never spend time iffing, always compare for exact equality.
A good test has two purposes: to have fun, and to fail.
The most important function of a test suite is to be a garden of fun starting points (hopefully a wild garden). Want to make a wreath out of orchids? Too bad, orchids are impossible to grow, you’ll never have a wreath’s worth of them. But if you’ve spent the past year cultivating an orchid greenhouse? Sure, make an orchid wreath, weave their petals into orchid-wicker furniture, you can do anything.
The story for most tests is that they pass once, and then continue to pass for as long as anyone is looking. Such tests matter only insomuch as they were fun — which is enough to matter a lot! Look at the colorful tests above - none of them have ever created a red X on a dashboard.
Which gets us to our most important point. Snapshot tests show you what you have. They ask nothing of you, they just show you what is there. And that’s fun! What you actually have is often surprising, and the easier it is to find that out, the better.
Assertions force you to mathematically declare what you want, but nobody cares what you want. The code you are testing doesn’t care what you want. You don’t even know what you want, you’re faking it. When an assertion fails, it says “You don’t have what you claim to want.” Then leaves you to die. With luck you can catch a ride with a println or debugger for a reluctant trip to the “what do I have” store. After making a purchase, the assertion will again tell you whether or not you have what you claim to want. When it fails (again), the assertion will make very little effort (again) to tell you what you have, let alone what you used to have before the assertion started going out of its way to leave you for dead. What you used to have when you were happy is git’s job, get him to do it.
Which is the other magic part of snapshot tests. When a snapshot test starts failing, it doesn’t just show you what you have, it shows you what you used to have too! With a beautiful diff right in the IDE, no git required.
And if you are able to use git, wow! There is so much you can see by watching the history of a snapshot. Here it is, spelled out in plaintext, what you have, not just now, but over time — everything you ever had, and when you had it. With assertions, you can see everything you ever claimed to want, and when you claimed to want it.
If snapshot tests are so great, how come no one uses them?
So if snapshot tests are so great, how come no one uses them? Because everyone is uninterested in what they have, fixated only on what they think they should want. Thusly are they robbed of ever discovering what they actually want, let alone getting any of it!
IDE
Also because everyone runs their tests through their IDE, and they don’t know how to add --updateSnapshot to the ⏵ button. For good reason - it’s impossible!
Lucky for you, there is a new snapshot testing library, selfie, which has figured this out: control comments!1 With selfie, the snapshots are always in read mode. When you want to write the snapshots (like --updateSnapshot), you just put //selfieonce into the source code. When selfie runs, it will update the snapshots and remove the //selfieonce comment so that you’re safely back into read mode. Just smash that ⏵ button, no command line arguments needed!
Another reason people don’t use snapshots is because stale snapshots from deleted tests tend to pile up. Selfie fixes this with precise garbage collection by parsing the test code to determine which snapshots might still be alive, and discreetly eliminating the others.
Of all the reasons people don’t use snapshots, there is only one that is good. People want the text of the test itself to tell a story! (comments don’t count)
👍 when <push button> then <html contains “ordered”>.
👎 when <push button> then <html equalsSnapshot> // it says ordered, trust me
One solution is to put the snapshot inline with the source code instead of on disk.
🤢 when <push button> then <html equalsSnapshot “<html><head><href=’internet stylesheet’></head><div><div><divvyMcDivface><alert><body><span class=”tw:x tw:2 tw:tw">ordered</span>”>.
Good tooling could write the long string for us with all the nice properties of snapshots, but it has low signal to noise. The assertion <text contains “ordered”> is clearly better, even if it will eventually leave us for dead. The solution is to transform the value under test first, and then do the inline snapshot.
🤤 when <push button> then <asMarkdown(html) equalsSnapshot “ordered”>
Transforming snapshots in this way is easy to do, and selfie’s lensing features make it especially easy.
Visit an aircraft factory, and you’ll see CNC machines humming along in the kiloflops, tracing precise paths to achieve precise goals. Visit a programmer’s laptop, and you’ll see vomit spattered from shoggoths writhing in the clouds above. The programmer opens the shutter and a deluge of teraflops floods the workspace until the levees orienting the programmer to their task start to creak. The shutter closes and the programmer reorients to aim the next blast of compute as they dynamite their way through conceptual space.
Testing software isn’t about verifying properties anymore, it’s about tailoring a suit for an octopus. Not because the octopus needs a suit; it’s the tailor that needs a map. And the tailor only has one hand - better hand some pins to the octopus!
Figuring out the control comments thing was thanks to Rich Hickey’s decision matrix idea, here was the matrix we used.