Guardrails
Guardrails before the lift
A checkout badge that lifts tap-through and quietly raises refunds is not a winning variant. It is a transfer from finance to a screenshot. Guardrail Grammar exists because we kept seeing that transfer dressed up as app analytics.
In every readout we score, the header has three lines before any chart: primary, MDE, guardrail. If the guardrail is missing, the memo is unfinished. Students sometimes put refunds in an appendix “for completeness.” Completeness is not the point. The point is that a PM will not scroll.
Which guardrails earn a header slot
Not every downstream metric deserves the same weight. We ask for the one that would make shipping embarrassing if it moved the wrong way. For marketplaces that is often refund rate or COD failure. For subscription apps it is cancel-before-pay or trial-to-paid. For games it may be crash-free sessions or support tickets about the new reward.
Two guardrails are allowed. Five are a way to never ship. If you cannot name one, you do not have a feature test yet. You have a curiosity.
How we treat a bleeding guardrail
Primary up, guardrail down: hold. Not “ship and watch.” Hold. The Bench Pair case on the checkout badge followed that rule. Tap-through looked handsome. Refunds in the same cohort rose 0.9pp. The badge came off. Thirty-day paid did not collapse. Marketing was briefly unhappy. Engineering was not.
Primary flat, guardrail worse: kill. Primary down, guardrail fine: still a hold unless you had pre-declared a non-inferiority frame, which almost nobody has.
Thailand-specific noise
Support-ticket volume spikes around holidays for reasons that have nothing to do with your flag. If you use tickets as a guardrail, declare the holiday window the same way you declare runtime. Otherwise you will blame a variant for Songkran.
Guardrail Grammar spends two of its four sessions on this calendar problem. The other two are on writing the header so a designer can repeat it. Charts still exist. They just do not lead.
See Guardrail Grammar on the catalogue · Back to the journal