We say what
moved.
Not why.
Every experiment is written down before it runs, judged by the rules it agreed to, and published whichever way it lands.
What “attributed under agreed methodology” means
It means two things. The number moved, and we counted it using a rule we wrote down before we started. It does not mean the experiment is the reason the number moved — almost nobody can honestly prove that, so we do not claim it. You will not find the words “caused” or “drove” next to a result anywhere on this site.
The rules are fixed before the money is set aside. They cannot be rewritten once the answer is in, and the result page says plainly what it could not account for.
This is not fine print. It is the part that tells you how much to trust the number above it.
The ten things written down before it starts
- 01 · Where it startedWhat the number was before, and over what stretch of time.
- 02 · What we expectOne claim that could turn out to be wrong. If it cannot be wrong, there is nothing to test.
- 03 · What changesExactly what will be built, bought or switched on.
- 04 · BudgetHow much is set aside from the pot to pay for it.
- 05 · The number we judge it onOne number, picked before we start.
- 06 · How we count itHow long we watch, who counts as a new customer, what happens to refunds — all agreed before any money moves.
- 07 · When it starts and stopsBoth dates fixed in advance, so nobody picks a flattering window later.
- 08 · What must stay the samePrices, the landing page, the sign-up flow — for this run only, fixed in advance and never more than thirty days. Change one and the scorecard is void: the run publishes nothing, and the money is still spent. You can change whatever you like the day it ends.
- 09 · What happenedWhat that one number did.
- 10 · What it cannot tell youHow many people it covers, what we could not rule out, and where to be careful.
Three outcomes, one treatment
Failures are published at the same size as wins — that is the whole point.A negative scorecard uses the same layout, the same type sizes, and the same share card as a win. Red is reserved for the one state meaning “do not rely on this yet”.
The number moved the way the idea said it would. Published in full.
The result cannot be separated from noise. Published in full.
The number moved the wrong way, or did not move at all. Published in full.
How much to trust one result
Confidence is a judgement, recorded when the result is written and published with the limits that produced it. It is not a calculation, and there is no significance test behind it — almost nothing here runs with a comparison group, so a number claiming statistical confidence would be inventing precision we do not have.
Enough people, over enough days, that the direction would be hard to explain any other way. Still not proof of cause.
The direction is probably real, but the run was small or short enough that a different month could have produced a different answer.
Worth publishing and not worth acting on alone. Treat it as one weak reading, not a finding.
Challenging a result
Anyone involved can dispute a result for fourteen days. Both the challenge and the answer are published.
Standing belongs to the participating founder, the thesis author, verified backers, that founder’s supporters, and the pilot reviewer. A challenge must identify a specific methodological or factual issue with evidence — unsupported commentary stays out of the process, and the sender is told why.
When one is upheld
The result is restated, never deleted. The history shows both versions, so a correction cannot be used to quietly erase what was originally claimed. A scorecard under review says so on the page, on the share card, and in the backer’s portfolio.
If a frozen variable turns out to have changed mid-run, the scorecard is void and the spend still stands. The money was spent either way, and saying so is the point.
A challenge with standing was filed. The result stays published while it is re-checked. It replaces the outcome word and the confidence grade until the challenge is answered — it does not grey the result out, move it down the page, or take it off the record.
No result on this site carries it. Nothing has been challenged yet, so this is a specimen rather than a live example.