Geysera Signal

Prove a change actually worked

The job: you raised prices, shipped a new landing page, or started a campaign. Revenue moved. Find out whether your change is why.

Who this is for: anyone who has to defend a decision, or decide whether to do it again.

What it costs you: one question and about fifteen seconds.

Before you start

  • You need an API key with the copilot scope — this analysis is only reachable through the assistant, not the four public GETs. Scopes are fixed at creation, so pick it when you mint the key; a read-only key gets 403.
  • You need a connected store, because every metric here comes from orders.
  • You need at least 14 days of history either side of the change. Ask the day after and you will get "inconclusive", correctly.

Why "revenue went up after" is not an answer

Revenue moves every week. The question is never "did it change" — it is "did it change more than it would have anyway". Answering that needs a counterfactual: what would have happened if you had done nothing.

You cannot observe that. You can estimate it, and that is what this does. It fits a Bayesian interrupted time series to the period before your change, projects that trend forward, and reports the gap between the projection and what actually happened — with a credible interval saying how sure it is.

This matters commercially, not academically. A price rise that coincides with your best season looks like a win in a spreadsheet and is often a wash. The seasonality is in the pre-period trend, so the projection carries it too.

Ask it

curl -X POST https://app.signal.geysera.com/agent-api/signal/copilot/ask \
  -H "Authorization: Bearer $SIGNAL_COPILOT_KEY" \
  -H "Content-Type: application/json" \
  -d '{"question": "Did the price change on 2026-07-14 move AOV?"}'

Name the date. If you leave it out, the largest recorded price change in the window is used, which is a reasonable guess and not the same as your intent — if you shipped a landing page the same week you changed prices, say which one you mean.

Four metrics can be tested: aov, revenue, orders, units_per_order.

Choosing between them is most of the skill. A price rise that lifts aov while orders falls may be flat on revenue, and all three are true at once. Ask about the one your decision actually turns on, then ask about its counterweight.

Read the answer

The assistant narrates it, and trace carries the numbers. Three shapes come back.

A clear effect. An interval that does not span zero. "AOV was about $6.40 higher than the projection, with a 94% credible interval of $2.10 to $10.70." That is a real finding: do it again, or undo it.

No detectable effect. An interval comfortably around zero. Also a real finding, and the one people skip past. It means the change did not move this metric enough to see — which is exactly what you want to know before rolling it out further.

Inconclusive. Not enough data either side, too much variance, or the window does not contain a usable pre-period. This is a valid answer and the system will give it rather than guess. Treat it as "ask again in two weeks", not as a failure.

The honest read of an inconclusive result is that you do not know. It is worth more than a confident number that was never supportable.

Why you cannot get this from the other tools

Ask "did X drive Y" of anything else and it will be refused. That refusal is deliberate and enforced in code rather than by instruction: the assistant's safety layer checks that a causal claim is backed by this analysis actually having run, and blocks the answer if it was not.

The reason is that every other tool measures association. Revenue was up, traffic was up, the campaign was live — all true, none of them evidence that one caused another. A tool that let a model narrate association as causation would produce confident, plausible, wrong answers about the decisions that matter most.

So a causal question is either answered by this or not answered.

Use it on a schedule, not just in an argument

The highest-value pattern is a standing job rather than a one-off:

  • After every price change. Run it fourteen days later, on aov and then orders. Two questions tell you whether you took margin or lost volume.
  • After every landing-page rewrite. Test orders. A page that converts better should show up there; if it does not, the prettier page was prettier.
  • Before repeating last quarter's campaign. The question is not "did we make money", it is "did the campaign make the money".

Wire it into the loop from Give Signal to any LLM as tools and your model can run it unprompted whenever it notices a dated change, then tell you what it found.

What it will not do

  • It will not tell you why a change worked, only whether the numbers moved beyond the projection.
  • It cannot separate two changes made on the same day. If you raised prices and launched a campaign together, they are one treatment and the answer is about both.
  • It needs a dated, discrete change. A gradual shift over two months has no treatment date and cannot be tested this way.
  • lookback_days accepts 60 to 730. More history means a better pre-period model, up to the point where your business stops resembling its past self.

Next

Markdown source: /developers/workflows/prove-a-change-actually-worked.md