# Prove a change actually worked **The job:** you raised prices, shipped a new landing page, or started a campaign. Revenue moved. Find out whether your change is why. **Who this is for:** anyone who has to defend a decision, or decide whether to do it again. **What it costs you:** one question and about fifteen seconds. --- ## Before you start - You need an **API key with the `copilot` scope** — this analysis is only reachable through the assistant, not the four public `GET`s. Scopes are fixed at creation, so pick it when you mint the key; a `read`-only key gets `403`. - You need a **connected store**, because every metric here comes from orders. - You need **at least 14 days of history either side of the change**. Ask the day after and you will get "inconclusive", correctly. --- ## Why "revenue went up after" is not an answer Revenue moves every week. The question is never "did it change" — it is "did it change *more than it would have anyway*". Answering that needs a counterfactual: what would have happened if you had done nothing. You cannot observe that. You can estimate it, and that is what this does. It fits a Bayesian interrupted time series to the period before your change, projects that trend forward, and reports the gap between the projection and what actually happened — with a credible interval saying how sure it is. This matters commercially, not academically. A price rise that coincides with your best season looks like a win in a spreadsheet and is often a wash. The seasonality is in the pre-period trend, so the projection carries it too. ## Ask it ```bash curl -X POST https://app.signal.geysera.com/agent-api/signal/copilot/ask \ -H "Authorization: Bearer $SIGNAL_COPILOT_KEY" \ -H "Content-Type: application/json" \ -d '{"question": "Did the price change on 2026-07-14 move AOV?"}' ``` Name the date. If you leave it out, the largest recorded price change in the window is used, which is a reasonable guess and not the same as your intent — if you shipped a landing page the same week you changed prices, say which one you mean. Four metrics can be tested: **`aov`**, **`revenue`**, **`orders`**, **`units_per_order`**. Choosing between them is most of the skill. A price rise that lifts `aov` while `orders` falls may be flat on `revenue`, and all three are true at once. Ask about the one your decision actually turns on, then ask about its counterweight. ## Read the answer The assistant narrates it, and `trace` carries the numbers. Three shapes come back. **A clear effect.** An interval that does not span zero. "AOV was about $6.40 higher than the projection, with a 94% credible interval of $2.10 to $10.70." That is a real finding: do it again, or undo it. **No detectable effect.** An interval comfortably around zero. Also a real finding, and the one people skip past. It means the change did not move this metric enough to see — which is exactly what you want to know before rolling it out further. **Inconclusive.** Not enough data either side, too much variance, or the window does not contain a usable pre-period. **This is a valid answer and the system will give it rather than guess.** Treat it as "ask again in two weeks", not as a failure. The honest read of an inconclusive result is that you do not know. It is worth more than a confident number that was never supportable. ## Why you cannot get this from the other tools Ask "did X drive Y" of anything else and it will be refused. That refusal is deliberate and enforced in code rather than by instruction: the assistant's safety layer checks that a causal claim is backed by this analysis actually having run, and blocks the answer if it was not. The reason is that every other tool measures **association**. Revenue was up, traffic was up, the campaign was live — all true, none of them evidence that one caused another. A tool that let a model narrate association as causation would produce confident, plausible, wrong answers about the decisions that matter most. So a causal question is either answered by this or not answered. ## Use it on a schedule, not just in an argument The highest-value pattern is a standing job rather than a one-off: - **After every price change.** Run it fourteen days later, on `aov` and then `orders`. Two questions tell you whether you took margin or lost volume. - **After every landing-page rewrite.** Test `orders`. A page that converts better should show up there; if it does not, the prettier page was prettier. - **Before repeating last quarter's campaign.** The question is not "did we make money", it is "did the campaign make the money". Wire it into the loop from [Give Signal to any LLM as tools](./give-signal-to-any-llm-as-tools.md) and your model can run it unprompted whenever it notices a dated change, then tell you what it found. ## What it will not do - It will not tell you *why* a change worked, only whether the numbers moved beyond the projection. - It cannot separate two changes made on the same day. If you raised prices and launched a campaign together, they are one treatment and the answer is about both. - It needs a dated, discrete change. A gradual shift over two months has no treatment date and cannot be tested this way. - `lookback_days` accepts 60 to 730. More history means a better pre-period model, up to the point where your business stops resembling its past self. --- ## Next - [Give Signal to any LLM as tools](./give-signal-to-any-llm-as-tools.md) — run this automatically from your own agent. - [Find the pages that create pipeline](./find-the-pages-that-create-pipeline.md) — association, honestly labelled, for the questions this cannot answer. - [Why attribution covers less than you think](../case-studies/why-attribution-covers-less-than-you-think.md) — what the other half of the measurement story actually supports.