Incrementality

Overview

The Incrementality report estimates the causal lift of an intervention by comparing observed post-intervention outcomes against a model-based counterfactual learned from pre-intervention behavior.

Use it when you need to answer: “Did this initiative actually create incremental impact, or would the metric have moved anyway?”

What You Get

• Absolute lift: average observed minus counterfactual effect in post period
• Relative lift: effect normalized by counterfactual baseline
• Cumulative lift: total effect across the intervention window
• Effect series: date-by-date observed vs counterfactual vs effect
• Significance and reliability: p-value, confidence weight, and reliability classification
• Data quality warnings: sample size and fit diagnostics for interpretation safety

Method (v1)

Current implementation uses a lightweight causal workflow:

1. Fit a regression counterfactual on the pre-intervention period.
2. Predict post-period counterfactual outcomes using controls (or a time-trend fallback).
3. Compute lift as observed minus counterfactual.
4. Estimate significance using bootstrap + t-approximation safeguards.

Important Note

This v1 model intentionally prioritizes practical robustness and speed.

Future enhancement planned: add a full BSTS (Bayesian Structural Time Series) backend for stronger causal robustness under structural shifts and richer uncertainty modeling.

Data Requirements

• Target and predictor series must be aligned by date and length.
• At least one pre and one post observation are required (more is strongly recommended).
• Intervention values must be non-negative; optional strict binary mode enforces exact 0/1.
• Control metric names must be unique.