How do you run a geo holdout test for CTV?

Why geography

Geo tests switch advertising on or off by region rather than by person. That makes them practical for TV, where individual-level tracking is limited, and it lets you measure outcomes in your own data: sales by region, sign-ups by postcode, store visits by market.

Step by step

1. Write the hypothesis and the metric. For example: "CTV will increase online orders by at least 5% in test regions over six weeks." Choose one primary metric that you record by region.
2. Pick the regions. Use designated market areas (DMAs), states or postcode groups. You need enough regions on each side to average out local noise.
3. Match test and control. Pair regions with similar historical trends in your metric, population and existing marketing. Many teams use a few months of history to check that matched pairs moved together before the test.
4. Freeze other changes. Keep search, social and promotions as steady as possible across both groups. If something must change, change it everywhere.
5. Set the window. Run long enough for delayed effects: TV often drives searches and direct visits days after exposure. Four to eight weeks is common, plus a short period afterwards to catch late effects.
6. Launch in test regions only. Configure the CTV platform's geo targeting so the control regions receive no campaign impressions.
7. Read the result. Compare the change in the metric in test regions against the change in control regions over the same period. The gap is the estimated lift. Divide incremental revenue by media spend for incremental ROAS (iROAS).
8. Decide in advance what you will do. Set a threshold for scaling, adjusting or stopping before you see the data.

Common mistakes

• Too few regions. One test city against one control city is easily distorted by a local event.
• Leaking impressions. If the platform's geo targeting is loose, some control households see ads and the measured lift shrinks.
• Changing creative mid-test. A new video halfway through mixes two treatments into one result.
• Reading too early. Week one often looks flat because of delayed response.

Where platforms help

Some platforms build geo tests into the product. Vibe's MCP page gives geo holdout examples among the prompts an AI assistant can run, and its integrations page names Haus, a measurement company focused on incrementality, among its partners. Other platforms describe lift analysis or brand and sales lift studies. Whatever the platform, keep your own copy of the regional data so you can check the result.

When a geo test is not the right tool

If your sales are concentrated in a few regions, or your budget cannot cover enough markets, a user-level holdout or a modelled approach may fit better. Ask the platform which designs it supports and what share of budget the control group needs.