The Causal Nyquist Rate: How Fast Do You Need to Experiment?

If you've ever wondered "how often should we run experiments?", you've stumbled on one of the most fundamental questions in causal evaluation. The answer isn't arbitrary—it's governed by a deep principle borrowed from signal processing: the Nyquist-Shannon sampling theorem.

In this post, we'll explain the Causal Nyquist Rate—the minimum experimentation frequency needed to track causal effects without aliasing. This is the conceptual foundation that makes Continuous Causal Calibration (CCC) work.

What Is Aliasing?

In signal processing, aliasing occurs when you sample a signal too slowly. High-frequency variations get mistaken for low-frequency patterns, creating false conclusions.

The classic example: a wagon wheel in old Western movies appears to spin backward because the camera samples at 24 fps—too slow to capture the wheel's true rotation speed.

Aliasing in Causal Evaluation

Now imagine you're evaluating an LLM system using an attribution model (surrogate) that drifts over time. The model's bias isn't static—it oscillates monthly due to:

• Seasonal user behavior changes
• Competitor product launches
• Internal model updates
• Data distribution shifts

If you only run experiments quarterly, you're sampling at 0.33 experiments/month. But if the bias drifts at a monthly frequency (1.0 cycle/month), you're sampling too slowly.

The result? Your quarterly experiments will always land at the same phase of the oscillation—making the bias appear stable when it's actually volatile.

Figure 1: Quarterly experiments (red dots) sample a monthly bias drift (purple wave) too slowly. The reconstructed signal (orange dashed line) falsely suggests the bias is stable.

The Nyquist-Shannon Theorem

The Nyquist-Shannon sampling theorem states:

The factor of 2 is called the Nyquist rate. It ensures you capture both the peaks and troughs of the oscillation.

The Causal Nyquist Rate

We can translate this principle directly to causal evaluation:

Where:

• f_exp: How often you run randomized experiments (e.g., 2 per month)
• ν_bias: The highest frequency component of your attribution model's bias drift (e.g., 1 cycle/month)

Concrete Example

Suppose your attribution model's bias oscillates with a monthly period:

• ν_bias = 1 cycle/month (bias completes one full oscillation per month)
• Causal Nyquist Rate = 2 × 1 = 2 experiments/month

This means you need to run experiments at least twice per month (every 2 weeks) to avoid aliasing.

If you only run quarterly experiments (0.33/month), you're sampling below the Nyquist rate—and your bias estimates will be aliased.

The "Sausage" Pattern: Visual Proof of the Nyquist Rate

When you sample at or above the Causal Nyquist Rate, you get a beautiful geometric signature: the Brownian Bridge credible intervals.

These intervals:

• Pinch tight at experimental anchor points (where you have oracle data)
• Balloon out between experiments (where uncertainty grows)

This creates a "sausage" pattern—a visual proof that your experimentation frequency is sufficient to track the bias drift.

Figure 2: The CCC credible intervals (gray shaded region) pinch tight at experimental anchors (red error bars) and balloon between them—the "sausage" pattern proving you're sampling above the Nyquist rate.

Why This Matters

The Causal Nyquist Rate bridges two critical perspectives:

Practical Implications

1. Estimate Your Bias Bandwidth

Before you can set your experimentation frequency, you need to estimate ν_bias. How fast does your attribution model's bias drift?

• Historical experiments: If you've run experiments in the past, look at how bias estimates vary over time
• Domain knowledge: Do you know of seasonal patterns, product cycles, or competitor events?
• Conservative default: If unsure, assume monthly drift (ν_bias = 1/month) and plan for bi-weekly experiments

2. Budget Accordingly

Once you know the Causal Nyquist Rate, you can plan your experimental budget:

3. Diagnose Aliasing

If you're already running experiments, you can check for aliasing:

• Flat bias estimates: If your bias appears constant across experiments, you might be undersampling
• Large uncertainty: If your credible intervals are very wide, you're likely below the Nyquist rate
• Inconsistent results: If adding one more experiment drastically changes your conclusions, you're aliasing

Connection to CCC

The Causal Nyquist Rate is the theoretical foundation for Continuous Causal Calibration (CCC):

• CCC assumes you're sampling at or above the Nyquist rate
• The Brownian Bridge prior in CCC encodes this assumption geometrically
• The "sausage" credible intervals are visual proof that the assumption holds

If you violate the Nyquist rate (experiment too rarely), CCC will still give you answers—but they'll be aliased and misleading.

Summary

Next Steps

Want to see how this works in practice?

References