ORDEEN · Choice Modelling · 3 MIN READ

What is the Benjamini-Hochberg Procedure? FDR Control Explained

Benjamini-Hochberg controls the False Discovery Rate when running many statistical tests — essential for honest interaction testing.

Book a 20-min demo What is the Benjamini-Hochberg Procedure? FDR Control Explained

The Benjamini-Hochberg (BH) procedure is a method for controlling the False Discovery Rate (FDR) — the expected proportion of false positives among all rejected null hypotheses — when running many statistical tests at once. In choice modelling it is the correct way to handle the multiplicity problem in interaction testing.

Ordeen applies Benjamini-Hochberg inside its interaction tests so the false-discovery rate across every reported attribute pair stays bounded, and reports each interaction with a 95% credible interval alongside the FDR-adjusted significance. See how it fits in the full platform in our methodology.

The multiplicity problem

A main-effects utility model treats each attribute as acting on choice independently. In practice, levels sometimes pull together or push apart beyond what main effects predict. Testing one interaction cell at $\alpha = 0.05$ is sound. Testing fifty attribute-pair cells at $\alpha = 0.05$ guarantees that two to three of them will look “significant” even if nothing is going on. Reporting these naive $p$-values without correction is how spurious interaction findings get into pricing decks.

How Benjamini-Hochberg works

For $m$ hypothesis tests with $p$-values sorted ascending $p_{(1)} \le p_{(2)} \le \ldots \le p_{(m)}$:

  1. Choose the target False Discovery Rate $q$ (commonly $q = 0.05$ or $q = 0.10$).

  2. Find the largest index $i$ for which: $$p_{(i)} \le \frac{i}{m} \cdot q$$

  3. Reject the null hypothesis for all tests $1 \ldots i$.

This controls the FDR at level $q$ regardless of how many of the nulls are true — a property Benjamini and Hochberg proved in 1995.

Why FDR over FWER (Bonferroni)

The alternative — family-wise error rate correction via Bonferroni — divides $\alpha$ by $m$. For 50 tests this means requiring $p < 0.001$ for each, which is often so strict it kills every real finding alongside the false ones. FDR instead accepts a controlled proportion of false positives in exchange for far more statistical power. For exploratory interaction testing, FDR is the appropriate trade-off.

How Ordeen uses it

Ordeen applies BH at $q = 0.05$ across every level-pair test in its reported interaction results. BH controls false discoveries in what you see; it does not decide which interactions enter the model (a data-coverage check does that). Interaction terms estimated inside the Hierarchical Bayes model are judged by their 95% credible intervals instead. Each reported cell is labelled Complement (positive interaction) or Substitute (negative interaction) and described in plain language. Cells that survive the correction are tagged HIGH. Cells that pass the uncorrected test but fail BH are tagged LOW, and the panel states how many of the tests survive correction.

Related terms

Haldane correction · Conjoint analysis · Full methodology

The Benjamini-Hochberg (BH) procedure is a method for controlling the False Discovery Rate (FDR) — the expected proportion of false positives among all rejected null hypotheses — when running many statistical tests at once. In choice modelling it is the correct way to handle the multiplicity problem in interaction testing.

Ordeen applies Benjamini-Hochberg inside its interaction tests so the false-discovery rate across every reported attribute pair stays bounded, and reports each interaction with a 95% credible interval alongside the FDR-adjusted significance. See how it fits in the full platform in our methodology.

The multiplicity problem

A main-effects utility model treats each attribute as acting on choice independently. In practice, levels sometimes pull together or push apart beyond what main effects predict. Testing one interaction cell at $\alpha = 0.05$ is sound. Testing fifty attribute-pair cells at $\alpha = 0.05$ guarantees that two to three of them will look “significant” even if nothing is going on. Reporting these naive $p$-values without correction is how spurious interaction findings get into pricing decks.

How Benjamini-Hochberg works

For $m$ hypothesis tests with $p$-values sorted ascending $p_{(1)} \le p_{(2)} \le \ldots \le p_{(m)}$:

  1. Choose the target False Discovery Rate $q$ (commonly $q = 0.05$ or $q = 0.10$).

  2. Find the largest index $i$ for which: $$p_{(i)} \le \frac{i}{m} \cdot q$$

  3. Reject the null hypothesis for all tests $1 \ldots i$.

This controls the FDR at level $q$ regardless of how many of the nulls are true — a property Benjamini and Hochberg proved in 1995.

Why FDR over FWER (Bonferroni)

The alternative — family-wise error rate correction via Bonferroni — divides $\alpha$ by $m$. For 50 tests this means requiring $p < 0.001$ for each, which is often so strict it kills every real finding alongside the false ones. FDR instead accepts a controlled proportion of false positives in exchange for far more statistical power. For exploratory interaction testing, FDR is the appropriate trade-off.

How Ordeen uses it

Ordeen applies BH at $q = 0.05$ across every level-pair test in its reported interaction results. BH controls false discoveries in what you see; it does not decide which interactions enter the model (a data-coverage check does that). Interaction terms estimated inside the Hierarchical Bayes model are judged by their 95% credible intervals instead. Each reported cell is labelled Complement (positive interaction) or Substitute (negative interaction) and described in plain language. Cells that survive the correction are tagged HIGH. Cells that pass the uncorrected test but fail BH are tagged LOW, and the panel states how many of the tests survive correction.

Related terms

Haldane correction · Conjoint analysis · Full methodology

Related guides

Contract Analysis

Jurisdiction-Grounded AI Contract Review & Risk

Why generic chatbots hallucinate legal contract risk, and how Ordeen grounds contract review, portfolio risk scoring and renewal drafting in governing law.

Read the guide
Forecasting

Intermittent Demand Forecasting: Croston, TSB & MinT

How to forecast lumpy, sparse SKU demand and reconcile hierarchies: Croston's method, SBA, TSB, MinT reconciliation and out-of-sample AI signal validation.

Read the guide
Price Monitoring

Retail Checkout Price Monitoring vs Sticker Scrapers

Why basic retail price scrapers miss real checkout undercutting: how delivery thresholds, variant SKU matching and screenshot evidence protect brand RRP.

Read the guide