The Benjamini-Hochberg (BH) procedure is a method for controlling the False Discovery Rate (FDR) — the expected proportion of false positives among all rejected null hypotheses — when running many statistical tests at once. In choice modelling it is the correct way to handle the multiplicity problem in interaction testing.
Ordeen applies Benjamini-Hochberg inside its interaction tests so the false-discovery rate across every reported attribute pair stays bounded, and reports each interaction with a 95% credible interval alongside the FDR-adjusted significance. See how it fits in the full platform in our methodology.
The multiplicity problem
A main-effects utility model treats each attribute as acting on choice independently. In practice, levels sometimes pull together or push apart beyond what main effects predict. Testing one interaction cell at $\alpha = 0.05$ is sound. Testing fifty attribute-pair cells at $\alpha = 0.05$ guarantees that two to three of them will look “significant” even if nothing is going on. Reporting these naive $p$-values without correction is how spurious interaction findings get into pricing decks.
How Benjamini-Hochberg works
For $m$ hypothesis tests with $p$-values sorted ascending $p_{(1)} \le p_{(2)} \le \ldots \le p_{(m)}$:
Choose the target False Discovery Rate $q$ (commonly $q = 0.05$ or $q = 0.10$).
Find the largest index $i$ for which: $$p_{(i)} \le \frac{i}{m} \cdot q$$
Reject the null hypothesis for all tests $1 \ldots i$.
This controls the FDR at level $q$ regardless of how many of the nulls are true — a property Benjamini and Hochberg proved in 1995.
Why FDR over FWER (Bonferroni)
The alternative — family-wise error rate correction via Bonferroni — divides $\alpha$ by $m$. For 50 tests this means requiring $p < 0.001$ for each, which is often so strict it kills every real finding alongside the false ones. FDR instead accepts a controlled proportion of false positives in exchange for far more statistical power. For exploratory interaction testing, FDR is the appropriate trade-off.
How Ordeen uses it
Ordeen applies BH at $q = 0.05$ across every level-pair test in its reported interaction results. BH controls false discoveries in what you see; it does not decide which interactions enter the model (a data-coverage check does that). Interaction terms estimated inside the Hierarchical Bayes model are judged by their 95% credible intervals instead. Each reported cell is labelled Complement (positive interaction) or Substitute (negative interaction) and described in plain language. Cells that survive the correction are tagged HIGH. Cells that pass the uncorrected test but fail BH are tagged LOW, and the panel states how many of the tests survive correction.
Related terms
Haldane correction · Conjoint analysis · Full methodology