The Benjamini-Hochberg (BH) procedure is a method for controlling the False Discovery Rate (FDR) — the expected proportion of false positives among all rejected null hypotheses — when running many statistical tests at once. In choice modelling it is the correct way to handle the multiplicity problem in interaction testing.
The multiplicity problem
A main-effects utility model treats each attribute as acting on choice independently. In practice, levels sometimes pull together or push apart beyond what main effects predict. Testing one interaction cell at $\alpha = 0.05$ is sound. Testing fifty attribute-pair cells at $\alpha = 0.05$ guarantees that two to three of them will look “significant” even if nothing is going on. Reporting these naive $p$-values without correction is how spurious interaction findings get into pricing decks.
How Benjamini-Hochberg works
For $m$ hypothesis tests with $p$-values sorted ascending $p_{(1)} \le p_{(2)} \le \ldots \le p_{(m)}$:
Choose the target False Discovery Rate $q$ (commonly $q = 0.05$ or $q = 0.10$).
Find the largest index $i$ for which: $$p_{(i)} \le \frac{i}{m} \cdot q$$
Reject the null hypothesis for all tests $1 \ldots i$.
This controls the FDR at level $q$ regardless of how many of the nulls are true — a property Benjamini and Hochberg proved in 1995.
Why FDR over FWER (Bonferroni)
The alternative — family-wise error rate correction via Bonferroni — divides $\alpha$ by $m$. For 50 tests this means requiring $p < 0.001$ for each, which is often so strict it kills every real finding alongside the false ones. FDR instead accepts a controlled proportion of false positives in exchange for far more statistical power. For exploratory interaction testing, FDR is the appropriate trade-off.
How Ordeen uses it
Every interaction grid applies BH at $q = 0.05$ by default. Surviving pairs are labelled Complement (positive interaction) or Substitute (negative interaction) and reported in plain language. Both raw and BH-corrected $p$-values are exposed so researchers can audit the correction.