Questions tagged [multiple-comparisons]
Signals situations where one is concerned about achieving intended power and size when more than one hypothesis test is performed.
1,913 questions
0
votes
0
answers
32
views
Comparing to zero in emmeans pairwise comparisons?
I'm analyzing data on daily foraging dynamics of animals in different treatments feeding on a diet consisting of two different qualities (high and low) using R. The problem arises when there are days ...
- 41
2
votes
1
answer
153
views
Multiple comparisons with uncertain number of tests
In metagenomics one typically collects data representing gene/species counts (or their proportion) in an individual. One then performs comparisons for every gene/species in groups of individuals ...
- 5,091
0
votes
0
answers
44
views
Question regarding Benjamini–Hochberg via Thissen et al. (2002)
I've often seen the formula to calculate the BH critical value as such $ q(k) = k/n\cdot\alpha $ (where $\alpha$ is your chosen FDR, $k$ is the rank of the p-value, and $n$ is the total number of p-...
1
vote
0
answers
15
views
How to apply FDR correction when one survey item is sex-specific
I am analyzing survey data with ~80 items. Most items apply to all respondents, but one item is male-specific and therefore has a smaller denominator (approx. n/2).
I am comparing symptom frequencies ...
- 11
1
vote
0
answers
13
views
Log-linear models and multiple comparisons: exploring multiple categorical and binary variables
I'm trying to understand how three categorical variables affect several binary variables. I am roughly following these instructions. Here is what my data look like (not my real data):
Binary answers ...
0
votes
0
answers
18
views
Temporal autocorrelation when my observations are pairwise comparisons between timesteps?
I have a timeseries of species abundances (continuous variable). I calculated the dissimilarity (Bray Curtis coefficient) between all time steps, and now I want to model this dissimilarities against ...
0
votes
1
answer
22
views
Handling multiple comparison corrections in association rule mining
We’re working on a project applying association rule mining to environmental DNA (eDNA) datasets, and run into some uncertainty regarding how to best handle multiple comparison corrections in this ...
0
votes
0
answers
57
views
Parametric bootstrap for multiple comparisons for glmm
When doing post-hoc treatment comparisons with the results from a glmm it is typical in my industry to use PROC GLIMMIX, method=rspl, ddfm = kr, and whatever control method is appropriate (ex. Tukey, ...
- 1
0
votes
0
answers
32
views
Running adaptive shrinkage on marginal means (conjoint analysis)
I am trying to use adaptive shrinkage (using the ashr R package) to adjust for multiple hypothesis testing in conjoint analysis. My understanding from recent ...
- 65
0
votes
1
answer
143
views
Comparing multiple groups to a reference group
I would like to compare several groups to a reference group, with the main idea being to show that the other groups are not inferior to the reference. Ideally, I would also like to test for ...
- 939
3
votes
2
answers
162
views
Group comparison (and pairwise tests) with non-independent data
I'm trying to determine if there are significant differences between groups and identify which groups differ from each other.
First, let me explain the experimental design:
I have 6 Locations. Inside ...
- 53
10
votes
3
answers
555
views
What qualifies to be a "family of statistical tests" for Bonferroni corrections?
I have looked at posts about multiple hypotheses, but not found what I'm looking for.
I have different sets of statistical analyses in my paper:
Comparisons of valid to excluded data (8 tests; just ...
- 141
3
votes
0
answers
214
views
What is the name for a hypothesis that won't be tested unless other hypotheses are supported
I am writing a research proposal for the OSF (Open Science Framework). I want to make it as clear as possible. I don't know what to call one of my hypotheses (dependent? conditional? something else?).
...
- 141
3
votes
1
answer
71
views
Can I test multiple FDRs when running BH Procedure?
Let's say I'm running an exploratory analysis in which I'm calculating hundreds of correlations between a number of biomarkers and a number of measures of intelligence. I want to use BH Procedure to ...
13
votes
3
answers
2k
views
Why is the Bonferroni correction needed?
Intuitively it makes sense that extracting multiple hypotheses from the same dataset is a bad idea because the probability of a false positive goes up. But when I think more about it, it doesn't make ...
