4
$\begingroup$

I am investigating associations between a single gene (independent variable) and the proteome in a study with approximately 3,000 proteins (dependent variables) across three distinct subgroups (e.g., younger, middle-aged, and older adults). To control the False Discovery Rate (FDR) at 5%, how should the Benjamini-Hochberg procedure be applied?

Should FDR correction be performed separately within each subgroup (~3,000 hypothesis tests per group)? Or should it be applied across all subgroups combined (~9,000 hypothesis tests in total)?

I would appreciate explanations of the theoretical considerations behind these approaches, as well as any relevant references.

Thanks in advance!

$\endgroup$
3
  • $\begingroup$ I don't think there are any strong theoretical reasons from statistics. There might be some from the subject area side. What do people in your field usually do? $\endgroup$ Commented Feb 5, 2025 at 11:57
  • $\begingroup$ It's not clear to me what hypothesis is being tested here. Are you looking for changes in protein levels between control and some type of treated individuals within each age group? Are you looking for differences in protein levels among the age groups? Or something else? The best answer might depend on just what those "9,000 hypothesis tests" are testing. $\endgroup$ Commented Feb 5, 2025 at 16:39
  • $\begingroup$ Thanks, I've updated the question. Perhaps it's most important to consider the follow-up analyses I have planned. $\endgroup$ Commented Feb 5, 2025 at 17:48

1 Answer 1

2
$\begingroup$

The choice mostly has to do with how many false leads that you want to risk tracking down in your follow-up studies and how much you want to investigate the age-associated differences in the gene-protein associations.

In this type of study it's not completely clear to me (as a biologist rather than a statistician) how well the assumptions underlying the Benjamini-Hochberg procedure to control false discovery rate (FDR) hold. Biologically, at least, the changes among levels of 3000 proteins as a function of your gene of interest are unlikely to be independent. I tend to think of FDR values as being a guide to future study rather than a definitive statement of which differences are "significant."

There's also no magic in a 5% FDR (or, for that matter, in the 5% family-wise error rate used so widely as a measure of "statistical significance"). If you are willing to risk a bit of extra work in follow-up studies, a 10% or 20% FDR could be defensible and could lead to keeping more true positives.

For publication, so long as you are clear about how you proceeded your choice shouldn't matter. If it does matter to a reviewer or editor, then you can always revise.

It sounds like you are planning to do separate analyses for each age group. I'd recommend instead evaluating the interaction between the gene and age in a single analysis. That will more directly tell you which proteins are differentially changed depending on age. If you have actual ages, you might consider moving away from binning by age and instead model age flexibly and continuously with splines. In the age-grouped data that single analysis will end up with 9000 tests, but you can always choose a different FDR to compensate with respect to designing follow-up studies.

$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.