3
$\begingroup$

I am conducting multigroup analysis in STATA with 990 participants to determine whether a proposed model differs across two groups (men and women). The model looks at the effect of variable A --> variable B --> variable C --> variable D. Given theoretical support, I have also included pathways from A --> C, A -->D, and B--> D. Additionally, all variables are continuous, the model controls for two covariates at each path (age and socioeconomic status), and the data uses complex sampling weights.

Variable C is comprised of seven items that were measured on different scales (three 5-point and four 6-point scales), so it has to be standardized. When examining the model across all participants, however, standardizing variable C changes the significance of relationship between variables C and D. Specifically, using the summed (unstandardized) seven items for variable C gave a p-value of .036, and using the standardized variable C (after z-scoring and summing all seven items) gives a p-value of .057.

Although a Google search for this question provided mixed suggestions, I came across a recommendation that all of the variables in a model like this should be standardized. My data shows that when both variables C and D are standardized, there is a p-value of .034 (which is closer to what I found after using the summed, unstandardized variable C).

What is the best approach in this situation? Is it best practice to standardize all of the coefficients in the model? Should I only standardize variable C and report p=.057? Thanks in advance for your help!

$\endgroup$

1 Answer 1

0
$\begingroup$

You've got different data, and that means that you have different results. That's to be expected.

Your p-value is changing, but how much are the estimates changing? Your p-value is changing by a small amount, but it crosses the magical 0.05, so it feels like a large amount. I often point people to The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant.

I would think of this as a robustness / sensitivity check. If a small change to the way you analyze your data changes from statistically significant to not statistically significant, the results are not robust.

$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.