2
$\begingroup$

It is known that for the normal distribution, the mean and the standard deviation are independent; it is also the case for samples as sample size tends to infinity.

However, for small samples, the sample mean and the sample standard deviation are not independent. Indeed, the sample mean is used to compute the sample standard deviation.

As an example, simulating 100,000 samples of size 1000, and computing for each means and standard deviations, I get a correlation of 0.0005 between the 2 statistics. For 1,000,000 samples of size 3, I get a negative correlation of -0.0003.

Is there a way to find the expected correlation as a function of sample size? Its distribution? Or is it zero on average?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ In what sense are you attempting to distinguish a "normal distribution" from samples? Can you help us understand what it might mean for two parameters of a distribution, such as its mean and SD, to be "independent"? Could you explain why you think the independence of sample mean and SD depends on sample size? At what size do these statistics become independent? What are the standard errors on the means from your simulations--that is, do you have any significant evidence at all that the correlation differs from zero? $\endgroup$ Commented Nov 11, 2017 at 20:19

1 Answer 1

Reset to default
2
$\begingroup$

While the (usual) formula for sample standard deviation contains the sample mean (so you might say that it depends on the mean algebraically), that doesn't impact their independence in the statistical sense.

The population correlation of the sample mean and the sample standard deviation is zero because the random variables (sample mean, sample standard deviation) are independent.

Indeed $\bar{X}$ and $X_i-\bar{X}$ are independent and the independence of the sample variance (and similarly standard deviation) from $\bar{X}$ follows from that.

Below is a plot of $x_1-\bar{x}$ vs $\bar{x}$ for samples of size 10 from a normal distribution; it's possible to show these are independent (and indeed, jointly normal).

plot of first residual vs mean for normal samples of size 10

The sample correlation between the random variables won't be exactly zero -- it, too, is a random variable. If you gather $m$ samples of size $n$ and compute the mean and sd in each sample of size $n$ and calculate their correlation over the $m$ samples, that will have some deviation from 0 each time, but the population quantity it estimates is 0. Imagine choosing say 50 points in the plot above for example; the sample correlation of those 50 points won't be exactly 0.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Ok, thanks. I should have looked at the standard error. It was indeed of the same magnitude as the estimates I obtained. $\endgroup$ Commented Nov 12, 2017 at 1:47

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.