How Do Quartiles Help Us Understand a Dataset?

Question

It’s confusing to understand how quartile values can actually be used to give insights into a dataset. Please assist with examples. I struggle to interpret the values in the context of providing insights.

Answer 1

Well, do you think the median is useful? It is a quartile (the middle one). But the median alone only tells you one thing: The point where half are below and half are above. That is useful, but it's just one point.

The first and third quartiles can give you an intuitive sense of skew (although not always). They can also give you an intuitive sense of spread: Half of all the data are in between. That's often useful. And it's more intuitive than the median absolute deviation (which is another good measure of spread for skewed distributions).

Of course, deciles give you even more info - but at the cost of having more numbers to look at.

Answer 2

Begin by thinking about what any quantile is - how it's defined. A quartile is a particular kind of quantile, as is a decile or a percentile.

In a population distribution, one way to define a $p$-th quantile is as the lowest value that has at least a proportion $p$ of the population values at or below it. For example, the median is a 0.5-quantile, because at least half the population is at or below it. The first and third quartiles (0.25 and 0.75 quantiles) are definied similarly; they have 1/4 and 3/4 of the population at or below them.

In a continuous distribution you'll get exact fractions below the respective quantiles - that is, exactly a fraction $p$ of the distribution is to the left of the $p$-th quantile - and so in that case the three quartiles would divide the distribution into four equal fractions of one quarter each.

Multiple sample definitions of quantiles exist (check your text for whatever definition it uses) but they are very similar to the typical population definitions.

If the variable is continuous (or discrete but no one value has much of the total probability in it) then about a quarter of the values would fall into the four regions delimnited by the three quartiles.

If you regard a median as in some sense a 'typical' value, in that it generally divides the collection of values into two near equal-sized groups (as long as not many values are exactly at the median), the lower quartile is similarly a 'typical' value in that lower half and the upper quartile is a 'typical' value in the upper half.

They can give a rough sense of 'where' the values are, keeping in mind the limited extent to which a few specific values could summarize a whole distribution - some sense of location and spread.

If the quartiles of adult male height in cm are 170, 175 and 180 respectively, then about a quarter of adult males will be 170 cm or below, about a half at 175 cm or below and about three quarters at 180 cm or below, with the rest being taller than that. About half the population would be above 170 cm but not more than 180 cm. This tells you something about where the values typically tend to be. It doesn't come near to telling you everything, naturally. You don't know how large or small the more extreme values might be and you don't know how the values in between might be distributed.

You can get a little information about shape from the three quartile values (the quartile skewness and Bowley skewness are defined in terms of the relative distance between the outer quartiles and the median).

Mean and standard deviation also tell you something about the location and spread of the distribution, but tell you nothing about shape. If the distribution is very skewed the mean might not seem very 'typical' (e.g. it may be larger than nearly all the values if the distribution is sufficiently right skewed). Similarly, if the distribution is quite skewed, the standard deviation might not seem much like a very typical distance of values from some 'central' value, nor seem much related to your sense of typical distance between pairs of values. They are still a kind of typical value and typical distance respectively, but any such summary of a large set of values down to only a few numbers isn't ideal for every use and every circumstance.

Answer 3

I think you can find a good example here: https://mlwithouttears.com/2024/01/17/conformalized-quantile-regression/comment-page-1/

Short version: estimating a quantile is a (conditional) point estimating process. It can be useful to understand potential ranges of outcomes (if you estimate e.g. an upper and lower quantile). In the end if you knew all quantiles you could specify the probability (density) of the outcomes.

In general, one quantile alone (a point estimate) is as useful as a point estimate of the mean value (which you obtain from mean squared error minimization)...

Answer 4

Well, if you struggle with interpreting quartiles (and quantiles for that matter), do not feel too bad. To be honest, I do find them pretty uninformative, and pretty much never compute/look at them.
The median (50% percentile) may be somehow of interest, rarely by itself, but sometimes when interpreted together with the mean (it gives a sense of the skewness; but then, I may as well compute the skewness ...).

And wrt boxplots, they are equally uninformative, if not outright misleading; see e.g.

I’ve Stopped Using Box Plots. Should You? by Nick Desbarats,

Be wary of boxplots, they could be hiding important information! (Reddit), or

Same Stats, Different Graphs. Generating Datasets with Varied Appearance and Identical Statistics through Simulated Annealing.

That last link, towards the end of the post, has graphs for distributions which all have exactly the same box plots, but which are widely different, like below.

And if they have the same boxplots, they have the same quartile.

My go-to plot for exploratory analysis is the "individual value plot", something like this

which you can then decorate with symbols for mean, CI of the mean, and possibly median.

Bottom line: I would not sweat it ...

Answer 5

In order to understand quartiles , you should look at how it is used.

(1) When thinking about quartiles , we must also consider the inter-quartile range $IQR=Q_3-Q_1$ , where the quartiles are $Q_1$ & $Q_3$.

Now , you can use IQR to calculate two "fences" for the given Data :
$l=Q_1-1.5IQR$
$u=Q_3+1.5IQR$

Data Points outside the range $(l,u)$ are generally Outliers : Outlier Detection using this method with this range is more robust , compared to usage of mean & median.

(2) We can turn the coin the other way , to claim that inter-quartile range gives the good range of valid Data Points , to measure spread / scatter / dispersion / variability of the Data Points. This method is more robust compared to mean & median , because extreme values do not enter the calculations & skewness might not impact $IQR$.

(3) We can then focus our attention to the inter-quartile range $(Q_1,Q_3)$ , in cases like advertisement targeting , product marketing , movie audiences , getting more "bang-for-the-buck" in general. We might later move beyond that range to target wider Population.

(4) We always want to "Summarize" a large amount of Data to a few numbers , hence we calculate Summary Statistics where quartiles & inter-quartile-range are well-known & widely used.

Made Up Example : rather than giving a list of $5,000,000$ numbers (ages of government employees in a large country) , we can more easily say $50\%$ of the ages are in the range $(21,57)$
That is more useful than just saying that the mean age is $36$ or the median age is $40$