3
$\begingroup$

I am working on clustering some time series and have decided to try to cluster based on features rather than the raw series (for now). I found this paper by Hyndman that chooses a series of features (serial correlation, non-linearity, skewness, kurtosis, periodicity) and discusses some of the algorithms used for clustering.

At one point he mentions the need to bring the features to [0,1] and he proposes 3 different normalization methods. The first two:

  • $f1$ mapping Q $\in$ [0,$\infty$) to q $\in$ [0,1]: $q = \frac{e^{aQ}-1}{b+e^{aQ}}$

  • $f2$ mapping $\in$ [0,1] to q $\in$ [0,1]: $q = \frac{(e^{aQ}-1)(b+e^{a})}{(b+e^{aQ})(e^a -1)}$

For $f1$ and $f2$ the constants a and b are chosen "such that $q$ satisfies the conditions: $q$ has 90th percentile of 0.10 when $Y_t$ (the time series) is standard normal white noise, and $q$ has value of 0.9 for a well-known benchmark dataset with the required feature."

For each feature, and depending on whether the original values lie in the [0,$\infty$) or [0,1] range, Hyndman gives the values of $a$ and $b$.

I am having trouble understanding this normalization technique. Why would we want a 90th percentile of 0.1 fo $q$? Are there any reasons why we can't simply use z-normalization? Is this a common and standard procedure?

Any hints, tips, insights, and links for further reading are greatly appreciated. Thanks!

Improve this question
$\endgroup$
3
  • $\begingroup$ 1) z-normalization would not result in target range of [0,1]. 2) q set to the 90th percentile only for a time series of standard white noise - thereby effectively asking that 90% of standard white noise results in an influence of less than 0.1 for q. 3) It is not a common precodure (to my knowledge) $\endgroup$ Commented Nov 18, 2016 at 15:46
  • $\begingroup$ Thanks for the answer Nikolas. I know that z-normalization won't bring the features to [0,1] but it will bring all values to a comparable scale. Are there any reason why we would want all values to be positive? For 2) I think I understand better now, thanks a lot. $\endgroup$ Commented Nov 18, 2016 at 15:57
  • $\begingroup$ z-Transformation does not result in a limited scale at all. Any values are possible yet unprobable. For possible processing it can be valuable to have all data only within a certain limit (knowing both max and min). Therefore z-normalization is not done here. $\endgroup$ Commented Nov 18, 2016 at 15:59

1 Answer 1

Reset to default
2
$\begingroup$

1) z-normalization would not result in target range of [0,1]. z-Transformation does not result in a limited scale at all. Any values are possible yet unprobable. For possible processing it can be valuable to have all data only within a certain limit (knowing both max and min). Therefore z-normalization is not done here.

2) q set to the 90th percentile only for a time series of standard white noise - thereby effectively asking that 90% of standard white noise results in an influence of less than 0.1 for q.

3) It is not a common precodure (to my knowledge)

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.