Lesson 9 of 25 · 7 min

What a correlation matrix is, and what it takes to estimate one

Twenty-three holdings need 253 pairwise numbers, and five years of monthly data supplies five observations for each.

A correlation matrix looks like a table and behaves like a budget. Every entry has to be paid for in observations, and the number of entries grows with the square of the holdings count while the data grows only with time.

The matrix is the grid from lesson 3 with the weights and volatilities stripped out: one row and one column per holding, each cell holding the correlation between that row's holding and that column's. The diagonal is all ones, since every holding is perfectly correlated with itself, and the cell above the diagonal equals the cell below it. So the distinct information is only the upper triangle.

distinct entries = n x (n - 1) / 2

distinct entries: How many separate correlations must be estimated. Each one is a parameter, and each parameter costs data.
n: The number of holdings in the matrix.

Eight holdings need 28 entries. Twenty-three need 253. Sixty need 1,770. Now price the twenty-three holding case in data. Add the 23 variances and you are estimating 276 parameters. Five years of monthly returns on 23 holdings is 23 times 60 = 1,380 observations, so 1,380 divided by 276 = 5.0 observations per parameter. Reaching ten per parameter would take 120 months of history, which is ten years, over which the holdings themselves will not have kept the same behaviour.

The consequence is not that such a matrix is wrong in an obvious way. It is that the matrix is noisy, and estimation noise does not look like noise — it looks like structure. A matrix with barely five observations per entry will show some pairs at 0.62 and others at 0.11 partly because that is true and partly because that is what small samples do. Six holdings measured over fifty ordinary months in an illustrative history used later in this track gave fifteen pairwise entries spread across 0.20 to 0.35, and the spread was almost entirely sampling. This is why practitioners collapse a matrix into a few common factors, which lesson 19 takes up, or into groups, which lesson 24 does.

Key terms

Correlation matrix: A square table holding the correlation between every pair of holdings. Ones down the diagonal, and symmetric about it.
Parameter count: How many separate numbers a model requires you to estimate. For a correlation matrix it grows with the square of the holdings count.
Estimation noise: The part of a measured figure that comes from the sample being finite rather than from the thing being measured. It is indistinguishable from real structure without more data.

Worked example

The data budget for one matrix

A pledged portfolio of twenty-three holdings, with five years of monthly return history available. Illustrative figures.

Distinct correlations to estimate: 253
Plus one variance per holding: 276 parameters
Observations available: 1,380
Observations per parameter: 5.0
History needed for ten per parameter: 120 months
For comparison, an eight-holding matrix: 28 correlations

Five observations per parameter is the reason a matrix can be fully populated and still mostly noise. The table has no way of showing which entries it is confident about.

Check yourself

A fifteen-holding portfolio has three years of monthly data. How many parameters does its correlation matrix and variance set require, and how many observations per parameter does that give?

Saved in this browser only. No account, nothing sent anywhere.

Educational only. Nothing in this lesson is investment, legal or tax advice, or a recommendation to enter into any transaction. Figures are illustrative and computed from the inputs stated in the lesson.