Portfolio Optimization

The last post gave us alpha: the gap between an asset’s expected return and what its risk exposure warrants. As a pricing measure, alpha marks an asset as cheap or expensive relative to its systematic risk. It does not tell us how far a portfolio should tilt toward an asset whose alpha is positive.

Answering that takes more machinery than we have. The variance formula we have so far covers two assets, and real portfolios hold hundreds of securities, more than that formula can handle by hand. This post scales the risk arithmetic to $N$ assets, states the optimization problem built on it, and then puts alpha to work choosing weights.

The Covariance Matrix

For two assets A and B held at weights $w_{A}$ and $w_{B}$ , post 8 gave the portfolio variance as:

σ_{p}^{2} = (w_{A} σ_{A})^{2} + (w_{B} σ_{B})^{2} + 2 w_{A} w_{B} ρ_{A B} σ_{A} σ_{B}

That expression has 3 unique terms: two variances and one covariance. With 3 assets we get 6, with 10 we get 55, and with 500 we get 125,250. The count grows as $N (N + 1) /2$ .

For $N$ assets, the general form is a double sum over every pair:

σ_{p}^{2} = i = 1 \sum N j = 1 \sum N w_{i} w_{j} Cov (r_{i}, r_{j})

When $i = j$ , $Cov (r_{i}, r_{i}) = Var (r_{i}) = σ_{i}^{2}$ and the term is asset $i$ ’s own variance. When $i \neq = j$ , the term is the cross-covariance between assets $i$ and $j$ .

Matrix notation compresses the whole double sum into three symbols:

σ_{p}^{2} = w^{⊤} Σ w

where $w$ is the $N \times 1$ weight vector and $Σ$ is the $N \times N$ covariance matrix. The diagonal entries are variances ( $σ_{i}^{2}$ ) and the off-diagonal entries are covariances ( $Cov (r_{i}, r_{j})$ ). The matrix is symmetric, since $Cov (r_{i}, r_{j}) = Cov (r_{j}, r_{i})$ .

We write the product out for $N = 2$ and recover the two-asset formula:

[w_{A} w_{B}] [σ_{A}^{2} Cov (A, B) Cov (A, B) σ_{B}^{2}] [w_{A} w_{B}]

Multiplying through gives $w_{A}^{2} σ_{A}^{2} + 2 w_{A} w_{B} Cov (A, B) + w_{B}^{2} σ_{B}^{2}$ , and two substitutions recover the form above. The first is cosmetic, since $(w_{A} σ_{A})^{2}$ is $w_{A}^{2} σ_{A}^{2}$ . For the second we borrow post 8’s definition of correlation, covariance divided by both standard deviations, and rearrange it into $Cov (A, B) = ρ_{A B} σ_{A} σ_{B}$ to replace the cross-term.

For 3 assets with $σ_{1} = 15%$ , $σ_{2} = 22%$ , $σ_{3} = 18%$ and correlations $ρ_{12} = 0.3$ , $ρ_{13} = 0.5$ , $ρ_{23} = 0.1$ , each diagonal entry is $σ_{i}^{2}$ and each off-diagonal entry is $ρ_{ij} σ_{i} σ_{j}$ . In percent squared, the covariance matrix is:

Σ = 22599135 99484 39.6 135 39.6 324

The first diagonal entry is $1 5^{2} = 225$ , and the entry linking the first two assets is $0.3 \times 15 \times 22 = 99$ . The matrix has three variance terms on the diagonal and three unique covariance terms off it. At $N = 3$ we can still write the matrix out by hand. At $N = 500$ we cannot, which is what the notation is for.

Drag the slider below to add assets to the matrix and watch the term counts move.

Assets (N) = 5 1122334455 Variance (diagonal) Covariance (off-diagonal) Assets 5 Variance terms 5 Unique cov terms 10 Total unique 15 % Covariance 66.7% In a large portfolio, covariances vastly outnumber variances. Portfolio risk is about how assets move together.

The highlighted cells along the diagonal are variance terms and the rest are covariances, with the faded cells above the diagonal repeating the ones below.

Why Covariances Dominate

For $N$ assets, the covariance matrix has $N$ variance terms (the diagonal) and $N (N - 1) /2$ unique covariance terms (the lower triangle). The ratio of covariance terms to variance terms is $(N - 1) /2$ to $1$ , and the two counts diverge as $N$ grows:

Assets	Variance terms	Covariance terms	Total unique
2	2	1	3
10	10	45	55
100	100	4,950	5,050
500	500	124,750	125,250

At $N = 500$ , covariance terms outnumber variance terms 249.5 to 1. Portfolio risk is overwhelmingly determined by how assets move together, not by how volatile they are individually. This is the algebraic version of the diversification insight we reached in post 8: individual risk washes out in a large portfolio, but correlated movements survive. When we add a new security to a portfolio, its own variance matters less than how it covaries with everything already held.

Mean-Variance Optimization

An investor wants the highest expected return available at a chosen level of risk, or equivalently the lowest risk at a target expected return. Mean-variance optimization states that goal as a constrained minimization:

w min w^{⊤} Σ w

subject to:

w^{⊤} μ = μ_{target}, w^{⊤} 1 = 1

where $μ$ is the $N \times 1$ vector of expected excess returns (each asset’s $E [r_{i}] - r_{f}$ ), $μ_{target}$ is the target portfolio excess return, and $1$ is a vector of ones enforcing that the weights sum to one.

The objective is to make portfolio variance as small as the constraints allow. The first constraint pins expected return to the target, and the second keeps the portfolio fully invested. We solve it at every possible $μ_{target}$ and trace out the efficient frontier from post 9, now for $N$ assets instead of two.

With a risk-free asset available, the problem simplifies to a search for the highest Sharpe ratio:

w max \frac{w ^{⊤} μ}{w ^{⊤} Σ w}

The solution is the tangent portfolio. For an investor with risk aversion $λ$ , the optimal risky portfolio is:

w^{*} = \frac{1}{λ} Σ^{- 1} μ

The optimal weights are the inverse covariance matrix times the vector of expected excess returns, scaled by $1/ λ$ . The covariance matrix acts as a penalty for redundant risk, down-weighting assets that covary heavily with the rest of the portfolio, while assets with high expected returns get more weight. The risk aversion parameter $λ$ scales the whole position, so a less risk-averse investor ( $λ$ small) takes larger positions and a more risk-averse investor ( $λ$ large) takes smaller ones.

For the tangent portfolio itself, $λ$ does not matter. Every value of $λ$ produces the same portfolio once we normalize the weights to sum to one, only at a different scale. This is Two Fund Separation again: $λ$ decides how much of the tangent portfolio an investor holds versus the risk-free asset, and the tangent portfolio is the same for everyone. Normalizing gives us the fully invested tangent weights directly, with no risk-free position:

w_{t an} = \frac{Σ ^{- 1} μ}{1 ^{⊤} Σ ^{- 1} μ}

The denominator $1^{⊤} Σ^{- 1} μ$ is a scalar, the sum of the unnormalized weights. Dividing by it rescales the vector so that $1^{⊤} w_{t an} = 1$ .

This is Markowitz’s result in its general form. The efficient frontier, Two Fund Separation, and the tangent portfolio all fall out of this one optimization.

The hard part in practice is not the math. It is estimating $μ$ and $Σ$ reliably. Small errors in expected returns produce large swings in the optimal weights, so unconstrained mean-variance optimization is notoriously sensitive to its inputs. The unconstrained solution can also produce negative weights (short positions), which many investors cannot take. Adding a no-shorting constraint ( $w_{i} \geq 0$ ) breaks the closed-form solution and calls for numerical optimization, typically quadratic programming.

Alpha Relative to Any Portfolio

In post 10 we measured alpha against the market, under the CAPM, or against a factor model. The definition generalizes to any reference portfolio. For asset $i$ measured against a portfolio $P$ :

α_{i} = E [r_{i}] - r_{f} - β_{i, P} (E [r_{P}] - r_{f})

where $β_{i, P} = Cov (r_{i}, r_{P}) / Var (r_{P})$ . The formula is post 10’s, with only the reference portfolio changed. CAPM alpha uses the market, and nothing stops us from using the portfolio we already hold, or any other benchmark. The question shifts from “does this asset beat the market?” to “does this asset offer more return than its risk contribution to my portfolio warrants?”

Alpha as a Gradient

The marginal improvement to a portfolio’s Sharpe ratio from adding a small amount of asset $i$ is proportional to that asset’s alpha:

\frac{\partial S R _{P}}{\partial w _{i}}_{w_{i} = 0} = \frac{α _{i}}{σ _{P}}

In words: the rate at which the Sharpe ratio improves as we begin tilting toward asset $i$ equals the asset’s alpha, measured against the portfolio held now, divided by that portfolio’s standard deviation.

Alpha is excess return after adjusting for risk, so an asset offering more return than its risk contribution warrants improves the risk-return tradeoff when we fold it in. A larger alpha makes the improvement steeper. A more volatile starting portfolio makes it shallower, since $σ_{P}$ is in the denominator.

That result reframes alpha from a scorecard for past performance into a direction. Positive alpha points toward a better portfolio, negative alpha points away from one, and zero alpha says the allocation to that asset is already right.

Suppose we hold a portfolio with a Sharpe ratio of 0.53 ( $E [r_{P}] = 10%$ , $σ_{P} = 15%$ , $r_{f} = 2%$ ), and an asset outside it offers $E [r] = 14%$ , $σ = 25%$ , and correlation $ρ = 0.6$ with the portfolio. We compute the asset’s beta relative to the portfolio, $β = ρ \times σ_{a sse t} / σ_{P} = 0.6 \times 25/15 = 1.0$ , and its alpha, $14 - 2 - 1.0 \times (10 - 2) = 4%$ . The gradient is $4/15 \approx 0.27$ , so each percentage point of tilt initially raises the Sharpe ratio by about 0.0027.

Tilting Toward Alpha

Suppose asset $i$ has $α > 0$ against the portfolio we hold. Tilting toward it raises the Sharpe ratio, and two things change as we tilt. The portfolio itself changes, since it now holds some of asset $i$ . The asset’s beta against the new blended portfolio changes with it, because the composition has shifted. The alpha shrinks.

At some tilt the alpha of the asset against the blended portfolio reaches zero, and no further tilt improves anything. Past that point the alpha turns negative and the Sharpe ratio declines with it.

That endpoint gives us an optimality condition. The maximum-Sharpe portfolio is the one where every asset has zero alpha relative to it, its return matching what its risk contribution warrants.

Drag the tilt slider below and watch the Sharpe ratio rise, peak, and fall.

Portfolio E[r] = 10.0% Portfolio σ = 15.0% r_f = 2.0% Asset E[r] = 14.0% Asset σ = 25% Correlation = 0.60 Tilt weight = 0% 0.50 0.55 0.60 0% 10% 20% 30% 40% 50% Tilt weight optimal Sharpe ratio Optimal tilt Current SR 0.53 Alpha (at tilt) 4.00% Optimal tilt 28.0% Max SR 0.57 Positive alpha means a tilt toward the asset raises the portfolio's Sharpe ratio. At the optimal tilt, alpha is zero.

At the default settings the asset has positive alpha against the portfolio, and the dashed line marks the tilt where the Sharpe ratio peaks. Raise the asset’s expected return and the curve lifts at every positive tilt.

CAPM as an Optimality Condition

The CAPM entered in post 10 as a pricing model. Read in the language of this post, it is a statement about optimality. The CAPM assumes the market portfolio has the highest Sharpe ratio among all portfolios of risky assets, and if that holds, every asset must have zero alpha relative to it. The CAPM equation

E [r_{i}] = r_{f} + β_{i} (E [r_{m}] - r_{f})

is the condition $α_{i} = 0$ for every asset $i$ , with the market portfolio as the reference. Beyond telling us what returns should be, the model says the market portfolio is mean-variance optimal. No tilt away from market weights can improve the Sharpe ratio.

Pricing and portfolio construction are the same statement read in two directions. Grant that the market portfolio is optimal and we get CAPM pricing for free. Claim instead that some asset is mispriced and has positive alpha, and we have implicitly said the market portfolio is suboptimal. The alpha-tilt result then shows how to improve on it.

What’s Next

We now have a complete toolkit for building portfolios and improving them. We have not yet asked whether any of this analysis yields an edge. If the market already reflects all available information, alpha should be zero everywhere and the tangent portfolio should be the index. The final post takes up the efficient market hypothesis and what it means for everything we’ve built.