Optimization of two random normals
Contents
Optimization of two random normals¶
— Kowshik Bettadapura
Problem statement.
Let \(X\) and \(Y\) be two random, normals. That is, two random variables distributed normally. Then for a given level of risk \(r^2\), what combination \(Z_{\lambda, \delta} = \lambda X + \delta Y\) will give the largest expectation?
Preliminaries¶
Independent randoms¶
The normal distribution is an instance of the class of stable distributions. This means any linear combination of random, normal variables will again be normally distributed. So if \(X\sim \mathcal N(\mu_X, \sigma^2_X)\) and \(Y\sim \mathcal N(\mu_Y, \sigma_Y^2)\) with \(X\) and \(Y\) independent, then any linear combination \(Z_{\lambda, \delta} = \lambda X + \delta Y\) for \(\lambda, \delta\in \mathbb R\) will be distributed according to
The expected value is \(\mathbb E Z_{\lambda, \delta} = \lambda\mu_X + \delta\mu_Y\) with variance, or “volatility”, \(\sigma_{Z_{\lambda, \delta}}^2 = (\lambda\sigma_X)^2 + (\delta\sigma_Y)^2\).
Non-independent randoms¶
In the case where \(X\) and \(Y\) are not random, they will have non-zero covariance, denoted \(c_{X, Y}\). This affects the probability law governing the linear combination \(Z_{\lambda, \delta}\).
Note
The covariance of \(X\) with itself is just the variance \(\sigma_X^2\), i.e., \(c_{X, X} = \sigma_X^2\).
With the covariance \(c_{X, Y}\) the combination \(Z_{\lambda, \delta}\) is distributed according to the law,
See that in both the independent or dependent case the expectation of \(Z_{\lambda, \delta}\), \(\mathbb EZ_{\lambda, \delta}\) remains the same. What changes is the volatility.
A remark on the statistics¶
Variance is the square of the standard deviation. Accordingly, the variance is always positive for mathematical reasons. In contrast, the standard deviation need not be positive for mathematical reasons, but must be positive if it is to be understood as a statistical parameter.
In the distribution law for \(Z_{\lambda, \delta}\), its variance is
Unlike in the case of independent randoms, it is not immediately clear here that \(\mathrm{Var}~Z_{\lambda,\delta} \geq 0\). In order to see that it is we will need to use the following:
Fact
For random variables \(X, Y\)
We will now prove the following.
Proposition
For normal randoms \(X, Y\), \(\mathrm{Var}~Z_{\lambda, \delta}\geq0\) for any \(\lambda, \delta\).
Proof.
The proof of this statement can be arrived at by contradiction. Since the statement in the proposition is about a statistical parameter, it is equivalent to proving the following mathematical proposition: let \(x, y\) be positive, real variables and \(\rho < 1\) a parameter. Then
To prove the above, suppose in converse that \(x^2 + y^2 < 2\rho xy\). We will intend on arriving at a contradiction. Firstly, with this assumption note that \(xy> 0\). Dividing through by \(xy\) and using that \(\rho\leq 1\) gives,
Set \(z = \frac{x}{y}\). Then \(z\) will again be a real, positive number. The above inequality becomes
We have now arrived at a contradiction. Since \(z\) is real, \(z-1\) is real so that \((z-1)^2\) will necessarily be positive. Hence that \((z-1)^2< 0\) cannot be true.
From here, using that \(c_{X, Y} = \rho\sigma_X\sigma_Y\) for \(\rho\) the Pearson correlation coefficient (a parameter between \(-1\) and \(1\)), we can deduce that the variance \(\mathrm{Var}~Z_{\lambda,\delta}\) above must indeed be positive for any \(\lambda, \delta\).
Optimization¶
From the problem statement, we want to maximize the expectation \(\mathbb EZ_{\lambda, \delta}\) subject to a fixed level of risk \(r\), which can be taken here to mean the standard deviation in \(Z_{\lambda, \delta}\). We first consider the case where \(X\) and \(Y\) are independent below, after which we proceed to study the dependent case.
Independent randoms¶
For two independent randoms \(X, Y\) recall that
Set \(r^2 = (\lambda\sigma_X)^2 + (\delta\sigma_Y)^2\). Our problem is thus:
maximize \(\lambda \mu_X + \delta\mu_Y\);
with \(r^2\) constant.
See that we need to maximize a linear form in the parameters \(\lambda, \delta\) subject to a quadratic constraint in \(\lambda, \delta\). Thus, we find here a typical problem in the area of quadratic programming. It will not be necessary to appeal to this area however as, in our setting, an analytic solution can be obtained through the method of Lagrange multipliers.
Solution via Lagrange multipliers¶
Let \(\mathcal L\) be the Lagrange functional which, in our setting, will be given by
Choosing \(\frac{C}{2}\) in \(\mathcal L\) above is convenient in undertaking forthcoming calculations. This factor, or the constant \(C\) itself, is usually referred to as the Lagrange multiplier.
Note
If \((\lambda^*, \delta^*)\) is an optimal value of \(\mathcal L\) subject to \(\partial \mathcal L/\partial C = 0\), then \(\mathcal L(\lambda^*, \delta^*, C)\) is at a minima or maxima along the curve \(\partial \mathcal L/\partial C = 0\). In particular, \(d\mathcal L|_{(\lambda^*, \delta^*)} =0\). This latter condition is our means of finding the optimal points \((\lambda^*, \delta^*)\).
The differential \(d\mathcal L\) of \(\mathcal L\) is,
Setting \(d\mathcal L = 0\) gives the equations,
This latter equation is just the specification of the curve \(\partial \mathcal L/\partial C = 0\) and, oestensibly, adds no further information. It is referred to as the constraint equation.
From the former two equations, see that we obtain expressions for the parameters \(\lambda, \delta\) in terms of the Lagrange multiplier \(C\),
In order to find the optimal values \(\lambda^*, \delta^*\) it remains to determine the multiplier \(C\). This is where we can appeal to the constraint equation (the last of the above three equations).
Recall that \(\mu_X,\sigma_Y, \mu_Y, \sigma_Y\) and \(r^2\) are all constants, established at the outset of the problem statement. Substituting \(\lambda, \delta\) above into the constraint equation results in the following quadratic equation for \(C\),
This readily gives the following values for \(C\),
Optimal parameters are therefore,
Now recall from the problem statement that we want \((\lambda^*, \delta^*)\) which maximizes the expectation \(\mathbb EZ_{\lambda^*, \delta^*} = \lambda^*\mu_X + \delta^*\mu_Y\). As the expectation is linear in the parameters \(\lambda^*, \delta^*\), the optima to choose from the possibilities depends entirely on the signs of the slope of the linear form, i.e., on \(\mu_X\) and \(\mu_Y\).
Example
In the case where \(\mu_X, \mu_Y > 0\) it is clear that \((\lambda^*, \delta^*) = (\lambda^*_+, \delta^*_+)\) are the appropriate optima which maximize the linear form.
Whichever optima \((\lambda^*, \delta^*)\) we find, it means the following: with \(r\) fixed, \(\mathbb EZ_{\lambda^*, \delta^*} \geq \mathbb EZ_{\lambda, \delta}\) for all \(\lambda, \delta\).
Non-independent randoms¶
The case of non-independent randoms is a bit more involved. The general principles are the same however. For \(X, Y\) non-independent means they have non-vanishing covariance \(c_{X, Y}\). Any linear combination \(Z_{\lambda, \delta} = \lambda X + \delta Y\) will be distributed according to,
We can use the method of Lagrange multipliers again to find the parameter values \(\lambda^*, \delta^*\) which maximize the expectation \(\mathbb EZ_{\lambda, \delta}\) for all \(\lambda, \delta\) subject to a fixed level of risk \(r^2 = (\lambda\sigma_X)^2 + (\delta\sigma_Y)^2 + 2c_{X, Y}\lambda\delta\).
Lagrange multipliers for dependent randoms¶
The Lagrangian functional \(\mathcal L\) mediating our optimisation is the following,
Its differential is,
Setting \(d\mathcal L = 0\) now gives the following equations,
Recall that the latter equation is referred to as the constraint equation. Solving the first two equations simultaneously gives the following expressions for \(\lambda\) and \(\delta\) in terms of the problem constants \(\mu_X, \sigma_X, \mu_Y, \sigma_Y\) and the yet to be determined multiplier \(C\),
Note
As a quick consistency check, note that if \(X\) and \(Y\) were independent, then \(c_{X, Y} = 0\). In this case, we recover the expressions for \(\lambda, \delta\) in the case of independent randoms.
Warning
If \(X\) and \(Y\) are perfectly correlated, then \(c_{X, Y} = \pm\sigma_X\sigma_Y\), i.e., that \(c_{X, Y}^2 = \sigma_X^2\sigma_Y^2\) in which case \(\lambda,\delta\) above are undefined. The case of perfectly correlated, random variables needs to be considered seperately. In what follows we assume \(c^2_{X, Y}\neq \sigma^2_X\sigma^2_Y\).
In order to determine the multiplier \(C\) we need to substitute the expressions \(\lambda, \delta\) above into the constraint equation. To ease our calculation define the following:
The constraint equation now becomes
and hence we find,
Note
Generally, our solution is well defined if and only if \(A^2_\lambda + A^2_\delta + 2A_{\lambda\delta}> 0\), i.e., if \(C\) is real. This is of course true however for the same reasons that \(\mathrm{Var}~Z_{\lambda,\delta} \geq 0\) (see the proof of this statement above).
With possible constants \(C_\pm\) established, the optimal values \(\lambda^*,\delta^*\) are given by
It remains to check the pairs \((\lambda_+, \delta_+), (\lambda_+, \delta_-), (\lambda_-, \delta_+), (\lambda_-, \delta_-)\) to see which pair will maximize the linear form \(\mathbb EZ_{\lambda,\delta} = \lambda \mu_X + \delta\mu_Y\).
Note
As a consistency check, if \(X, Y\) are independent randoms, \(c_{X, Y} = 0\). In this case the above expressions reduce to what we found earlier for independent randoms.
Applications¶
We consider an application of these optimizations to portfolio theory. In this context, how might one interpret \(X, Y\) and \(Z_{\lambda, \delta}\)? Generally, \(X\) and \(Y\) might refer to particular stocks or funds while \(Z\) a combination, or, portfolio.
There are two obvious metrics to which \(X\) and \(Y\) could refer to: stock prices and growth rates. The relation between the two are the following. For a time \(T\), if \(p_0^X\) and \(p_1^X\) refer to the price in \(X\) at the start and end of \(T\) respectively, then the growth rate \(r_X\) over \(T\) is
That is, \(r_X = \frac{1}{T} \ln(p_1^X/p_0^X)\).
Note
Assuming growth rates, rather than prices, are normally distributed is another way to say prices are log-normal.
Should the statistics we observe refer to the variation in prices or the variation in growth rates? We address the issue with prices in the following.
Warning
Suppose \(X\) is a stock and \(p^X\) the random variable representing the price of \(X\). Based on a sampling of prices over a time period, let \(\mu_X\) be the average and \(\sigma_X^2\) the variance in prices. The issue now is that the distribution \(\mathcal N(\mu_X, \sigma_X^2)\) does not represent growth in the unit price of \(X\). Rather, it assumes the price of \(X\) will hover around the average \(\mu_X\) with volatility \(\sigma^2_X\). This will be the case if one expects zero growth in the price of \(X\). If there is evidence of growth in the price of \(X\) however, the distribution \(\mathcal N(\mu_X, \sigma_X^2)\) is doomed to grossly mispredict future values of \(p_X\).
Growth rates¶
Continuing on from the above warning, a probability distribution is more appropriate for modelling rates of growth. This is because a constant, non-zero rate of growth will, depending on its sign, represent changes in the price more consistent with a growing or declining value. If the average growth rate is positive, it will indicate an upward trend in the price, ideally reflecting overall growth in value.
Rates of growth can be sampled from historical data by segmenting a time range \(T_{range}\) into time periods of length \(T\) and calculating the daily growth of the asset or stock over each such period.
For stocks \(X, Y\) let \(r_X, r_Y\) be the random variables representing the respective growth rates in \(X,Y\) over time periods of length \(T\). Then \(\mu_X, \mu_Y\) are the average rates of growth over the whole time range \(T_{range}\).
Note
These averages \(\mu_X, \mu_Y\) reflect the expected level of growth in \(X, Y\) over a given day. To annualize these rates we need to simply multiply by the number of days in a year. Similarly, the standard deviation represents expected volatility in daily growth. To annualize it, it is again a matter of multiplying by the number of days in a year.
Over the time range a portfolio \(Z = \lambda X + \delta Y\) grows at an averate rate \(\mu_Z = \lambda \mu_X + \delta \mu_Y\).
Warning
Here \(r_{Z} = \lambda r_X + \delta r_Y\) does not mean we are holding \(\lambda\) units of \(X\) and \(\delta\) units of \(Y\).
Let \(\lambda^*, \delta^*\) be optimal parameters. Then \(\mu_{Z^*}\) represents the maximum expected growth subject to a fixed risk tolerance, being the variance in growth rates \(\sigma^2_{Z}\) for the portfolio \(Z\).
Now suppose we know \(\mu_{Z^*}\) by having found the optimal parameters \(\lambda^*, \delta^*\). Suppose moreover that we have an amount \(N\) to invest. How much of \(N\) should we invest into holding \(X\) and how much into holding \(Y\)?
Optimal unit holdings¶
To answer this question, recall from earlier that unit prices are related to growth rates over a time period of length \(T\) by,
Now let \(N\) be our total investment amount. Write \(N = N_X + N_Y\) where \(N_X\) represents the amount invested in \(X\) and \(N_Y\) the amount invested in \(Y\).
Above, the prices \(p_i^X, p_i^Y, p_i^Z\) are over a time period. We invest at the start of the time range \(T_{range}\) however, so these prices are now at the start and end of the time range. That is, \(p_0^X, p_1^X\); \(p_0^Y, p_1^Y\) and \(p_0^Z, p_1^Z\) now denote the starting and ending prices respectively of \(X, Y\) and \(Z\) over the time range \(T_{range}\). The respective unit holdings are,
Our objective is to find a formula relating \(\lambda_X, \delta_Y\) with the optimal parameters \(\lambda^*, \delta^*\) in the average growth rate \(\mu_{Z^*}\). With \(\lambda_X,\delta_Y\) as above, note that \(p_0^{Z}, p_1^Z\) will be related to the prices \(p_0^X, p_1^X\) and \(p_0^Y, p_1^Y\) by
Let \(\ell\) denote the length of the time range \(T_{range}\). With \(\mu_{Z} = \lambda \mu_X + \delta \mu_Y\), see that
From the above expressions for \(p_0^Z, p_1^Z\) we find
Hence,
Now recall that we have a specified investment amount \(N\). Using this gives,
Rearranging for \(\delta_Y\) now gives an expression for \(\delta_Y\) in terms of known parameters. Substituting this into \(\lambda_X\) then gives \(\lambda_X\) in terms of known parameters. In known parameters then we find,
As a consistency check, see that \(p_0^X\lambda_X + p_0^Y\delta_Y = N\). Note moreover that it is relatively symmetric, as we might expect since \(X\) and \(Y\) enter symmetrically (i.e., one is not preferable to the other).
Note
By relatively symmetric it is meant symmetric if \(\lambda = \delta\). So in the case \(\lambda = \delta\), interchanging \(X\) and \(Y\) has no effect on the resepctive holdings.
Warning
The formulae for the holdings \(\lambda_X, \lambda_Y\) are not defined in the case \(\mu_X = \mu_Y\).
We can conclude now that for optimal parameters \(\lambda^*, \delta^*\) defining the maximum growth rate \(\mu_{Z^*}\) subject to a fixed volatility: for a given investment amount \(N\), we should hold \(\lambda_X^*\) units of \(X\) and \(\delta_Y^*\) units of \(Y\) where \(\lambda_X^*, \delta_Y^*\) are given by the above formulas for \(\lambda_X, \delta_Y\) with \(\lambda = \lambda^*\) and \(\delta = \delta^*\).
Note
Since unit holdings are integers, we ought to hold the respective floors of \(\lambda_X^*, \delta_Y^*\). Moreover, these holdings could be negative which signifies taking a short position in the stock in question.
The optimal amount to invest in \(X\) is then \(p_0^X\lambda_X^*\) and in \(Y\) it is \(p_0^Y\delta^*_Y\).
Note
The answer we have obtained above does not generalize to find optimal holdings among three or more stocks. It was essential to use the relation \(N_X = N - N_Y\) to eliminate one degree of freedom and thereby return a unique solution. If there are three or more stocks, this relation will still eliminate one degree of freedom, but there will be more degrees of freedom overall. As such, this relation alone will not be sufficient to yield a unique solution.