Interpretation of Linear Models
In this short handout we will consider the interpretation of linear regression model coefficients in models with different combinations of outcome and regressor variables:
continuous level-level
continuous-discrete
discrete-continuous
discrete-discrete
log-level
level-log
log-log
In all instances, we will work on the CLRM model assumptions 1 & 2, which tell us that the conditional expectation function is linear in parameters:
\[ E[Y_i|X_i] = X_i'\beta \]
Continuous, level-level models
If \(Y_i\) and \(X_i\) are both continuously distributed random variables then,
\[ \beta_j = \frac{\partial E[Y_i|X_i]}{\partial X_{ij}} \] or, as a vector,
\[ \beta = \frac{\partial E[Y_i|X_i]}{\partial X_{i}} = \begin{bmatrix}\frac{\partial E[Y_i|X_i]}{\partial X_{i1}}\\ \vdots \\ \frac{\partial E[Y_i|X_i]}{\partial X_{ik}}\end{bmatrix} = \begin{bmatrix}\beta_1\\ \vdots \\ \beta_k\end{bmatrix} \]
The regression parameter has a partial derivative interpretation with respect to the CEF. As discussed in Handout 1, this is often used to motivate the experimental language of ceteris paribus: “holding all else fixed.
Continuous-discrete models
Consider a case where there is a single discrete regressor: \(D_i \in \{0,1\}\). For example,
\[ Y_i = \beta_1 + \beta_2 D_i + \varepsilon_i \] We cannot apply the partial derivative interpretation since \(D\) is not continuous. Instead, we will look at differences:
\[ \begin{aligned} E[Y_i|D_i=1] =& \beta_1 + \beta_2 \\ E[Y_i|D_i=0] =& \beta_1 \\ \Rightarrow \beta_2 =& E[Y_i|D_i=1] - E[Y_i|D_i=0] \end{aligned} \]
We can easily extend this the case where the model includes additional (discrete or continuous) covariates, as well as case where the variable takes on multiple discrete values.
Discrete-continuous models
If the outcome is discrete (\(Y_i\in\{0,1\}\)) while the regressors are continuous, the resulting linear model is referred to as a linear probability model.
\[ E[Y_i|X_i] = Pr(Y_i = 1|X_i) = X_i'\beta \] This is differentiable, since \(X\) is continuous and the same partial derivative interpretation follows.
\[ \beta_j = \frac{\partial Pr(Y_i=1|X_i)}{\partial X_{ij}} \]
Note, the unit of \(Y\) is probability-points (\(\in[0,1]\)), not %-points (\(\in[0,100]\)). Of course, the conversion of units can be made by \(\times 100\) to measure in %-points.
Discrete-discrete models
If both the outcome and regressor(s) are discrete, then the parameter identifies a difference in conditional probabilities, \[ \beta_2 = Pr(Y_i|D_i=1) - Pr(Y_i=1|D_i=0) \] Note, the unit of \(Y\) is probability-points (\(\in[0,1]\)), not %-points (\(\in[0,100]\)).
Log-level models
Consider the model,
\[ \ln(Y_i) = X_i'\beta + \varepsilon_i \] Then,
\[ X_i'\beta = E[\ln(Y_i)|X_i] \]
\[ \beta_j = \frac{\partial E[\ln(Y_i)|X_i]}{\partial X_{ij}} \]
The coefficient is therefore measured in log-units of \(Y\). The relation to a change in the (expected) level of \(Y\) is given by,
\[ \%\Delta E[Y_i|X_i] = (exp(\beta)-1)\times 100 \] For reasonably small values of \(\beta\) (i.e. within the range \([-0.1,0.1]\)) this can be approximated by,
\[ \%\Delta E[Y_i|X_i] = \beta\times 100 \] A 1-unit change in \(X_{i1}\) is associated with a \(\beta_1\) percent change in the expected value of \(Y\).
This referred to as a semi-elasticity.
Level-log models
If the regressor(s) is measure in log-units; for example,
\[ Y_i = \beta_1 + \beta_2 \ln(X_i)_i + \varepsilon_i \] Then,
\[ \beta_2 = \frac{\partial E[Y_i|X_i]}{\partial \ln(X_{i})} \]
A 1 percent increase in \(X\) is given by \(X\times1.01\). This is equivalent to a change in \(\ln(X)\) of,
\[ \ln(X_i\times1.01) - \ln(X_i) = \ln(1.01) \approx 0.01 \] Thus, a 1 percent increase in the level of \(X\) is associated with a \(\beta_2/100\) increase in the expected value of \(Y\). Or, more accurate
\[ \Delta E[Y_i|X_i] = \beta_2\times \ln(1.01) \] This is also a semi-elasticity.
Log-log models
In models where both the outcome and regressor are log-transformed with an elasticity interpretation.
\[ \ln(Y_i) = \beta_1 + \beta_2 \ln(X_i)_i + \varepsilon_i \] \[ \beta_2 = \frac{\partial E[\ln(Y_i)|X_i]}{\partial \ln(X_{i})} \] \(\beta_2\) is a the % change in the expected value of \(Y\) from a 1 % change in \(X\).