The many ways “ you could be tricked”!

So I am not talking about life in general here, I am talking about the many ways you could be tricked if your life involves doing regression analysis. And specifically, Ordinary Least Square regression. So. This post is for fellow geeks.

There are many scenarios that cause your slope coefficients to be biased, or inconsistent, or both. Some examples include having multicollinearity involving the independent variable, non-linearity between the y and x, heteroscedasticity , or measurement error in x or y. When these scenarios are present , your variables could look more or less significant than they really are. As professor Tim McDaniel puts it “ you could be tricked EITHER way” .

But I want to highlight one specific scenario: if you have omitted a "relevant variable" that belongs to your regression model, you will get slope coefficients that are both biased and inconsistent. In this case, the independent variables could appear more OR less significant than they really are.

This example demonstrates the importance of having a theory before we start our analysis to help us reduce the likelihood of "being tricked". Knowing the literature , as inconsistent or scarce as it may be in some cases, will go a long way in helping us choose our model specifications and decide what is relevant. If you do it the other way around, i.e. go straight to your data to see "what it says" before having any theory, well, then you should prepare to deal with the consequences.

Resources: Fox, J, Regression Diagnostics, Sage University Paper Series on Quantitative Applications in Social Sciences; Berry, W. Understanding Regression Assumptions. Sage University Paper Series on Quantitative Applications in the Social Sciences #902