Why is linear regression useful

Multiple regressions are based on the assumption that there is a linear relationship between both the dependent and independent variables. It also assumes no major correlation between the independent variables.

As mentioned above, there are several different advantages to using regression analysis. These models can be used by businesses and economists to help make practical decisions. A company can not only use regression analysis to understand certain situations like why customer service calls are dropping, but also to make forward-looking predictions like sales figures in the future, and make important decisions like special sales and promotions.

Consider an analyst who wishes to establish a linear relationship between the daily change in a company's stock prices and other explanatory variables such as the daily change in trading volume and the daily change in market returns. If he runs a regression with the daily change in the company's stock prices as a dependent variable and the daily change in trading volume as an independent variable, this would be an example of a simple linear regression with one explanatory variable.

If the analyst adds the daily change in market returns into the regression, it would be a multiple linear regression. Financial Ratios. Advanced Technical Analysis Concepts. Risk Management. Financial Analysis. Your Privacy Rights. To change or withdraw your consent choices for Investopedia. At any time, you can update your settings through the "EU Privacy" link at the bottom of any page.

These choices will be signaled globally to our partners and will not affect browsing data. We and our partners process data to: Actively scan device characteristics for identification. I Accept Show Purposes. Your Money. For example, the goal of a physicist may be to understand their model of a particle interaction. If they create a neural network to do this, we may have low error in the system but our human understanding of the laws of nature is no better. Likewise, an economist may be more concerned with understanding the general relationship between political uncertainty in elections, uncertainty in public policy, and uncertainty in financial markets, than creating a complex interactive system that prevents generalizations that could be applied to governance, electioneering, or market trading.

So, for the time being, let us return from the vast jungles of machine learning to the tamed and comfortable community of traditional statistical models, the first of which will be linear regression. It is a powerful tool because of that simplicity.

Understanding linear regression starts with the name. A regression analysis is simply a method of estimating the relationship between a dependent variable and a set of independent variables.

The term originated many centuries ago and has stuck. For example, a tall person is likely to have children that are also tall, but probably less so. Note that regression is modeling a numeric relationship. The output of the regression is a number. This is different from something like categorization, which outputs a class or label for the input data.

The second part of the name is linear, referring to the fact that in linear Regression, we only care about linear relationships; our model is just going to be the sum of things. A Linear Regression would attempt to model the interaction between petal length and sepal length as a line of best fit. From the chart below, we can see there is a general linear trend in the data.

As sepal length increases, petal length increases in a similar manner. Linear regression produces a linear equation which is the model of the system. We can see the result of that regression plotted along with the data. Of course, there will be some error because there is some variability in the data, but the fit generally describes the relationship well. The equation for this relationship is:. This is a simple, interpretable form that is very useful for analysis and understanding.

From the equation, we can say that for each additional centimeter of sepal length, we generally observe around 0. When, why, and how the business analyst should use linear regression Posted on October 5, by Eric Benjamin Seufert.

When is linear regression appropriate? The sensible use of linear regression on a data set requires that four assumptions about that data set be true: The relationship between the variables is linear. The data is homoskedastic , meaning the variance in the residuals the difference in the real and predicted values is more or less constant. The residuals are independent , meaning the residuals are distributed randomly and not influenced by the residuals in previous observations.

The residuals are normally distributed. This assumption means the probability density function of the residual values is normally distributed at each x value. Heracles helps consumer technology and media companies reach massive audiences through strategy consulting, deep analysis, and marketing technology.

The interpretation of the features in the linear regression model can be automated by using following text templates. Another important measurement for interpreting linear models is the R-squared measurement.

R-squared tells you how much of the total variance of your target outcome is explained by the model. The higher R-squared, the better your model explains the data. The formula for calculating R-squared is:. The SSE tells you how much variance remains after fitting the linear model, which is measured by the squared differences between the predicted and actual target values.

SST is the total variance of the target outcome. R-squared tells you how much of your variance can be explained by the linear model. R-squared usually ranges between 0 for models where the model does not explain the data at all and 1 for models that explain all of the variance in your data.

It is also possible for R-squared to take on a negative value without violating any mathematical rules. This happens when SSE is greater than SST which means that a model does not capture the trend of the data and fits to the data worse than using the mean of the target as the prediction.

There is a catch, because R-squared increases with the number of features in the model, even if they do not contain any information about the target value at all. Therefore, it is better to use the adjusted R-squared, which accounts for the number of features used in the model. Its calculation is:. It is not meaningful to interpret a model with very low adjusted R-squared, because such a model basically does not explain much of the variance.

Any interpretation of the weights would not be meaningful. The importance of a feature in a linear regression model can be measured by the absolute value of its t-statistic. The t-statistic is the estimated weight scaled with its standard error.

Let us examine what this formula tells us: The importance of a feature increases with increasing weight. This makes sense. This also makes sense. In this example, we use the linear regression model to predict the number of rented bikes on a particular day, given weather and calendar information. For the interpretation, we examine the estimated regression weights. The features consist of numerical and categorical features.

For each feature, the table shows the estimated weight, the standard error of the estimate SE , and the absolute value of the t-statistic t. Interpretation of a numerical feature temperature : An increase of the temperature by 1 degree Celsius increases the predicted number of bicycles by When the weather is misty, the predicted number of bicycles is This is because of the nature of linear regression models.

The predicted target is a linear combination of the weighted features. The weights specify the slope gradient of the hyperplane in each direction. The good side is that the additivity isolates the interpretation of an individual feature effect from all other features. On the bad side of things, the interpretation ignores the joint distribution of the features. Increasing one feature, but not changing another, can lead to unrealistic or at least unlikely data points.

For example increasing the number of rooms might be unrealistic without also increasing the size of a house. The information of the weight table weight and variance estimates can be visualized in a weight plot. The following plot shows the results from the previous linear regression model. Some confidence intervals are very short and the estimates are close to zero, yet the feature effects were statistically significant.

Temperature is one such candidate. The problem with the weight plot is that the features are measured on different scales. You can make the estimated weights more comparable by scaling the features zero mean and standard deviation of one before fitting the linear model. The weights of the linear regression model can be more meaningfully analyzed when they are multiplied by the actual feature values.

The weights depend on the scale of the features and will be different if you have a feature that measures e. The weight will change, but the actual effects in your data will not. It is also important to know the distribution of your feature in the data, because if you have a very low variance, it means that almost all instances have similar contribution from this feature. The effect plot can help you understand how much the combination of weight and feature contributes to the predictions in your data.

Start by calculating the effects, which is the weight per feature times the feature value of an instance:. The effects can be visualized with boxplots. The vertical line in the box is the median effect, i. The dots are outliers.