3.5 Bivariate descriptive statistics: the regression equation  (Page 2/2)

$\epsilon$ = the Greek letter epsilon

For each data point, you can calculate the residuals or errors, $y_i-{\hat{y}}_i={\epsilon }_i$ for $i=\text{1, 2, 3, ..., 11}$ .

Each $\left|\epsilon \right|$ is a vertical distance.

For the example about the third exam scores and the final exam scores for the 11 statistics students, there are 11 data points. Therefore, there are 11 $\epsilon$ values. If you square each $\epsilon$ and add, you get

$({\epsilon }_1{)}^2+({\epsilon }_2{)}^2+\text{...}+({\epsilon }_{11}{)}^2=\stackrel{11}{\underset{\text{i = 1}}{\Sigma }}{\epsilon }^2$

This is called the Sum of Squared Errors (SSE) .

Using calculus, you can determine the values of $a$ and $b$ that make the SSE a minimum. When you make the SSE a minimum, you have determined the points that are on the line of best fit. It turns out thatthe line of best fit has the equation:

where

• $a=\overline{y}-b\cdot \overline{x}$
• $\overline{x}$ and $\overline{y}$ are the sample means of the $x$ values and the $y$ values, respectively. The best fit line always passes through the point $(\overline{x},\overline{y})$ . and
• $b=r\cdot \left(\frac{s_y}{s_x}\right)$
• where $s_y$ = the standard deviation of the $y$ values and $s_x$ = the standard deviation of the $x$ values. $r$ is the correlation coefficient which is discussed in the next section.

Least squares criteria for best fit

Using the summary statistics and correlation coefficient for the relationship between third exam score and final exam score we will calculate the regression equation, the line of best fit.

Start by calculating the slope of the line.

• $b=r\cdot \left(\frac{s_y}{s_x}\right)=\mathrm{(0.6632)}\cdot \left(\frac{\mathrm{20.8008}}{\mathrm{2.85721}}\right)=\mathrm{4.827}$

Using the slope find the u-intercept.

• $a=\overline{y}-\mathrm{(b)}\cdot \left(\overline{x}\right)=160.4545-(4.827)(69.1818)=-173.49$

The least squares regression line (best fit lint) for the third exam/final exam example has the equations: $\hat{y}=-\mathrm{173.49}+4.83x$

Third exam vs final exam example:

Understanding slope

INTERPRETATION OF THE SLOPE: The slope of the best fit line tells us how the dependent variable (y) changes for every one unit increase in the independent (x) variable, on average.

Third exam vs final exam example

• Slope: The slope of the line is b = 4.83.
• Interpretation: For a one point increase in the score on the third exam, the final exam score increases by 4.83 points, on average.