Linear Regression is a linear approach to modeling the relationship given examples and their labels. The algorithm seeks coefficients a, b such that the classifier y = a*x + b. The red line is the classifier. The loss function is a squared loss because the algorithms might miss and not be precise so the classifier measures the distance from the real points.
Next: Exploding Gradient Problem