Example: Simple Linear Regression Model

Calculus provides tools for finding minimums and maximums using derivatives and the concept of gradient descent. These tools apply to fundamental machine learning models, such as Simple Linear Regression.

The goal of simple linear regression is to find the best straight line that describes the relationship between a single input feature and a target output. Imagine you have some data points plotted on a graph. Linear regression tries to draw a line through these points that fits them as closely as possible.

The Model Equation

The equation for a straight line is likely familiar:

$$y = mx + b$$

Let's break down the components in a machine learning context:

• $x$: This is our input feature. It's the information we have, also called the independent variable. For example, $x$ could be the square footage of a house.
• $y$: This is the target variable we want to predict, also called the dependent variable. For example, $y$ could be the selling price of the house.
• $m$: This is the slope of the line. It determines how much $y$ changes for a one-unit increase in $x$. In our house example, it represents how much the price increases for each additional square foot.
• $b$: This is the y-intercept of the line. It's the value of $y$ when $x$ is zero. In some cases, this might represent a baseline value (e.g., the base price of a house regardless of size, though this interpretation isn't always meaningful depending on the context).

In machine learning terminology, $m$ and $b$ are the parameters (or sometimes called weights or coefficients) of our model. Our objective during training is to find the optimal values for $m$ and $b$ that make our line fit the data best.

Visualizing the Goal

Consider a small dataset relating study hours ($x$) to exam scores ($y$).

The blue dots represent our data. The green dashed line seems to capture the trend reasonably well, while the red dotted line is clearly a poor fit. Linear regression aims to find the line parameters ($m$ and $b$) that result in the best possible fit, like the green line or even better.

This process of finding the "best fit" line is a supervised learning task. We are given example pairs of $(x, y)$ (study hours, exam score) and we want the machine learning algorithm to learn the relationship, represented by the parameters $m$ and $b$. Once learned, we can use the model $y = mx + b$ to predict the exam score for a new number of study hours.

The core idea is that we can measure how good a particular line (defined by specific values of $m$ and $b$) fits the data. If we can measure this "goodness of fit" (or conversely, the "badness of fit" or error), we can then use calculus, specifically gradient descent, to systematically adjust $m$ and $b$ to make the line fit better and better.

This "measure of fit" is what we call a cost function, which we will define in the next section. Remember, for our optimization algorithms, $m$ and $b$ are the variables we will be adjusting to minimize this cost.