Derivative Notation (Leibniz and Lagrange)

The derivative represents the instantaneous rate of change, or the slope of the tangent line at a point on a function's graph. How this mathematical concept is written down varies. Just like different programming languages have different syntax for similar concepts, mathematicians have developed a couple of common ways to denote the derivative. Knowing these notations is important because you'll encounter them frequently in machine learning literature and resources.

Lagrange Notation: The Prime Symbol

One of the most common and straightforward notations was introduced by Joseph-Louis Lagrange. If you have a function, say $f(x)$, its derivative is often written as $f'(x)$. You read this as "f prime of x".

The prime symbol ($'$) simply indicates that we are talking about the derivative of the original function $f$.

• If the function is $f(x) = x^2$, its derivative is $f'(x) = 2x$. (We'll learn how to calculate this soon).
• If the function is defined using a different variable, like $g(t)$, its derivative is $g'(t)$.
• If we are talking about the function's output $y = f(x)$, sometimes the derivative is written as $y'$.

This notation is compact and clearly links the derivative back to the original function name ($f$, $g$, etc.). It's especially convenient when you're evaluating the derivative at a specific point. For example, $f'(3)$ would mean "the derivative of the function $f$, evaluated at $x=3$."

Leibniz Notation: The Ratio of Infinitesimals

Another widely used notation comes from Gottfried Wilhelm Leibniz, one of the inventors of calculus. This notation looks like a fraction:

$$\frac{dy}{dx}$$

You typically read this as "the derivative of y with respect to x," or sometimes "dee y dee x."

Let's break this down:

• Recall that the slope of a line is the change in $y$ divided by the change in $x$, often written as $\frac{\Delta y}{\Delta x}$ (read "delta y over delta x").
• The derivative is the limit of this ratio as the change in $x$ becomes infinitesimally small.
• Leibniz used $dy$ and $dx$ (using 'd' instead of '$\Delta$') to represent these infinitesimally small changes.

So, $\frac{dy}{dx}$ visually represents the idea of an infinitesimal change in $y$ resulting from an infinitesimal change in $x$. It emphasizes which variable the output ($y$) is changing with respect to ($x$).

If our function is explicitly written as $f(x)$, like $f(x) = x^2$, Leibniz notation might look like this:

$$\frac{d}{dx} f(x) \quad \text{or} \quad \frac{d}{dx} (x^2)$$

Here, $\frac{d}{dx}$ acts like an operator, meaning "take the derivative with respect to $x$ of whatever follows." So, $\frac{d}{dx} (x^2) = 2x$.

Why Two Notations?

Both notations are useful in different contexts:

• Lagrange ($f'(x)$): Often cleaner when dealing with a function and evaluating it at specific points. It's compact.
• Leibniz ($\frac{dy}{dx}$): Very clear about which variable we are differentiating with respect to. This becomes particularly helpful when dealing with functions of multiple variables (which we'll see in later chapters) or when working with related rates. For instance, if cost ($C$) depends on a model parameter ($w$), $\frac{dC}{dw}$ clearly states we're looking at how cost changes as the parameter $w$ changes.

You don't need to exclusively pick one. You'll see both used, sometimes even together. The important thing is to recognize them and understand that $f'(x)$ and $\frac{dy}{dx}$ (when $y=f(x)$) refer to the same fundamental concept: the derivative of the function $f$ with respect to its input $x$.

In the next sections, we'll start using these notations as we learn the rules for actually calculating derivatives.