Partial Derivative Notation

Derivatives of single-variable functions, such as $f(x)$, are commonly represented by notations like $\frac{df}{dx}$ or $f'(x)$. For functions involving multiple variables, like $f(x, y)$ or $f(x_1, x_2, \dots, x_n)$, it becomes essential to specify the exact variable for differentiation, keeping all other variables constant.

The Partial Derivative Symbol: ∂

To distinguish partial derivatives from the ordinary derivatives we saw earlier, we use a different symbol: $\partial$. This symbol is often called the "curly d," "del," or simply "partial." It signals that we are performing a partial differentiation.

Leibniz-Style Notation

The most common notation you'll encounter, similar to the $\frac{dy}{dx}$ notation for ordinary derivatives, uses the $\partial$ symbol.

If we have a function $f(x, y)$, its partial derivative with respect to $x$ is written as:

$$\frac{\partial f}{\partial x}$$

This notation reads as "the partial derivative of $f$ with respect to $x$." It explicitly tells us:

1. $\partial f$: We are looking at the change in the function $f$.
2. $\partial x$: We are considering this change as the variable $x$ changes.
3. Implicitly: All other variables (in this case, just $y$) are treated as constants during this differentiation.

Similarly, the partial derivative of $f$ with respect to $y$ is written as:

$$\frac{\partial f}{\partial y}$$

If the function output is assigned to another variable, say $z = f(x, y)$, you might also see the notation written as $\frac{\partial z}{\partial x}$ or $\frac{\partial z}{\partial y}$.

Subscript Notation

Another common and often more compact notation uses subscripts to indicate the variable of differentiation. For the same function $f(x, y)$:

• The partial derivative with respect to $x$ can be written as $f_x$ or $f_x(x, y)$.
• The partial derivative with respect to $y$ can be written as $f_y$ or $f_y(x, y)$.

This notation is especially convenient when writing out the gradient vector, which we'll see shortly.

Example

Let's consider a simple function $g(a, b) = 3a^2b + 5b^3$.

• The partial derivative of $g$ with respect to $a$ using Leibniz-style notation is $\frac{\partial g}{\partial a}$. Using subscript notation, it's $g_a$.
• The partial derivative of $g$ with respect to $b$ using Leibniz-style notation is $\frac{\partial g}{\partial b}$. Using subscript notation, it's $g_b$.

(We'll learn how to calculate these in the next section, but for now, focus on understanding what the notation represents).

Evaluating at a Point

Just like with ordinary derivatives, we often want to evaluate a partial derivative at a specific point. If we want to find the partial derivative of $f(x, y)$ with respect to $x$ at the point $(x_0, y_0)$, we can write it as:

$$\frac{\partial f}{\partial x}\bigg|_{(x_0, y_0)} \quad \text{or} \quad \frac{\partial f}{\partial x}(x_0, y_0) \quad \text{or} \quad f_x(x_0, y_0)$$

All these notations mean the same thing: calculate the partial derivative expression with respect to $x$, and then substitute $x = x_0$ and $y = y_0$ into the result.

Understanding this notation is fundamental because it's the language used to describe how complex functions, like the cost functions in machine learning models, change as their individual inputs or parameters are adjusted.