Gradient Calculation Methods

Training Variational Quantum Algorithms (VQAs) relies heavily on classical optimization loops. As we discussed, a VQA involves a Parameterized Quantum Circuit (PQC) $U(\theta)$ and a cost function $C(\theta)$, often defined as the expectation value of an observable $H$ measured on the output state prepared by the PQC:

$$C(\theta) = \langle \psi(\theta) | H | \psi(\theta) \rangle = \langle 0 | U^\dagger(\theta) H U(\theta) | 0 \rangle$$

To minimize this cost function using gradient-based methods (like gradient descent and its variants), we need to compute the gradient vector $\nabla_\theta C(\theta)$, whose components are the partial derivatives $\frac{\partial C}{\partial \theta_j}$ with respect to each parameter $\theta_j$ in the circuit. Since the cost function evaluation involves running a quantum circuit and performing measurements, calculating these gradients requires specific techniques tailored to this hybrid quantum-classical setup. Let's examine the primary methods used in practice.

Numerical Finite Differences

The most straightforward approach borrows directly from classical numerical methods: finite differences. To estimate the partial derivative with respect to a parameter $\theta_j$, we can evaluate the cost function at slightly perturbed parameter values. The central difference formula is commonly used:

$$\frac{\partial C}{\partial \theta_j} \approx \frac{C(\theta + \epsilon e_j) - C(\theta - \epsilon e_j)}{2\epsilon}$$

Here, $e_j$ is a unit vector with a 1 in the $j$-th position and zeros elsewhere, and $\epsilon$ is a small step size. This method requires evaluating the quantum circuit (and estimating the expectation value) twice for each parameter $\theta_j$ to compute its partial derivative.

While simple to understand and implement, finite differences have significant drawbacks in the context of quantum computation:

1. Choice of $\epsilon$: Selecting an appropriate $\epsilon$ is difficult. If $\epsilon$ is too large, the approximation is inaccurate (truncation error). If $\epsilon$ is too small, the difference $C(\theta + \epsilon e_j) - C(\theta - \epsilon e_j)$ might be dominated by statistical noise (shot noise) inherent in estimating expectation values from a finite number of measurements, leading to large errors in the gradient estimate (subtraction error).
2. Statistical Noise: The subtraction in the numerator can amplify the effect of shot noise, especially when $C(\theta + \epsilon e_j)$ and $C(\theta - \epsilon e_j)$ are very close. Achieving sufficient precision often requires a very large number of measurements (shots) for each evaluation, increasing the computational cost.

Due to these limitations, while finite differences can be useful for quick checks or simple problems, more specialized methods are generally preferred for training VQAs.

The Parameter-Shift Rule

A more widely adopted method for calculating gradients of PQCs is the parameter-shift rule. For a specific class of parameterized gates, this rule provides an analytical expression for the gradient, avoiding the numerical instability associated with choosing $\epsilon$ in finite differences.

Consider a gate in the PQC of the form $G_j(\theta_j) = e^{-i \frac{\theta_j}{2} P_j}$, where $P_j$ is an operator satisfying $P_j^2 = I$. This includes common single-qubit rotation gates like $R_X(\theta_j) = e^{-i \frac{\theta_j}{2} X}$, $R_Y(\theta_j) = e^{-i \frac{\theta_j}{2} Y}$, and $R_Z(\theta_j) = e^{-i \frac{\theta_j}{2} Z}$, since $X^2 = Y^2 = Z^2 = I$.

If the overall PQC $U(\theta)$ can be written as $U(\theta) = V G_j(\theta_j) W$, where $V$ and $W$ are other quantum circuits independent of $\theta_j$, then the derivative of the expectation value $C(\theta) = \langle 0 | U^\dagger(\theta) H U(\theta) | 0 \rangle$ with respect to $\theta_j$ can be computed exactly using the following formula:

$$\frac{\partial C(\theta)}{\partial \theta_j} = \frac{1}{2} \left[ C\left(\theta + \frac{\pi}{2} e_j\right) - C\left(\theta - \frac{\pi}{2} e_j\right) \right]$$

This remarkable result states that the exact derivative is proportional to the difference between the cost function evaluated with the parameter $\theta_j$ shifted forward and backward by a specific amount, $s = \pi/2$.

Why does this work? Let's sketch the idea. The derivative involves differentiating the gate $G_j(\theta_j)$: $\frac{\partial G_j(\theta_j)}{\partial \theta_j} = -\frac{i}{2} P_j e^{-i \frac{\theta_j}{2} P_j} = -\frac{i}{2} P_j G_j(\theta_j)$ Inserting this into the derivative expression for $C(\theta)$ leads to terms involving $P_j$. The main insight is that for gates where $P_j^2=I$, the operator $P_j$ can be related to shifted versions of the original gate $G_j(\theta_j)$. Specifically, one can show that: $P_j G_j(\theta_j) = \frac{i}{2} \left[ G_j\left(\theta_j + \frac{\pi}{2}\right) - G_j\left(\theta_j - \frac{\pi}{2}\right) \right]$ Substituting this back into the derivative expression for $C(\theta)$ and simplifying eventually yields the parameter-shift rule formula.

Visual representation of the parameter-shift rule for calculating the gradient component $\partial C / \partial \theta_j$. The expectation value $\langle H \rangle$ is estimated by running the circuit with the parameter $\theta_j$ shifted by $+\pi/2$ and $-\pi/2$. The results are combined to yield the exact gradient.

Advantages:

• Analytically Exact: It provides the true gradient, not a numerical approximation (up to shot noise in evaluating $C$).
• No Step Size Tuning: It avoids the problematic choice of a small $\epsilon$. The shift $s = \pi/2$ is fixed.
• Robustness: Often more resilient to shot noise compared to finite differences, as the shift is large, making the difference $C(\theta + s e_j) - C(\theta - s e_j)$ typically larger relative to the noise floor.

Cost: Like central finite differences, the parameter-shift rule requires two circuit evaluations for each parameter $\theta_j$.

Applicability: The basic rule applies directly to gates generated by Pauli operators ($X, Y, Z$). Most common PQC ansätze used in QML are constructed primarily from such gates, making the parameter-shift rule widely applicable.

Generalizations and Other Methods

• Generalized Parameter-Shift Rules: The parameter-shift concept can be extended to gates generated by operators $G$ that have only two unique eigenvalues, even if $G^2 \neq I$. This covers gates like controlled rotations under certain conditions. The shift values $s$ and coefficients in the formula might change depending on the generator $G$.
• Linear Combination of Unitaries (LCU): More advanced techniques exist, such as methods based on expressing the derivative operator itself as a linear combination of unitary operations. These can sometimes offer advantages, potentially calculating the gradient with fewer circuit executions under specific circumstances, but often involve more complex circuit constructions or measurements.
• Stochastic Parameter-Shift: For gradients involving many parameters, variations exist where only a subset of parameter shifts are computed per optimization step, introducing stochasticity similar to classical mini-batch gradient descent.

Gradients from Measurements

It's important to remember that in practice, whether using finite differences or parameter-shift rules, the cost function values $C(\theta \pm s e_j)$ are themselves estimated from a finite number of measurements (shots) on the quantum computer. Therefore, the calculated gradient $\nabla_\theta C(\theta)$ is also an estimate. The accuracy of this gradient estimate depends directly on the number of shots used for each expectation value calculation. This statistical uncertainty is inherent in VQAs and is a primary consideration when choosing optimization algorithms and interpreting training dynamics, which we will discuss next.

These gradient calculation methods form a link between the quantum circuit evaluation and the classical optimization routine, enabling the training of VQAs for machine learning tasks. The parameter-shift rule, in particular, represents a standard and effective technique implemented in most quantum software frameworks like Qiskit, PennyLane, and TensorFlow Quantum.