Understanding JAX Computation Graphs (jaxpr)

While jax.jit compiles Python functions for accelerators, sometimes looking under the hood is necessary to understand exactly what JAX is compiling. This is particularly true when debugging performance issues or unexpected behavior. JAX uses an intermediate representation called jaxpr (JAX Program Representation) to represent the computation graph derived from your Python code before it is handed off to the XLA compiler. Understanding jaxpr helps identify how JAX traces your functions and can pinpoint sources of inefficiency or recompilation.

What is a Jaxpr?

Think of jaxpr as a simplified, functional, and explicit representation of your computation. When JAX traces your Python function (which happens the first time you call a jit-compiled function with specific input types and shapes), it doesn't execute the Python code directly. Instead, it substitutes special tracer objects for your inputs and records every JAX primitive operation performed on these tracers. The result of this tracing process is the jaxpr.

Characteristics of jaxpr include:

1. Functional: Jaxprs are pure functions. They define outputs solely based on their inputs, without side effects.
2. Statically Typed: All variables within a jaxpr have well-defined shapes and data types (dtypes).
3. Explicit Operations: Jaxprs consist of a sequence of equations, each applying a specific JAX primitive (like add, sin, dot_general, reduce_sum) to input variables or constants to produce output variables. Python control flow like if statements or for loops (unless using JAX control flow primitives like lax.cond or lax.scan) are unrolled during tracing.
4. Intermediate Representation (IR): It sits between your high-level Python code and the low-level XLA HLO (High-Level Operations) representation used for compiler optimization.

Generating and Inspecting Jaxprs

You don't usually interact with jaxprs directly during typical model development, but JAX provides tools to inspect them when needed. The primary function for this is jax.make_jaxpr. It takes a function and example inputs (like jit) and returns an object representing the jaxpr, along with other information.

Let's look at a simple example:

import jax
import jax.numpy as jnp

def example_function(x, y):
  a = jnp.sin(x)
  b = jnp.cos(y)
  return a + b * 2.0

# Define example inputs
key = jax.random.PRNGKey(0)
x_example = jax.random.normal(key, (10,))
y_example = jax.random.normal(key, (10,))

# Generate the jaxpr
jaxpr_obj = jax.make_jaxpr(example_function)(x_example, y_example)
print(jaxpr_obj.jaxpr)

Running this code will print the jaxpr object. Its structure might look something like this (details can vary slightly between JAX versions):

{ lambda ; a:f32[10] b:f32[10]. let
    c:f32[10] = sin a
    d:f32[10] = cos b
    e:f32[10] = mul d 2.0
    f:f32[10] = add c e
  in (f,) }

Let's break down this printed representation:

• { lambda ; ... }: Defines the jaxpr as a lambda function.
• a:f32[10] b:f32[10]: These are the input variables (invars) with their types (float32) and shapes ([10]). They correspond to the x and y arguments of example_function.
• let ... in ...: This introduces the body of the jaxpr.
• c:f32[10] = sin a: This is an equation (eqn). It applies the sin primitive to input variable a and binds the result to a new intermediate variable c, also of type f32[10].
• d:f32[10] = cos b: Another equation applying the cos primitive to b, resulting in d.
• e:f32[10] = mul d 2.0: Applies the multiplication primitive (mul) to variable d and a constant 2.0. Constants captured from the function's environment (constvars) also appear here.
• f:f32[10] = add c e: Applies the add primitive to intermediate variables c and e, producing f.
• in (f,): Specifies the output variables (outvars) of the jaxpr, in this case, just f.

Notice how the Python code's structure is translated into a linear sequence of primitive operations acting on typed variables. This explicit, simplified form is much easier for a compiler to analyze and optimize than the original Python source.

Why Jaxprs Matter for Optimization

Understanding jaxpr helps in several ways when optimizing your JAX code:

1. Visibility into Tracing: Jaxpr shows you exactly what sequence of operations JAX derived from your Python code. This can reveal if JAX is tracing more computation than you expected, or perhaps using primitives differently than you anticipated. For example, subtle differences in NumPy vs. jax.numpy usage might lead to different primitives.
2. Understanding Transformations: You can inspect the jaxpr after applying transformations like jax.grad or jax.vmap to see how they modify the computation graph. This is useful for debugging the behavior of differentiation or vectorization.
3. Diagnosing Recompilation: jax.jit recompiles your function if the jaxpr structure changes between calls. This usually happens if the sequence of operations depends on the values of arguments, rather than just their shapes and types (a common issue with Python-level control flow based on tensor values). Comparing jaxprs generated from different calls can highlight why a recompilation was triggered. If the eqns list differs significantly, it points to dynamic behavior that JAX had to re-trace.
4. Bridging to XLA: Jaxpr is the representation that JAX feeds into the XLA compiler. While XLA performs its own sophisticated optimizations (like operator fusion, discussed later), the structure of the input jaxpr influences what XLA can optimize. A clean, predictable jaxpr is often easier for XLA to optimize effectively than one resulting from complex, unrolled Python logic.

Example with Control Flow

Consider a function using jax.lax.cond, the JAX primitive for conditional execution within jit-compiled code:

import jax
import jax.numpy as jnp
from jax import lax

def conditional_function(use_sin, x):
  # Note: `pred` for lax.cond must be a scalar boolean
  pred = use_sin > 0.5

  # Define functions for true and false branches
  def true_fun(operand):
      return jnp.sin(operand)

  def false_fun(operand):
      return jnp.cos(operand)

  # Use lax.cond
  return lax.cond(pred, true_fun, false_fun, x)

# Example inputs
x_example = jnp.ones((3,))
pred_true_example = jnp.array(0.7) # scalar value > 0.5
pred_false_example = jnp.array(0.3) # scalar value <= 0.5

# Jaxpr when pred is True
jaxpr_true = jax.make_jaxpr(conditional_function)(pred_true_example, x_example)
print("Jaxpr (True Branch potentially taken):")
print(jaxpr_true.jaxpr)

# Jaxpr when pred is False
jaxpr_false = jax.make_jaxpr(conditional_function)(pred_false_example, x_example)
print("\nJaxpr (False Branch potentially taken):")
print(jaxpr_false.jaxpr)

You'll notice that the generated jaxprs contain a cond primitive. Importantly, even though we provided concrete boolean values (0.7 leading to True, 0.3 leading to False), the jaxpr itself doesn't just contain the operations for one branch. Instead, it includes the cond primitive, which encapsulates the logic for both branches. The jaxpr looks similar regardless of the predicate's concrete value during tracing, as long as the types and shapes of inputs and outputs for both branches are consistent. This is essential for jit compilation, as the compiled code needs to handle either branch at runtime.

# Simplified Representation of the Jaxpr (structure may vary)
{ lambda ; a:f32[] b:f32[3]. let
    c:bool[] = gt a 0.5
    # Definition of true branch jaxpr (e.g., { lambda ; x:f32[3]. let y = sin x in (y,) })
    # Definition of false branch jaxpr (e.g., { lambda ; x:f32[3]. let y = cos x in (y,) })
    d:f32[3] = cond c true_branch false_branch b # Operands passed to cond
  in (d,) }

Inspecting the jaxpr reveals the cond primitive and the structure passed to XLA, confirming that JAX correctly traced the conditional logic using its specific primitive.

Limitations

While informative, jaxpr is still an intermediate step. It doesn't show the final optimizations performed by XLA, such as operator fusion (combining multiple primitives into a single, more efficient kernel) or the exact low-level code generated for the target accelerator. However, it provides an important view into the compiler's input, making it an indispensable tool for debugging performance and understanding JAX's internal workings when simple @jit application isn't enough.

By familiarizing yourself with jaxpr, you gain a deeper understanding of how JAX translates your Python code into a form suitable for high-performance compilation, enabling you to write more efficient and predictable JAX programs.