Before tackling more advanced techniques, it's essential to ensure our understanding of the core Reinforcement Learning (RL) principles is solid. This chapter serves as a focused refresher on the foundational concepts that underpin the field.

We will briefly revisit:

• The standard RL framework involving agents, environments, states, actions, and rewards.
• Markov Decision Processes (MDPs) as the mathematical formalization for sequential decision problems.
• The central role of value functions, specifically the state-value function $V(s)$ and the action-value function $Q(s, a)$.
• The Bellman equations, which form the basis for many RL algorithms: $V(s) = \mathbb{E}[R_{t+1} + \gamma V(S_{t+1}) | S_t = s]$
• Classic tabular methods like Q-Learning and SARSA, including their update mechanisms.

Finally, we will examine the inherent limitations of these tabular approaches, particularly when dealing with large or continuous state spaces. Understanding these constraints sets the stage for the function approximation methods explored in subsequent chapters. This review ensures we have a common ground before proceeding to Deep Q-Networks and Policy Gradient methods.