Reinforcement Learning: An Introduction, Richard S. Sutton and Andrew G. Barto, 2018 (A Bradford Book, The MIT Press) - This classic textbook offers a comprehensive theoretical discussion of policy gradient methods, covering the direct parameterization of policies and their advantages over value-based approaches.
UCL Course on RL - Lecture 6: Policy Gradient, David Silver, 2015 - This lecture provides a clear explanation of policy gradient methods, including the representation of policies with neural networks for both discrete and continuous action spaces, and the policy gradient theorem.