Convex Optimization, Stephen Boyd and Lieven Vandenberghe, 2004 (Cambridge University Press) - A standard reference for convex optimization, presenting foundational theory for constrained problems, feasible regions, and optimality conditions.
Numerical Optimization, Jorge Nocedal and Stephen J. Wright, 2006 (Springer)DOI: 10.1007/978-0-387-40065-5 - A comprehensive text on numerical optimization, addressing theory and algorithms for both unconstrained and constrained problems.
Nonlinear Programming, Dimitri P. Bertsekas, 2016 (Athena Scientific) - A detailed textbook on nonlinear programming, providing rigorous analysis of constrained optimization theory and methods. 3rd edition.
CS229 Lecture Notes: Constrained Optimization, KKT conditions, Duality, Andrew Ng and Anqi Liu, 2018 (Stanford University) - Stanford CS229 lecture notes introducing constrained optimization basics, problem formulation, and leading into KKT conditions and duality, highly pertinent to machine learning.