The control barrier function (CBF) method is becoming a popular tool that transforms nonlinear constrained optimal control problems into a sequence of quadratic programs (QPs). In this tutorial paper, we show how to employ machine learning techniques to ensure the feasibility of these QPs, which is a challenging problem, especially for high relative degree constraints where high order CBFs (HOCBFs) are employed. We present two complementary learning approaches: (i) parameter learning for regular unsafe sets; (ii) sampling learning for irregular unsafe sets, where “regularity” of an unsafe set is formally defined in terms of the dependence of QP feasibility on initial system conditions. The first approach compensates for the myopic nature of the QP-based approach by parameterizing the HOCBFs and using machine learning techniques to select parameters that maximize a feasibility robustness metric related to system performance. This feasibility robustness metric measures the extent to which QP feasibility is maintained in the presence of time-varying and unknown unsafe sets. The sampling learning approach addresses “irregular” unsafe sets in which the problem feasibility heavily depends on the initial conditions. This approach learns a new feasibility constraint that guarantees the QP feasibility, and it is then enforced by another HOCBF added to the QPs. The accuracy of the learned feasibility constraint can be recursively improved by the proposed recurrent training algorithm. We demonstrate the advantages of the proposed learning approaches to constrained optimal control problems with specific focus on a robot control problem and on autonomous driving in an unknown environment.