The aim of this work is to study several problems of the calculus of variations, where the dynamics of the state function is given by a generalized fractional derivative. This derivative combines two well-known concepts: fractional derivative with respect to another function and fractional derivative of variable order. We present the Euler–Lagrange equation, which is a necessary condition that every optimal solution of the problem must satisfy. Other problems are also studied: with integral and holonomic constraints, with higher order derivatives, and the Herglotz variational problem.

In this work, we extended fractional optimal control (OC) theory by proving a version of Pontryagin's maximum principle and establishing sufficient optimality conditions for an OC problem. The dynamical system constraint in the OC problem under investigation is described by a generalized fractional derivative: the left-sided Caputo distributed-order fractional derivative with an arbitrary kernel. This approach provides a more versatile representation of dynamic processes, accommodating a broader range of memory effects and hereditary properties inherent in diverse physical, biological, and engineering systems.

In this paper, we expanded the concept of tempered fractional derivatives within both the Riemann-Liouville and Caputo frameworks, introducing a novel class of fractional operators. These operators are characterized by their dependence on a specific arbitrary smooth function. We then investigated the existence and uniqueness of solutions for a particular class of fractional differential equations, subject to specified initial conditions. To aid our analysis, we introduced and demonstrated the application of Picard's iteration method. Additionally, we utilized the Gronwall inequality to explore the stability of the system under examination. Finally, we studied the attractivity of the solutions, establishing the existence of at least one attractive solution for the system. Throughout the paper, we provide examples and remarks to support and reinforce our findings.

This paper examines tempered fractional derivatives with respect to a kernel function, extending classical operators like Caputo and tempered derivatives. We investigate fractional differential equations (FDEs) that incorporate these generalized derivatives, focusing on the existence and uniqueness of solutions for boundary value problems. Using fixed-point theorems, we establish conditions for the existence and uniqueness of solutions. Additionally, we analyze the stability of these equations under different criteria. Our approach addresses inaccuracies in previous studies and contributes to the broader theory of fractional equations with generalized derivatives.