Applications of optimal control to dynamical systems [615241]

Applications of optimal control to dynamical systems
Aurelia Florea, Lars-Erik Persson
April 11, 2019
Abstract
In the present paper we study the optimal control problems using Lie geometric methods. We
apply the Pontryagin Maximum Principle at the level of a new working space called Lie algebroid,
which is an holonomic distribution of the tangent bundle. Finally, we give an application of driftless
control a¢ ne systems.
MSC2010: 49J15, 93B05, 93B27, 70H05, 17B66
Keywords: control a¢ ne systems, controllability, dynamical systems, optimal control, Hamilton-
Jacobi-Bellman equations, Lie geometric methods.
1 Introduction
In recent years signi…cant applications of dynamical systems and control theory have been witnessed
in diversed areas such as …nance sciences, social sciences, engineering, management and physics. There
are many prominent areas of systems and control theory that include systems governed by linear and
nonlinear ordinary di¤erential equations. Also, the Lie geometric methods have been applied successfully
in di¤erent domains of research using dynamical systems or optimal control theory. Our purpose is
the study of Lagrangian systems with some external holonomic or nonholonomic constraints. These
systems have a wide application in many di¤erent areas as optimal control theory, di¤erential equations
or dynamical systems. In the last decades, the Lie geometric methods in the control theory have been
applied by many authors (see for instance [1, 2, 4, 5, 6, 9, 10]). An important method in the geometric
approach is given by Pontryagin’ s Maximum Principle. A curve c(t) = (x(t);u(t))is an optimal trajectory
if there exists a lifting of x(t)to the cotangent bundle (x(t);p(t))satisfying the Hamilton-Jacobi-Bellman
equations.
In this paper we show that the framework of Lie algebroids is better than the cotangent bundle in the
study of driftless control a¢ ne systems with holonomic distribution and positive homogeneous costs. In
the second section the known results about Lie geometric methods in optimal control theory are presented,
including the controllability problems in the case of holonomic and nonholonomic distributions. Also, we
give only the necessary notions about Lie algebroids and present the relation between the Hamiltonian
functionHon dual Lie algebroid Eand the Hamiltonian function Hon the cotangent bundle TM.
Finally, we give an application of driftless control a¢ ne system with positive homogeneous cost, which is
1

more general than the quadratic cost and show that the Hamilton-Jacobi-Bellman equations, provided by
Pontryagin Maximum Principle on cotangent bundles, lead to a very complicated system of di¤erential
equations. Moreover, it is very di¢ cult to …nd the Hamiltonian function without dependence on control
variables. For these reasons we will use a di¤erent approach considering the framework of Lie algebroids.
However, we prove that the distribution generated by vector …elds is holonomic and it determines a
foliation in three dimensional space. In the last part of the paper we …nd the complete solution of the
problem using the framework of Lie algebroids.
2 Methods used in Optimal Control
We consider Ma di¤erentiable, smooth n-dimensional manifold. A control system given by di¤erential
equations, depending on some parameters has the form
dxi
dt=gi(x;u);
wherex2Mis the state of the system and u2URmrepresents the control variables. For x0andx1
two points of M, an optimal control problem means to …nd the trajectories of our control system which
connectsx0andx1and minimizing the Lagrangian
minZT
0L(x(t);u(t))dt; x (0) =x0; x(T) =x1;
The Pontryagin’ s Maximum Principle leads to the di¤erential equations of …rst order, necessary for the
optimal solutions. For each optimal trajectory, c(t) = (x(t);u(t)), it gives a lift on the cotangent bundle
(x(t);p(t))satisfying Hamilton-Jacobi-Bellman equations. The Hamiltonian has the form
H(x;p;u ) =hp;g(x;u)iL(x;u); p2TM;
while the maximization condition with respect to the control variables u
H(x(t);p(t);u(t)) = max
vH(x(t);p(t);v);
which yields@H
@u= 0. The extreme trajectories satisfy the equations
_x=@H
@p;_p=@H
@x: (1)
2.1 Controllability of A¢ ne Systems
We remind that a control a¢ ne system has the form [6]
_x=X0(x) +mX
i=1uiXi(x); (2)
2

wherex= (x1;:::;xn)are local coordinates on a smooth ndimensional manifold M,u(t) = (u1(t);:::;um(t))
2URm,mnandX0;X1:::Xmare smooth vector …elds on M. The system is controllable if for any
two points x0andx1onMthere exists a …nite Tand an admissible control u: [0;T]!Usuch that for
xsatisfyingx(0) =x0we have that x(T) =x1. Hence, the system is controllable if for any two states
x0,x1, there exists a solution curve of (2) connecting x0tox1. A distribution
on the manifold Mis a
map which assigns to each point in Ma subspace of the tangent space at this point x!
(x)TxM:
The distribution
is locally …nitely generated if there is a family of vector …elds fXigi=1;mwhich spans

, i.e.
(x) =spanfX1(x);:::;Xm(x)gTxM. The distribution
has dimension kifdim
(x) =k,
for all points xinM. The Lie bracket of two vector …elds is given by
[f;h](x) =@h
@x(x)f(x)@f
@x(x)h(x);
A distribution
onMis said to be involutive if for any x2Mwe have
f(x);h(x)2
(x))[f;h] (x)2
(x):
If the involutive distribution is generated by vector …elds fXigi=1;mthen it results
[Xi;Xj] (x) =mX
k=1Lk
ij(x)Xk(x):
We know that a foliation fSg2AofMis a partition of M=[
2ASofMinto disjoint connected
(immersed) submanifolds S, called leaves. A distribution
of constant dimension on Mis called
integrable (holonomic) if there exists a foliation fSg2AonMwhose tangent bundle is
, that is
TxS=
(x), whereSis the leaf passing through x. The Frobenius theorem says that if a distribution

has constant dimension, then
is integrable if and only if
is involutive. The distribution
=
spanfX1;:::;XmgonMis said to be bracket generating if the iterated Lie brackets
Xi;[Xi;Xj];[Xi;[Xj;Xk]];


;1i;j;km;
span the tangent space TMofMat every point. Using the Lie brackets of vector …elds, we construct
the ‡ ag of subsheaves


2



r


TM
with

2=
+ [
;
];:::;
r+1=
r+ [
;
r]
where
[
;
r] =spanf[X;Y ] :X2
; Y2
rg:
If there exists an r2such that
r=TM, we say that
is a bracket generating distribution and r
is called the step of the distribution
. In this case the distribution
is not integrable and is called
nonholonomic. This condition is also known as strong Hörmander condition , orLie algebra rank condition .
3

Ifr= 2the distribution is called strong bracket generating . In the following we consider the driftless
control a¢ ne system ( X0= 0) in the form
_x=mX
i=1uiXi(x): (3)
The vector …elds Xi; i=1;m, generate a distribution
onMwhich is assumed to be connected,
such that the rank of
is constant. The Chow-Rashevsky theorem says that if the distribution

=spanfX1;:::;Xmgis bracket generating (nonholonomic), then the driftless control a¢ ne system
is controllable. If
is not bracket generating and is integrable (holonomic) then the system is not con-
trollable and
determines a foliation on Mwith the property that any curve is contained in a single
leaf of the foliation, and the restriction of
to each leaf of the foliation is bracket generating. We will
study in this paper the case of holonomic distributions. Next, we will present some notions about Lie
algebroids, which are useful in the study of driftless control a¢ ne systems.
2.2 Preliminaries on Lie Algebroids
We consider Mbe a real,C1-di¤erentiable, n-dimensional manifold and (TM;M;M)its tangent bundle.
A vector bundle is a triple (E;;M )whereEandMare manifolds, called the total space and the base
space, and the map :E!Mis a surjective submersion. Using the paper [8] we know:
De…nition 1 A Lie algebroid over a manifold Mis a triple (E;[
;
]E;), where (E;;M )is a vector
bundle of rank moverM;which satis…es the conditions:
a)C1(M)-module of sections (E)is equipped with a Lie algebra structure [
;
]E.
b):E!TM is a bundle map, called the anchor, which induces a Lie algebra homomorphism from
the Lie algebra of sections ((E);[
;
]E)to the Lie algebra of vector …elds ((M);[
;
])which satis…es the
Leibnitz rule
[s1;fs2]E=f[s1;s2]E+ ((s1)f)s2;8s1;s22(E); f2C1(M): (4)
Also, it results:
1[
;
]Eis aR-bilinear operation,
2[
;
]Eis skew-symmetric, i.e.
[s1;s2]E=[s2;s1]E;8s1;s22(E);
3[
;
]Everi…es the Jacobi identity
[s1;[s2;s3]E]E+ [s2;[s3;s1]E]E+ [s3;[s1;s2]E]E= 0;
4

Iffis a function on M, thendf(x)2E
xis given byhdf(x);ai=(a)f, for8a2Ex. For!2Vk(E)
the exterior derivative dE!2Vk+1(E)has the form
dE!(s1;:::;sk+1) =k+1X
i=1(1)i+1(si)!(s1;:::;^si;:::;sk+1) +
+X
1i<jk+1(1)i+j!([si;sj]E;s1;:::;^si;:::;^sj;:::sk+1):
wheresi2(E),i=1;k+ 1, and the hat over an argument means the absence of the argument. We
have that (dE)2= 0. For the local coordinates (xi)on an open UMand a local basis fsgof the
sections of the bundle 1(U)!Uwe have the local coordinates (xi;y)onE. The local functions
i
(x),L

(x)onMgiven by
(s) =i
@
@xi;[s;s]E=L

s
; i=1;n; ;;
=1;m;
are called the structure functions of Lie algebroids. Let us consider a control system on the Lie algebroid
(E;[
;
]E;)(see [7]) with the control space :A!Mwe mean a section ofEalong. A trajectory
of the system is an integral curve of the vector …eld (). Given the cost function L2C1(A), we have
to minimize the integral of Lover the set of those system trajectories which satisfy certain boundary
conditions. The Hamiltonian function H2C1(EMA)is de…ned by
H(;u) =h;(u)iL (u);
whereas the associated Hamiltonian control system His given by the symplectic equation on Lie alge-
broid
iH!E=dEH:
where!Eis the canonical symplectic form. The critical trajectories are given by [7]
@H
@uA= 0;dxi
dt=i
@H
@;d
dt=i
@H
@xi
L

@H
@: (5)
We can associate to any Lagrangian L:E!Ron Lie algebroids E;a Lagrangian LonImTM
de…ned by
L(v) =fL(u)ju2Ex; (u) =vg;
wherev2(Im)xTxM,x2M.
From [9] we have:
Theorem 2 The relation between the Hamiltonian function Hon the cotangent bundle TMand the
Hamiltonian function Hon the dual Lie algebroid Eis given by
H(p) =H(?(p));  =?(p); p2T
xM; 2E
x: (6)
5

Proof. The Fenchel-Legendre dual of Lagrangian Lis the Hamiltonian Hgiven by
H(p) = sup
vfhp;viL(v)g= sup
vfhp;viL (u);(u) =vg
= sup
ufhp;(u)iL (u)g= sup
ufh?(p);uiL (u)g=H(?(p));
and we get
H(p) =H();  =?(p);
or locally
=i
pi; (7)
u t
3 Applications
Let us consider the following driftless control a¢ ne system with positive homogeneous cost:
8
<
:_x1=u2
_x2=u1+u2x2
_x3=u1+u2x3(8)
min
u(
)ZT
0q
u2
1+u2
2+"u1
dt; 0"<1;
We are looking for the optimal trajectories starting from the point (0;1;0)tand parameterized by
arclength (minimum time problem) and free endpoint. The system can be written in the form
_x=u1X1+u2X2; x =0
@x1
x2
x31
A2R3; X1=0
@0
1
11
A; X2=0
@1
x2
x31
A
min
u(
)RT
0F(u(t))dt;F(u) =p
(u1)2+ (u2)2+"u1;0"<1(9)
The vector …elds are given by
X1=@
@x2+@
@x3; X 2=@
@x1+x2@
@x2+x3@
@x3;
The Lie bracket is
[X1;X2] =@
@x2+@
@x3;@
@x1+x2@
@x2+x3@
@x3
=X1:
and it results that the associated distribution
=spanfX1;X2gis holonomic and has the constant rank
2. Moreover, from the system (8) we obtain
_x2_x3= _x1(x2x3);
6

which yields
lnx2x3=x1+c: (10)
(cis a constant) and it results that
determines a foliation on R3given by the surfaces (10). In order to
solve this optimal control problem we can use the Pontryagin Maximum Principle on the cotangent bundle.
The Lagrangian has the form L=1
2F2(for minimum time problem, every minimizer parametrized by
arclength, or constant speed F= 1is also a minimizer of the so called energy cost L=1
2F2) and we get
the Hamiltonian
H(u;x;p ) =pi_xiL=p1u2+p2(u1+u2x2) +p3(u1+u2x3)1
2p
(u1)2+ (u2)2+"u12
;
The Hamilton-Jacobi-Bellman equation@H
@ui= 0leads to the following system
8
>><
>>:p2+p3q
(u1)2+ (u2)2+"u1
"+u1p
(u1)2+(u2)2
= 0
p1+p2x2+p3x3q
(u1)2+ (u2)2+"u1
u2p
(u1)2+(u2)2
= 0(11)
and is di¢ cult to …nd the Hamiltonian Hwithout dependence on the control variables. For this reason
we will use a di¤erent approach, involving the framework of Lie algebroids.
In order to use the framework of Lie algebroids, we consider E=
(holonomic distribution with
constant rank), the anchor :E!TMis the inclusion and [;]Ethe induced Lie bracket. In the case of
previous example, the anchor has the components
i
=0
@0 1
1×1
1×21
A;
and we consider the Lagrangian function given by
L=1
2p
(u1)2+ (u2)2+"u12
:
Using the result from [3] we can …nd the Hamiltonian on Egiven by
H() =1
2 s
(1)2
(1"2)2+(2)2
1"2"1
1"2!2
: (12)
Using (7) we can calculate the Hamiltonian HonTMin the form H(x;p) =H(); =?(p), where
1
2
=0 1 1
1x1x20
@p1
p2
p31
A:
7

We get that
1=p2+p3;
2=p1+p2x1+p3x2;
and it results the Hamiltonian on the cotangent bundle
H(x;p) =1
2 s
(p2+p3)2
(1"2)2+(p1+p2x1+p3x2)2
1"2"(p2+p3)
1"2!2
: (13)
Unfortunately, with H(x;p)from (13) the Hamilton’ s equations (1) on TMlead to a very complicated
system of di¤erential equations.
_x1=@H
@p1= s
(p2+p3)2
(1"2)2+(p1+p2x1+p3x2)2
1"2"(p2+p3)
1"2!p1+p2x1+p3x2
1"2r
(p2+p3)2
(1"2)2+(p1+p2x1+p3x2)2
1"2;
_x2=@H
@p2= s
(p2+p3)2
(1"2)2+(p1+p2x1+p3x2)2
1"2"(p2+p3)
1"2!0
BB@(p2+p3)
(1"2)2+(p1+p2x1+p3x2)x1
1"2
r
(p2+p3)2
(1"2)2+(p1+p2x1+p3x2)2
1"2"
1"21
CCA;
_x3=@H
@p3= s
(p2+p3)2
(1"2)2+(p1+p2x1+p3x2)2
1"2"(p2+p3)
1"2!0
BB@(p2+p3)
(1"2)2+(p1+p2x1+p3x2)x2
1"2
r
(p2+p3)2
(1"2)2+(p1+p2x1+p3x2)2
1"2"
1"21
CCA;

p1=@H
@x1= s
(p2+p3)2
(1"2)2+(p1+p2x1+p3x2)2
1"2"(p2+p3)
1"2!(p1+p2x1+p3x2)p2
1"2r
(p2+p3)2
(1"2)2+(p1+p2x1+p3x2)2
1"2

p2=@H
@x2= s
(p2+p3)2
(1"2)2+(p1+p2x1+p3x2)2
1"2"(p2+p3)
1"2!(p1+p2x1+p3x2)p3
1"2r
(p2+p3)2
(1"2)2+(p1+p2x1+p3x2)2
1"2

p3=@H
@x3= 0)p3=a=ct:
8

For this reason, we will use the geometric model of a Lie algebroid. From the relation [X;X] =
L

X
we obtain the non-zero components L1
12= 1; L1
21=1while from (3) we deduce that
_x1=@H
@2;_x2=@H
@1+x2@H
@2;_x3=@H
@1+x3@H
@2;
_1=1@H
@2;_2=1@H
@1;
where
@H
@1=
1 +"2

1
(1"2)2"q
(1)2
(1"2)2+(2)2
1"2
1"2"2
1
(1"2)3q
(1)2
(1"2)2+(2)2
1"2;
@H
@2=2
1"2"12
(1"2)2q
(1)2
(1"2)2+(2)2
1"2:
The form of the last relations leads to the following change of variables
1(t) = (1"2)r(t)sech(t); 2(t) =p
1"2r(t) tanh(t): (14)
where
sinh=ee
2;cosh=e+e
2;tanh=sinh
cosh;sech=1
cosh:
In these conditions we have s
(1)2
(1"2)2+(2)2
1"2=jrj;
and the di¤erential equations
_1=1@H
@2;
with the relations (14) yields
p
1"2_r
r_tanh
=r(tanh+"sechtanh): (15)
Also, from the equation
_2=1@H
@1;
and (14) we get
p
1"2_r
rtanh+_sech2
=r((1 +")2sech2"sech"sech3): (16)
9

Now, reducing _and_r
rfrom the equations (15) and (16), we obtain
p
1"2_r=r2"sechtanh("sech1);
and p
1"2_=r("sech1)2:
The last two equations lead to
_r
_=r"sechtanh
"sech1;
and respectively to
dr
r="sechtanh
"sech1d;
with the solution
lnjrj=ln("sech1)lnc:
Therefore
jrj=1
c("sech1):
Since the optimal trajectories are parameterized by arclength, the conclusion corresponds exactly to the
1=2level of the Hamiltonian and we have
H=r2
2(1"sech)2=1
2c2:
Now,c=1and
r=1
"sech1:
The equation
_1=1_x1;
implies that
x1() = lnc1(1"sech)
(1"2)sech; c 12R:
Since we are looking for the trajectories starting from the point (0;1;0)t, we havex1(0) = 0 and
lnc1
1 +"= 0)c1= 1 +";
which leads to
x1() = ln1"sech
(1")sech= lncosh"
1":
We obtain also that
_2=1
_x2x2@H
@2
=1_x2+x2_1;
10

-1.0-0.50.00.502401234and, consequently, 2=1×2+c2. Further,
x2() =sinhp
1"2c2(1"sech)
(1"2)sech:
Fromx2(0) = 1 we obtain that c2= 1 +"and this yields
x2() =sinhp
1"2+cosh"
1":
In the same way we get
x3() =sinhp
1"2c3(1"sech)
(1"2)sech:
Fromx3(0) = 0 we obtain that c3= 0and it results
x3() =sinhp
1"2:
Using (8) we have u2= _x1,u1= _x3u2x3= _x2u2x2and by direct computation, we obtain the control
variables
u2() =sinh
cosh"; u 1() =1p
1"21"cosh
cosh":
If"= 0we obtain the case of driftless control a¢ ne systems with quadratic cost with the solution
x1(t) = ln cosht; x2(t) = sinht+ cosht; x3(t) = sinht;
11

and control variables
u2(t) = tanht; u 1(t) =secht:
Conclusions . In this paper we treat some topics of dynamical systems using Lie geometric methods.
In the case of driftless control a¢ ne systems with holonomic distribution and positive homogeneous cost
we proved that the framework of Lie algebroids is better than cotangent bundles in order to apply the
Pontryagin Maximum Principle and …nd the optimal solution. As futher developments, we try to use
the framework of Lie algebroids in the case of nonholonomic distribution (in particular, strong bracket
generating) and characterize the solutions using the geometry of Lie algebroids.
Acknowledgement.
This research was carried out while one of the authors was visiting the University of North Carolina
at Charlotte (UNCC), USA and was supported by the Horizon 2020 – 2017 RISE – 777911 project. It is a
pleasure to thank to Professor Douglas Shafer (UNCC) for his hospitality and for important discussions
which we had during my stay in Charlotte.
References
[1] A. Agrachev, Y.L. Sachkov, Control theory from the geometric viewpoint . Encyclopedia of Mathe-
matical Sciences, 87, Control Theory and Optimization, II. Springer-Verlag, Berlin, 2004.
[2] R. Brocket, Lie algebra and Lie groups in control theory , Geometrical Methods in Control Theory,
Dordrecht, (1973), 43– 82.
[3] D. Hrimiuc, H. Shimada, On the L-duality between Lagrange and Hamilton Manifold , Nonlinear
World, 3 (1996), 613-641.
[4] D. Hrimiuc, L. Popescu, Geodesics of sub-Finslerian Geometry , In: Di¤erential Geometry and its
Applications, Proc. Conf. Prague 2004, Charles Univ. (2005) 59-67.
[5] V. Jurdjevic, Geometric Control Theory , Cambridge University Press, 1997.
[6] S. M. LaValle, Planning Algorithms , Cambridge University Press, 2006.
[7] E. Martinez, Reduction in optimal control theory , Rep. Math. Phys. 53 (2004), 79– 90.
[8] K. Mackenzie, Lie groupoids and Lie algebroids in di¤erential geometry , London Mathematical So-
ciety Lecture Note Series 124, 1987.
[9] L. Popescu, Mechanical structures on Lie algebroids , Mediterranean Journal of Mathematiucs,
no.15(4), (2018), 154.
[10] A. Szatkowski, Geometric Methods in Dynamical Systems Modelling: Electrical, Mechanical and
Control Systems , Systems Analysis Modelling Simulation, vol.42, 11 (2002), 1659-1707.
12

Author’ s address:
University of Craiova
Faculty of Sciences
Department of Applied Mathematics
Al. I. Cuza, Street no. 13
Craiova, Romania
e-mail: aurelia_‡ orea@yahoo.com
13