J. Oper. Res. Soc. China (2014) 2:481–498 DOI 10.1007/s40305-014-0064-9

Stochastic Nash Games for Markov Jump Linear Systems with State- and Control-Dependent Noise Huai-Nian Zhu • Cheng-Ke Zhang • Ning Bin

Received: 23 June 2013 / Revised: 13 November 2014 / Accepted: 17 November 2014 / Published online: 23 December 2014 Ó Operations Research Society of China, Periodicals Agency of Shanghai University, and SpringerVerlag Berlin Heidelberg 2014

Abstract This paper investigates Nash games for a class of linear stochastic systems governed by Itoˆ’s differential equation with Markovian jump parameters both in finite-time horizon and infinite-time horizon. First, stochastic Nash games are formulated by applying the results of indefinite stochastic linear quadratic (LQ) control problems. Second, in order to obtain Nash equilibrium strategies, crosscoupled stochastic Riccati differential (algebraic) equations (CSRDEs and CSRAEs) are derived. Moreover, in order to demonstrate the validity of the obtained results, stochastic H2/H? control with state- and control-dependent noise is discussed as an immediate application. Finally, a numerical example is provided. Keywords Stochastic differential games  Markov jump linear systems  indefinite stochastic LQ control problem

This research was supported by the National Natural Science Foundation of China (No. 71171061), China Postdoctoral Science Foundation (No. 2014M552177), the Natural Science Foundation of Guangdong Province (No. S2011010004970), the Doctors Start-up Project of Guangdong University of Technology (No. 13ZS0031), and the 2014 Guangzhou Philosophy and Social Science Project (No. 14Q21). H.-N. Zhu (&)  C.-K. Zhang School of Economics & Commerce, Guangdong University of Technology, Guangzhou 510520, China e-mail: [email protected] C.-K. Zhang e-mail: [email protected] N. Bin School of management, Guangdong University of Technology, Guangzhou 510520, China e-mail: [email protected]

123

482

H.-N. Zhu et al.

1 Introduction Over the last four decades, Nash differential games have been extensively investigated [30]. It has attracted much attention and has been widely applied to various fields, such as control theory (see [2, 7, 15, 19], and reference therein), management science and economics [9], ecology [34], etc. Recent advances in stochastic LQ control problems have allowed us to expand the study on the Nash games for stochastic systems with state- and control-dependent noise (see [5, 6, 17, 18]). On the other hand, the systems with Markovian jump are frequently used to describe the evolution of some physical processes subject to abrupt variations of the parameters. Partially, this is due to the fact that often dynamic systems are inherently vulnerable to component failures or repairs, changing of subsystem interconnections, or abrupt variations of the nominal operating conditions. There exists a very rich list of references of articles and books dealing with control problems for this class of systems (see, e.g., [3, 11, 12, 22] and the references therein). Now, this kind of system has proven being useful in describing hybrid dynamics arising in electric power systems [21], communications systems [1], control of nuclear power plants [27], manufacturing systems [4, 14], and economic systems (see [8, 13, 20, 37, 38], etc). Recently, Yong [35], Mou and Yong [24], McAsey and Mou [23], and Zhu and Zhang [39] investigated a special kind of stochastic differential games for Itoˆ systems with stateand control-dependent noise. Stochastic differential games were recently studied by many researchers, such as Wang and Yu [31, 32], Yu [36], Hui and Xiao [16], and Xu and Zhang [33], with the backward stochastic differential equation approach and stochastic maximum principle to obtain the Nash strategies. In Song, Yin, and Zhang [29], numerical methods using Markov chain approximation techniques were developed for zero-sum stochastic differential games of regime-switching diffusions. In Pan and Basar [26], the existence of a stabilizing solution for a system of game-theoretic algebraic Riccati equations associated to a linear system with Markov jump perturbations was studied in connection with piecewise deterministic differential games; and in Dragan and Morozan [10], several properties of the stabilizing solution of a class of systems of Riccati-type differential equations with indefinite sign associated to controlled systems described by differential equations with Markovian jumps were discussed. However, we note that the results above focused on stochastic systems with only state-dependent noise. However, in some practical models, not only the state but also the control input maybe corrupted by noise. For example, a practical model with the control input-dependent noise can be found in Qian and Gajic [28], which comes from the stochastic power control in CDMA systems. In addition, in the field of mathematical finance, an optimal portfolio selection problem is modeled by a stochastic Itoˆ equation with state- and control-dependent noise, see Example 11.2.5 of Øksendal [25]. Therefore, stochastic Nash games for Markov jump linear systems with state- and control-dependent noise deserve further study. Inspired by this, we investigate the Nash games for a class of continuous-time Markov jump linear systems with state- and control-dependent noise, which are expressed by the Itoˆ stochastic differential equations. The main contributions of this paper are as follows. First, finite time horizon stochastic Nash games are investigated by applying the results of indefinite stochastic LQ control problems with Markovian jumps. Then, we extend the

123

Stochastic Nash Games

483

results into infinite-time horizon case. Moreover, as an important application, stochastic H2/H? control for Markov jump linear systems with state- and controldependent noise is discussed. Finally, in order to demonstrate the validity of the obtained results, a numerical example is provided. The rest of the paper is organized as follows: Sect. 2 discusses stochastic Nash games in finite-time horizon; Sect. 3 extends the results of finite-time horizon stochastic Nash games into infinite-time horizon case; Sect. 4 provides the application to stochastic H2/H? control; and Sect. 5 concludes the paper with some remarks. For convenience, we will make use of the following notations throughout this paper. The notations used in this paper are fairly standard. A0 : transpose of a matrix A. In : the n  n identity matrix. k  k: the Euclidean norm of a matrix. Efjrt ¼ ig : the conditional expectation operator with respect to the event frt ¼ ig. vA : indicator function of a set A. Rn : the n-dimensional Euclidean space. Rnm : the set of all n  m matrices; Mln;m : space of all A ¼ ðAð1Þ; Að2Þ;    ; AðlÞÞ with AðiÞ being n  m matrix, i ¼ 1; 2;    ; l. Mln :¼ Mln;n . Sn : space of all n  n symmetric matrices. Sln : space of all A ¼ ðAð1Þ; Að2Þ;    ; AðlÞÞ with AðiÞ being n  n symmetric matrix, i ¼ 1; 2;    ; l. 2 Finite-Time Horizon Stochastic Nash Games 2.1 Problem Formulation Throughout this paper, let ðX; F; fFt jt > 0g; PÞ be a given filtered probability space where exists a standard one-dimensional Wiener process fWðtÞjt > 0g and a right continuous homogeneous Markov chain frt jt > 0g with state space N ¼ f1; 2;    ; lg. In a similar assumption of the existing results, it is supposed that rt is independent of WðtÞ. Furthermore, it is also assumed that the Markov process rt has the transition probabilities given by  pij D þ oðDÞ ; i 6¼ j; Pr½rtþD ¼ jjrt ¼ i ¼ ð2:1Þ 1 þ pii D þ oðDÞ ; i ¼ j; where pij > 0; i 6¼ j; pii ¼ 

l P

pij . Ft stands for the smallest r-algebra gener-

j¼1;j6¼i

ated by process WðsÞ; rs ; 0 6 s 6 t, i.e., Ft ¼ rfWðsÞ; rs j0 6 s 6 tg. Consider the following linear stochastic differential equations subject to Markovian jumps defined by 8 > < dxðtÞ ¼ ½Aðrt ÞxðtÞ þ B1 ðrt Þu1 ðtÞ þ B2 ðrt Þu2 ðtÞdt þ½Cðrt ÞxðtÞ þ D1 ðrt Þu1 ðtÞ þ D2 ðrt Þu2 ðtÞdWðtÞ; ð2:2Þ > : xðsÞ ¼ y 2 Rn ; where xðtÞ 2 Rn is the state variable, uk ðtÞ 2 Rmk is control strategy taken by player Pk ; k ¼ 1; 2.

123

484

H.-N. Zhu et al.

Given a fixed ðs; yÞ 2 ½0; T  Rn , let Uk ½0; T, k ¼ 1; 2, be the set of the Rmk valued, square integrable processes adapted with the r-field generated by WðtÞ; rt , respectively. In the present paper, we suppose s\T to guarantee ½s; T is an interval. Associated with each ðu1 ; u2 Þ 2 U½0; T  U1 ½0; T  U2 ½0; T, the cost performance Jk ðu1 ; u2 ; s; y; iÞ of player Pk is defined by Jk ðu1 ; u2 ; s; y; iÞ ¼ 9 8 2 3 xðtÞ > > = > ; : s u2 ðtÞ 2

Qk ðrt Þ Lk1 ðrt Þ Mk ðrt Þ ¼ 4 L0k1 ðrt Þ Rk1 ðrt Þ L0k2 ðrt Þ 0

3 Lk2 ðrt Þ 0 5; k ¼ 1; 2: Rk2 ðrt Þ

ð2:3Þ

In (2.2) and (2.3), Aðrt Þ ¼ AðiÞ; Bk ðrt Þ ¼ Bk ðiÞ; Cðrt Þ ¼ CðiÞ; Dk ðrt Þ ¼ Dk ðiÞ and Mk ðrt Þ ¼ Mk ðiÞ whenever rt ¼ i; i 2 N. Moreover, whenever rT ¼ i; Hk ðrT Þ ¼ Hk ðiÞ; k ¼ 1; 2. Here the matrices mentioned above are given real matrices of suitable sizes. Referring to Li and Zhou [17], the value function Vk ðs; y; iÞ is defined as Vk ðs; y; iÞ ¼ inf Jk ðuk ; us ; s; y; iÞ; uk 2Uk

k; s ¼ 1; 2; k 6¼ s; i 2 N;

ð2:4Þ

where us is the optimal control strategy of player Ps ; s ¼ 1; 2. Since the symmetric matrices 2 3 Qk ðiÞ Lk1 ðiÞ Lk2 ðiÞ 6 7 0 5 Mk ðiÞ ¼ 4 L0k1 ðiÞ Rk1 ðiÞ L0k2 ðiÞ 0 Rk2 ðiÞ are allowed to be indefinite, the above optimization problem is referred to as indefinite stochastic Nash games. Definition 2.1 The stochastic Nash equilibrium strategy pair ðu1 ; u2 Þ 2 U½0; T is defined as satisfying the following conditions: J1 ðu1 ; u2 ; s; y; iÞ 6 J1 ðu1 ; u2 ; s; y; iÞ; J2 ðu1 ; u2 ; s; y; iÞ 6 J2 ðu1 ; u2 ; s; y; iÞ; Definition 2.2

8 u1 2 U1 ;

ð2:5aÞ

8u2 2 U2 ; i 2 N:

ð2:5bÞ

The indefinite stochastic Nash games (2.2)–(2.5a,b) are well posed if

1\Vk ðs; y; iÞ\ þ 1;

8ðs; yÞ 2 ½0; T  Rn ; k ¼ 1; 2; i 2 N:

An admissible triple ðx ; u1 ; u2 Þ is called optimal with respect to (w.r.t.) the initial condition ðs; y; iÞ if u1 achieves the infimum of J1 ðu1 ; u2 ; s; y; iÞ and u2 achieves the infimum of J2 ðu1 ; u2 ; s; y; iÞ.

123

Stochastic Nash Games

485

For the indefinite stochastic Nash games (2.2)–(2.5a,b), we restrict uk ðtÞ to be composed of linear feedback strategies of the form: uk ðtÞ ¼ Kk ðrt ÞxðtÞ; k ¼ 1; 2, and Kk 2 Mlmk ;n are matrix-valued functions. In the next section, we discuss the one-player case, i.e., indefinite stochastic LQ control problems [5, 6]. 2.2 One-Player Case First, one-player case is discussed. The result obtained for that particular case is used as the basis for the derivation of results for 2-player case. Consider the linear stochastic controlled system with Markovian jumps defined by  dxðtÞ ¼ ½Aðrt ÞxðtÞ þ B1 ðrt Þu1 ðtÞdt þ ½Cðrt ÞxðtÞ þ D1 ðrt Þu1 ðtÞdWðtÞ; ð2:6Þ xðsÞ ¼ y; where ðs; yÞ 2 ½0; T  Rn are the initial time and initial state, respectively. For each ðs; yÞ and u1 2 U½0; T, the associated cost is (Z     T xðtÞ 0 Q1 ðrt Þ L1 ðrt Þ xðtÞ Jðu1 ; s; y; iÞ ¼ E dt L01 ðrt Þ R11 ðrt Þ u1 ðtÞ u1 ðtÞ s

ð2:7Þ

0

þ x ðTÞH1 ðrt ÞxðTÞjrs ¼ ig; where Q1 ðrt Þ ¼ Q1 ðiÞ; R11 ðrt Þ ¼ R11 ðiÞ and L1 ðrt Þ ¼ L1 ðiÞ when rt ¼ i, and H1 ðrT Þ ¼ H1 ðiÞ whenever rT ¼ i, whereas Q1 , etc., i 2 N, are given matrices with suitable sizes. The objective of the optimal control problem is to minimize the cost function Jðu1 ; s; y; iÞ, for a given ðs; yÞ 2 ½0; T  Rn , over all u1 2 U½0; T. The value function is defined as Vðs; y; iÞ ¼ inf Jðu1 ; s; y; iÞ: u1 2U1

Note that as the symmetric matrices   Q1 ðiÞ L1 ðiÞ ; L01 ðiÞ R11 ðiÞ

ð2:8Þ

i2N

are allowed to be indefinite; and we call the above optimization problem as an indefinite LQ problem with Markovian jumps [17, 18]. Now we introduce a type of coupled Riccati differential equations associated with the LQ problems (2.6)–(2.8) and some useful lemmas that are important in our subsequent analysis. Definition 2.3 The following system of constrained differential equations (with the time argument t suppressed)

123

486

H.-N. Zhu et al.

8 l P > > _ þ PðiÞAðiÞ þ A0 ðiÞPðiÞ þ C 0 ðiÞPðiÞCðiÞ þ Q1 ðiÞ þ pij PðjÞ > PðiÞ > > > j¼1 > > >  1 < ðPðiÞB1 ðiÞ þ C 0 ðiÞPðiÞD1 ðiÞ þ L1 ðiÞÞ R11 ðiÞ þ D01 ðiÞPðiÞD1 ðiÞ  0  >  B1 ðiÞPðiÞ þ D01 ðiÞPðiÞCðiÞ þ L01 ðiÞ ¼ 0; > > > > > > PðT; iÞ ¼ H1 ðiÞ; > > : R11 ðiÞ þ D01 ðiÞPðiÞD1 ðiÞ [ 0 ; i 2 N

ð2:9Þ

is called a system of coupled stochastic Riccati differential equations (CSRDEs). Lemma 2.4 (generalized Itoˆ’s formula) [3]: Let bðt; x; iÞ and rðt; x; iÞ be given Rn valued, Ft -adapted process, i ¼ 1; 2;    ; l, and xðtÞ satisfy dxðtÞ ¼ bðt; xðtÞ; rt Þdt þ rðt; xðtÞ; rt ÞdWðtÞ: Then for given uð; ; iÞ 2 C2 ð½0; 1Þ  Rn Þ; i ¼ 1; 2;    ; l, we have EfuðT; xðTÞ; rT Þ  uðs; xðsÞ; rs Þjrs ¼ ig Z T  ½ut ðt; xðtÞ; rt Þ þ ruðt; xðtÞ; rt Þdtjrs ¼ i ; ¼E

ð2:10Þ

s

where 1 ruðt; x; iÞ ¼ b0 ðt; x; iÞux ðt; x; iÞ þ ½r0 ðt; x; iÞuxx ðt; x; iÞrðt; x; iÞ 2 l X þ pij uðt; x; jÞ: j¼1

The following lemma presents the existence condition for an optimal feedback control. Lemma 2.5 Suppose CSRDEs (2.9) admit a solution P : ½0; T ! Sln , with P ¼ ðPð1Þ; Pð2Þ;    ; PðlÞÞ, then the LQ problems (2.6)–(2.8) are well posed w.r.t. any initial ðs; yÞ 2 ½0; T  Rn . Moreover, there exists an optimal control that can be represented by the state feedback form: u1 ðtÞ ¼

l X

K1 ðiÞðtÞxðtÞvrt ¼i ; i 2 N;

ð2:11Þ

i¼1

where  1  0  K1 ðiÞ ¼  R11 ðiÞ þ D01 ðiÞPðiÞD1 ðiÞ B1 ðiÞPðiÞ þ D01 ðiÞPðiÞCðiÞ þ L01 ðiÞ are matrix-value functions with suitable sizes. Furthermore, the following value function

123

Stochastic Nash Games

487

Vðs; y; iÞ  inf Jðu1 ; s; y; iÞ ¼ y0 Pðs; iÞy; i 2 N u1 2U1

is uniquely determined by P ¼ ðPð1Þ; Pð2Þ;    ; PðlÞÞ 2 Sln : Proof Let P ¼ ðPð1Þ; Pð2Þ;    ; PðlÞÞ 2 Sln be a solution of the CSRDEs (2.9). Setting uðt; x; iÞ ¼ x0 Pðt; iÞx and applying the generalized Itoˆ’s formula (Lemma 2.4) to the linear system (2.6), we have E½x0 ðTÞPðrT ÞxðTÞ  y0 Pðrs Þyjrs ¼ i ¼ E½uðT; xðTÞ; rT Þ  uðs; xðsÞ; rs Þjrs ¼ i Z T  ruðt; xðtÞ; tÞdtjrs ¼ i ; ¼E

ð2:12Þ

s

where ruðt; x; iÞ ¼ ut ðt; x; iÞ þ bðt; x; u; iÞ0 ux ðt; x; iÞ l X 1 þ ½r0 ðt; x; u; iÞuxx ðt; x; iÞrðt; x; u; iÞ þ pij uðt; x; jÞ 2 j¼1 " # l X 0 _ 0 0 ¼ x PðiÞ þ PðiÞAðiÞ þ A ðiÞPðiÞ þ C ðiÞPðiÞCðiÞ þ pij PðjÞ x j¼1

þ

2u01 ½B01 ðiÞPðiÞ

þ

D01 ðiÞPðiÞCðiÞx

þ

u01 D01 ðiÞPðiÞD1 ðiÞu1 :

Substituting (2.12) back into (2.7), we get Z T  0 ½u1  K1 ðrt Þx0 D01 ðrt ÞPðrt ÞD1 ðrt Þ þ R11 ðrt Þ J ðu1 ; s; y; iÞ ¼ y Pðs; iÞy þ E s

½u1  K1 ðrt Þxdtjrs ¼ ig: ð2:13Þ From the definition of the CSRDEs, we have ruðt; x; iÞ þ x0 Q1 ðiÞx þ 2u01 L01 ðiÞx þ u01 R11 ðiÞu1 " 0 _ ¼ x PðiÞ þ PðiÞAðiÞ þ A0 ðiÞPðiÞ þ C0 ðiÞPðiÞCðiÞ þ Q1 ðiÞ þ

l X

# pij PðjÞ x þ 2u01 ½B01 ðiÞPðiÞ þ D01 ðiÞPðiÞCðiÞ þ L1 ðiÞx

j¼1

ð2:14Þ

þ u01 ½R11 ðiÞ þ D01 ðiÞPðiÞD1 ðiÞu1 ¼ x0 f½PðiÞB1 ðiÞ þ C0 ðrt ÞPðiÞD1 ðiÞ þ L1 ðiÞ½R11 ðiÞ þ D01 ðiÞPðiÞD1 ðiÞ1    B01 ðiÞPðiÞ þ D01 ðiÞPðiÞCðiÞ þ L01 ðiÞ x þ 2u01 ½B01 ðiÞPðiÞ þ D01 ðiÞPðiÞCðiÞ þ L01 ðiÞx þ u01 ½R11 ðiÞ þ D01 ðiÞPðiÞD1 ðiÞu1 :

123

488

H.-N. Zhu et al.

Applying the square completion technique to (2.14), we have ruðt; x; iÞ þ x0 Q1 ðiÞx þ 2u01 L01 ðiÞx þ u01 R11 ðiÞu1  ¼ ½u1  K1 ðiÞx0 R11 ðiÞ þ D01 ðiÞPðiÞD1 ðiÞ ½u1  K1 ðiÞx:

ð2:15Þ

Then the equation (2.13) can be expressed as J ðu1 ; s; y; iÞ ¼ y0 Pðs; iÞy Z T  0 0 þE ½u1  K1 ðrt Þx D1 ðrt ÞPðrt ÞD1 ðrt ÞþR11 ðrt Þ½u1  K1 ðrt Þxdtjrs ¼ i : s

ð2:16Þ From (2.16) we can see that Jðu1 ; s; y; iÞ is minimized by the control given by (2.11) with the optimal value y0 Pðs; iÞy. This completes the proof. 2.3 Stochastic Nash Equilibrium Strategies The solution of the stochastic Nash games is given below. Theorem 2.6 Suppose there exist P ¼ ðP1 ; P2 Þ : ½0; T ! Sln  Sln , with P1 ¼ ðP1 ð1Þ;    ; P1 ðlÞÞ; P2 ¼ ðP2 ð1Þ;    ; P2 ðlÞÞ that satisfy the following CSRDEs ði; j 2 NÞ. 8 l P > >  þ A0 ðiÞP1 ðiÞ þ C0 ðiÞP1 ðiÞCðiÞ  þQ 1 ðiÞ þ pij P1 ðjÞ > P_ 1 ðiÞ þ P1 ðiÞAðiÞ > > > j¼1 > > >  1 <  0  P1 ðiÞB1 ðiÞ þ C ðiÞP1 ðiÞD1 ðiÞ þ L11 ðiÞ R11 ðiÞ þ D01 ðiÞP1 ðiÞD1 ðiÞ  0  >  B ðiÞP1 ðiÞ þ D0 ðiÞP1 ðiÞCðiÞ  þ L0 ðiÞ ¼ 0; > 1 1 11 > > > > > P ðT; iÞ ¼ H ðiÞ; 1 1 > > : R11 ðiÞ þ D01 ðiÞP1 ðiÞD1 ðiÞ [ 0; i 2 N; ð2:17Þ 8 l P > > ~ þ A~0 ðjÞP2 ðjÞ þ C~0 ðjÞP2 ðjÞCðjÞ ~ þQ ~2 ðjÞ þ > P_ 2 ðjÞ þ P2 ðjÞAðjÞ pjk P2 ðkÞ > > > k¼1 > > > < P ðjÞB ðjÞ þ C~0 ðjÞP ðjÞD ðjÞ þ L ðjÞR ðjÞ þ D0 ðjÞP ðjÞD ðjÞ1 2 2 2 2 22 22 2 2 2   >  B0 ðjÞP2 ðjÞ þ D0 ðjÞP2 ðjÞCðjÞ ~ þ L0 ðjÞ ¼ 0; > 2 2 22 > > > > > P ðT; jÞ ¼ H ðjÞ; 2 2 > > : R22 ðjÞ þ D02 ðjÞP2 ðjÞD2 ðjÞ [ 0; j 2 N;

ð2:18Þ

where  1  0   þ L0 ðiÞ ; K1 ¼  R11 ðiÞ þ D01 ðiÞP1 ðiÞD1 ðiÞ B1 ðiÞP1 ðiÞ þ D01 ðiÞP1 ðiÞCðiÞ 11  1  0  ~ þ L0 ðjÞ ; K2 ¼  R22 ðjÞ þ D02 ðjÞP2 ðjÞD2 ðjÞ B2 ðjÞP2 ðjÞ þ D02 ðjÞP2 ðjÞCðjÞ 22

123

Stochastic Nash Games

489

 1 ¼ Q1 þ L12 K2 þ K0 2 L0 þ K0 2 R12 K2 ; A ¼ A þ B2 K2 ; C ¼ C þ D2 K2 ; Q 12 0 0 ~ ~ ~ A ¼ A þ B1 K1 ; C ¼ C þ D1 K1 ; Q2 ¼ Q2 þ L21 K1 þ K 1 L21 þ K0 1 R21 K1 : Denote F1 ðiÞ ¼ K1 ðiÞ; F2 ðiÞ ¼ K2 ðiÞ, then the stochastic Nash equilibrium strategy ðu1 ; u2 Þcan be represented by 8 l P >  > > F1 ðiÞðtÞxðtÞvrt ¼i ; < u1 ðtÞ ¼ i¼1 ð2:19Þ l > P >   > F2 ðiÞðtÞxðtÞvrt ¼i : : u2 ðtÞ ¼ i¼1

Furthermore, the indefinite stochastic Nash games (2.2)–(2.5a,b) is well posed (w.r.t. ðs; yÞ 2 ½0; T  Rn ), and the optimal value is determined by Vk ðs; y; iÞ ¼ inf Jk ðuk ; us ; s; y; iÞ ¼ y0 Pk ðs; iÞy; uk 2Uk

k; s ¼ 1; 2; k 6¼ s; i 2 N:

Proof These results can be proved by using the concept of Nash equilibrium described in definition 2.1 as follows. Given u2 ¼ F2 ðrt ÞxðtÞ is the optimal control strategy implemented by player P2 , player P1 facing the following optimization problem in which the cost function (2.20) is minimal at u1 ¼ F1 ðrt ÞxðtÞ. # (Z  ) "   T xðtÞ xðtÞ 0 Q 1 ðrt Þ L11 ðrt Þ 0 min E dt þ x ðTÞH1 ðrt ÞxðTÞjrs ¼ i ; F1 ðrt Þ2U1 u1 ðtÞ L011 ðrt Þ R11 ðrt Þ u1 ðtÞ s s:t:   t ÞxðtÞ þ D1 ðrt Þu1 ðtÞdWðtÞ;  t ÞxðtÞ þ B1 ðrt Þu1 ðtÞdt þ ½Cðr dxðtÞ ¼ ½Aðr xðsÞ ¼ y 2 Rn ; ð2:20Þ  1 ¼ Q1 þ ðF  Þ0 L0 þ L12 F  þ ðF  Þ0 R12 F  ; A ¼ A þ B2 F  , C ¼ C þ D2 F  . here Q 2 12 2 2 2 2 2 Note that the above optimization problem defined in (2.20) is a standard indefinite stochastic LQ problem. Applying lemma 2.5 to this optimization problem as " #   1 ðrt Þ L11 ðrt Þ Q1 L1 Q ) ; A ) A; C ) C: L01 R11 L011 ðrt Þ R11 ðrt Þ We can easily get the optimal control u1 ðtÞ ¼ F1 ðrt ÞxðtÞ:

ð2:21Þ

V1 ðs; y; iÞ ¼ y0 P1 ðs; iÞy; i 2 N:

ð2:22Þ

and the optimal value function

Similarly, we can prove that u2 ¼ F2 ðrt ÞxðtÞ is the optimal control strategy of player P2 . This completes the proof of Theorem 2.6.

123

490

H.-N. Zhu et al.

3 Infinite-Time Horizon Stochastic Nash Games 3.1 Problem Formulation In this section, we investigate the infinite-time horizon stochastic Nash games for linear Markovian jump systems with state- and control-dependent noise. In particular, infinite-time horizon stochastic Nash games for linear Markovian jump systems with state-dependent noise was considered in Zhu et al. [40]. Consider the games described by the following linear stochastic differential equation with Markovian parameter jumps  dxðtÞ ¼ ½Aðrt ÞxðtÞ þ Bðrt ÞuðtÞdt þ ½Cðrt ÞxðtÞ þ Dðrt ÞuðtÞdWðtÞ; ð3:1Þ xð0Þ ¼ x0 2 Rn ; with the cost performances Z 1    xðtÞ Jk ðu; x0 ; iÞ ¼ E ½ x0 ðtÞ u0 ðtÞ Mk ðrt Þ dtjr0 ¼ i ; k ¼ 1; 2; uðtÞ 0 2 3 Qk ðrt Þ Lk1 ðrt Þ Lk2 ðrt Þ 6 7 0 5; B ¼ ðB1 ; B2 Þ; D ¼ ðD1 ; D2 Þ; Mk ðrt Þ ¼ 4 L0k1 ðrt Þ Rk1 ðrt Þ L0k2 ðrt 0 Rk2 ðrt Þ ð3:2Þ where ðx0 ; iÞ 2 Rn  N is the initial state, xðtÞ and uðtÞ ¼ ðu1 ðtÞ; u2 ðtÞÞ0 have similar meanings described in Sect. 2. Referring to Li et al. [18], for each initial value xð0Þ ¼ x0 , the value function Vk ðx0 ; iÞ is defined as Vk ðx0 ; iÞ ¼ inf Jk ðuk ; us ; x0 ; iÞ; ð3:3Þ uk 2Uk

where us is the optimal control strategy of player Ps ; s ¼ 1; 2. We emphasize again that we are dealing with an indefinite stochastic Nash game, namely, the symmetric matrix 2 3 Qk ðiÞ Lk1 ðiÞ Lk2 ðiÞ 6 7 0 5; k ¼ 1; 2; i 2 N Mk ðiÞ ¼ 4 L0k1 ðiÞ Rk1 ðiÞ L0k2 ðiÞ 0 Rk2 ðiÞ is possibly indefinite. Definition 3.1 The stochastic Nash equilibrium strategy pair ðu1 ; u2 Þ 2 U½0; 1Þ is defined as satisfying the following conditions. J1 ðu1 ; u2 ; x0 ; iÞ 6 J1 ðu1 ; u2 ; x0 ; iÞ; J2 ðu1 ; u2 ; x0 ; iÞ 6 J2 ðu1 ; u2 ; x0 ; iÞ;

8u1 2 U1 ;

ð3:4aÞ

8u2 2 U2 ; i 2 N;

ð3:4bÞ

where U½0; 1Þ ¼ U1 ½0; 1Þ  U2 ½0; 1Þ; U1 ½0; 1Þ and U2 ½0; 1Þ denote the space of all admissible strategies for player Pk ; k ¼ 1; 2 (see reference [2]).

123

Stochastic Nash Games

Definition 3.2 posed if

491

The generalized stochastic Nash games (3.1)–(3.4a,b) are well 1\Vk ðx0 ; iÞ\ þ 1;

8x0 2 Rn ; i 2 N; k ¼ 1; 2:

A well-posed problem is attainable (w.r.t. ðx0 ; iÞ) if there is a control uk ðÞ achieves Vk ðx0 ; iÞ. In this case the control uk ðÞ is optimal (w.r.t. ðx0 ; iÞ). 3.2 Main Results The definition of stochastic stabilizability, which was an essential assumption in the section introduced by Li et al. [18], Dragan and Morozan [11], Dragan et al. [12]. Definition 3.3 Consider the following linear stochastically controlled system with Markovian jumps dxðtÞ ¼ ½Aðrt Þ þ Bðrt ÞKðrt ÞxðtÞdt þ ½Cðrt Þ þ Dðrt ÞKðrt ÞxðtÞdWðtÞ;

ð3:5Þ

which is asymptotically mean-square stable, i.e., lim EfkxðtÞk2 jr0 ¼ ig ¼ 0: t!1

Similar to the finite-time horizon stochastic Nash games discussed in Sect. 2, we can get the corresponding results of the infinite-time horizon stochastic Nash games stated as Theorem 3.4, which can be verified by following the line of Theorem 2.6. Theorem 3.4 Assume there exist uk ðtÞ; k ¼ 1; 2, the closed-loop system is asymptotically mean square stable. Suppose there exists a stabilizing solution P ¼ ðP1 ; P2 Þ :! Sln  Sln ; P1 ¼ ðP1 ð1Þ;    ; P1 ðlÞÞ; P2 ¼ ðP2 ð1Þ;    ; P2 ðlÞÞ of the following CSRAEs ði; j 2 NÞ. 8 l P >  þ A0 ðiÞP1 ðiÞ þ C0 ðiÞP1 ðiÞCðiÞ  þQ  1 ðiÞ þ pij P1 ðjÞ > P1 ðiÞAðiÞ > > > j¼1 > <    0  ðiÞP1 ðiÞD1 ðiÞ þ L11 ðiÞ R11 ðiÞ þ D0 ðiÞP1 ðiÞD1 ðiÞ 1 ð3:6Þ  P ðiÞB ðiÞ þ C 1 1 1 >  0  > > 0  þ L0 ðiÞ ¼ 0; > >  B1 ðiÞP1 ðiÞ þ D1 ðiÞP1 ðiÞCðiÞ 11 > : R11 ðiÞ þ D01 ðiÞP1 ðiÞD1 ðiÞ [ 0; i 2 N; 8 l P > ~ þ A~0 ðjÞP2 ðjÞ þ C~0 ðjÞP2 ðjÞCðjÞ ~ þQ ~ 2 ðjÞ þ > pjk P2 ðkÞ > P2 ðjÞAðjÞ > > k¼1 > <   1  P2 ðjÞB2 ðjÞ þ C~0 ðjÞP2 ðjÞD2 ðjÞ þ L22 ðjÞ R22 ðjÞ þ D02 ðjÞP2 ðjÞD2 ðjÞ >  0  > > 0 ~ þ L0 ðjÞ ¼ 0; >  B ðjÞP ðjÞ þ D ðjÞP ðjÞ CðjÞ > 2 2 2 2 22 > : R22 ðjÞ þ D02 ðjÞP2 ðjÞD2 ðjÞ [ 0; j 2 N; ð3:7Þ where

123

492

H.-N. Zhu et al.

 1  0   þ L0 ðiÞ ; K1 ¼  R11 ðiÞ þ D01 ðiÞP1 ðiÞD1 ðiÞ B1 ðiÞP1 ðiÞ þ D01 ðiÞP1 ðiÞCðiÞ 11  1  0  ~ þ L0 ðjÞ ; K2 ¼  R22 ðjÞ þ D02 ðjÞP2 ðjÞD2 ðjÞ B2 ðjÞP2 ðjÞ þ D02 ðjÞP2 ðjÞCðjÞ 22  1 ¼ Q1 þ L12 K2 þ K0 2 L0 þ K0 2 R12 K2 ; A ¼ A þ B2 K2 ; C ¼ C þ D2 K2 ; Q 12 ~ 2 ¼ Q2 þ L21 K1 þ K0 1 L0 þ K0 1 R21 K1 : A~ ¼ A þ B1 K1 ; C~ ¼ C þ D1 K1 ; Q 21 Recall that ðP1 ; P2 Þ is a stabilizing solution of CSRAEs (3.6)–(3.7) if the following closed-loop system dxðtÞ ¼ ½Aðrt Þ þ B1 ðrt ÞK1 ðrt Þ þ B2 ðrt ÞK2 ðrt ÞxðtÞdt þ ½Cðrt Þ þ D1 ðrt ÞK1 ðrt Þ þ D2 ðrt ÞK2 ðrt ÞxðtÞdWðtÞ is exponentially stable in mean square, where K1 ðiÞ, K2 ðiÞ are defined after (3.6)–(3.7). Denote F1 ðiÞ ¼ K1 ðiÞ; F2 ðiÞ ¼ K2 ðiÞ, then the stochastic Nash equilibrium strategy ðu1 ; u2 Þcan be represented by 8 l P >  > > F1 ðiÞxðtÞvrt ¼i ; < u1 ðtÞ ¼ i¼1 ð3:8Þ l > P >   > F2 ðiÞxðtÞvrt ¼i : : u2 ðtÞ ¼ i¼1

Furthermore, the generalized stochastic Nash games (3.1)–(3.4a,b) are well posed (w.r.t. ðx0 ; iÞ), and the optimal value is determined by Vk ðx0 ; iÞ ¼ inf Jk ðuk ; us ; s; y; iÞ ¼ y0 Pk ðiÞy; uk 2Uk

k; s ¼ 1; 2; k 6¼ s; i 2 N:

Remark 3.5 It is worth mentioning that CSRAEs as (3.6)–(3.7) may have more solutions but not all are stabilizing solutions. It remains as a challenge for future research to find conditions which guarantee the existence of a stabilizing solution of CSRAEs like (3.6)–(3.7).

4 Application to Stochastic H 2 =H 1 Control Now, we apply the above developed theory to solve some problems related to stochastic H2/H? control. First, we state the stochastic H2/H? control problem for Markov jump linear systems; then, we demonstrate the usefulness of the above developed theory in the study of stochastic H2/H? control. For notational simplification, we only consider the case of infinite-time horizon, which is similar for finite-time horizon. Let us now give the detailed formulation of the problem. Consider the following stochastic controlled system with state- and controldependent noise:

123

Stochastic Nash Games

493

8 < dxðtÞ ¼ ½Aðrt ÞxðtÞþ Bðrt ÞvðtÞ þ Cðrt ÞuðtÞdt þ ½Dðrt ÞxðtÞ þ Fðrt ÞuðtÞdWðtÞ; Lðrt ÞxðtÞ ; xð0Þ ¼ x0 2 Rn ; : zðtÞ ¼ uðtÞ ð4:1Þ where uðtÞ; vðtÞ; zðtÞ are the control input, external disturbance, and controlled output, respectively. Define two associated performances as follows: Z 1 h  i J1 ðu; v; x0 ; iÞ ¼ E kzðtÞk2 c2 kvðtÞk2 dtjr0 ¼ i 0

and

Z J2 ðu; v; x0 ; iÞ ¼ E

1

2



kzðtÞk dtjr0 ¼ i ;

i 2 N:

0

The infinite-time horizon stochastic H2/H? control problem of system (4.1) is described as follows (Huang et al. [15], Zhu et al. [40]). Definition 4.1 For given disturbance attenuation level c [ 0, if we can find u ðtÞ  v ðtÞ 2 U½0; 1Þ, such that (1)

u ðtÞ stabilizes system (4.1) internally, i.e., when vðtÞ ¼ 0; u ¼ u , the state trajectory of (4.1) with any initial value ðx0 ; iÞ 2 Rn  N that satisfies lim EfkxðtÞk2 jr0 ¼ ig ¼ 0:

t!1

(2)

jLu j1 \c with  jLu j1 ¼

sup v2U2 ½0;1Þ; v6¼0;u¼u;x0 ¼0



hR i1=2 l P 1 2 E 0 kzðtÞk dtjr0 ¼ i

i¼1

: hR i1=2 l P 1 2 E 0 kvðtÞk dtjr0 ¼ i

i¼1

(3)

When the worst case disturbance v ðtÞ 2 U2 ½0; 1Þ, if existing, is applied to (4.1), u ðtÞ minimizes the output energy Z 1  2  J2 ðu; v ; x0 ; iÞ ¼ E kzðtÞk dtjr0 ¼ i ; i 2 N: 0

Then we say that the infinite-time horizon stochastic H2/H? control problem has a pair of solutions. Obviously, ðu ; v Þ is the Nash equilibrium strategies [7], such that J1 ðu ; v ; x0 ; iÞ 6 J1 ðu ; v; x0 ; iÞ; J2 ðu ; v ; x0 ; iÞ 6 J2 ðu; v ; x0 ; iÞ; i 2 N: According to Theorem 3.4 discussed in Sect. 3, the following results can be obtained straightly.

123

494

H.-N. Zhu et al.

Theorem 4.2 For system (4.1), suppose the following CSRAEs (i; j 2 N). 8 l P > ~ þ A~0 ðiÞP1 ðiÞ þ D ~ þ pij P1 ðjÞ ~ 0 ðiÞP1 ðiÞDðiÞ ~ þ QðiÞ > > P1 ðiÞAðiÞ < j¼1

> > > :

þ c2 P1 ðiÞB1 ðiÞB01 ðiÞP1 ðiÞ ¼ 0;

ð4:2Þ

K1 ðiÞ ¼ c2 B01 ðiÞP1 ðiÞ ; i 2 N;

8 l P >  þ A0 ðjÞP2 ðjÞ þ D0 ðjÞP2 ðjÞDðjÞ þ L0 ðjÞLðjÞ þ > P ðjÞ AðjÞ pjk P2 ðkÞ 2 > > > k¼1 < þðP2 ðjÞCðjÞ þ D0 ðjÞP2 ðjÞFðjÞÞK2 ðjÞ ¼ 0; > > > > I þ F 0 ðjÞP2 ðjÞFðjÞ [ 0; > : K2 ðjÞ ¼ ðI þ F 0 ðjÞP2 ðjÞFðjÞÞ1 ðC0 ðjÞP2 ðjÞ þ F 0 ðjÞP2 ðjÞDðjÞÞ ; j 2 N;

ð4:3Þ

where ~ ¼ L0 L þ K0 2 K2 ~ ¼ D þ FK2 ; Q A~ ¼ A þ CK2 ; A ¼ A þ BK1 ; D have stabilizing solutions P ¼ ðP1 ; P2 Þ :! Sln  Sln ; P1 ¼ ðP1 ð1Þ;    ; P1 ðlÞÞ; P2 ¼ ðP2 ð1Þ;    ; P2 ðlÞÞ, and ðP1 ; P2 Þ is a stabilizing solution of CSRAEs (4.2)–(4.3) if the following closed-loop system dxðtÞ ¼ ½Aðrt Þ þ Bðrt ÞK1 ðrt Þ þ Cðrt ÞK2 ðrt ÞxðtÞdt þ ½Dðrt Þ þ Fðrt ÞK2 ðrt ÞxðtÞdWðtÞ is exponentially stable in mean square, where K1 ðiÞ, K2 ðiÞ are defined in (4.2)–(4.3). Then the stochastic H2/H? control has a pair of solutions ðu ðtÞ; v ðtÞÞ with the feedback form u ðtÞ ¼ K2 ðrt ÞxðtÞ; v ðtÞ ¼ K1 ðrt ÞxðtÞ: In this case, u ðtÞ is a solution to the stochastic H2/H? control of system (4.1), and v ðtÞ is the corresponding worst case disturbance. Remark 4.3 Similar to remark 1, the CSRAEs as (4.2)–(4.3) may have more solutions, but not all are stabilizing solutions; so how to find conditions which guarantee the existence of a stabilizing solution of CSRAEs like (4.2)–(4.3) deserves future study. Illustrative example—consider the following numerical example, assign the coefficients of system (4.1) as follows:       0 1 0 1 0:2 0:2 N ¼ f1; 2g; P ¼ ; Að1Þ ¼ ; Að2Þ ¼ ; 0:8 0:8 2 3 1 0           1 0 1 3 0:1 0 Bð1Þ ¼ ; Bð2Þ ¼ ; Cð1Þ ¼ ; Cð2Þ ¼ ; Dð1Þ ¼ ; 1 1 0 1 0 0:3       0:5 0 0 0 Dð2Þ ¼ ; Fð1Þ ¼ ; Fð2Þ ¼ : 0 0:2 0:1 0

123

Stochastic Nash Games

495

Set c ¼ 0:7; and solving (4.2)–(4.3) by using the algorithm presented in Li et al. [18], we have     0:034 8 0:024 6 0:042 7 0:068 2 Pð1Þ ¼ ; Pð2Þ ¼ : 0:024 6 0:051 2 0:068 2 0:311 2 Therefore, the stochastic H2/H? controller is given by uðtÞ ¼ 0:035 0x1 ðtÞ 0:026 1x2 ðtÞ, while rt ¼ 1; and uðtÞ ¼ 0:128 1x1 ðtÞ  0:204 6x2 ðtÞ, while rt ¼ 2. Given initial values r0 ¼ 1; x1 ð0Þ ¼ 2 and x2 ð0Þ ¼ 1, using the Euler-Maruyama method with step size M ¼ 0:00 1, computer simulation of the paths of rt ; uðtÞ; x1 ðtÞ and x2 ðtÞ are shown in Fig. 1, 2 and 3. 3

2.5

r

t

2

1.5

1

0.5

0 0

1

2

3

4

5

6

7

8

9

10

8

9

10

t Fig. 1 Curve of rt

0.02

0

u(t)

−0.02

−0.04

−0.06

−0.08

−0.1 0

1

2

3

4

5

6

7

t Fig. 2 Curve of uðtÞ

123

496

H.-N. Zhu et al. 2.5 2

x (t) 1

1.5 1 0.5 0

0

1

2

3

4

5

6

7

8

9

10

6

7

8

9

10

t 2

2

x (t)

1 0 −1 −2

0

1

2

3

4

5

t Fig. 3 Curve of x1 ðtÞ and x2 ðtÞ

5 Conclusion In the present paper, stochastic Nash games of Markov jump linear systems governed by Itoˆ’s differential equation with state- and control-dependent noises both in finite-time horizon and infinite-time horizon have been considered. The defined Nash equilibrium strategies can be calculated by solving CSRDEs (CSRAEs). Moreover, the obtained results have been applied to stochastic H2/H? control for Markov jump linear systems with state- and control-dependent noises. Finally, the numerical example has shown the validity of the proposed method. These results are only theoretical analysis; how to extend them into practical applications, such as in the engineering/economics or anything in the social sciences, needs future investigations. Acknowledgments The authors wish to thank anonymous reviewers for their suggestions and penetrating comments which led to a substantial improvement of the final draft.

References [1] Athans, M.: Command and control (C2) theory: a challenge to control science. IEEE Trans. Autom. Control 32(4), 286–293 (1987) [2] Basar, T., Olsder, G.J.: Dynamic Non-cooperative Game Theory. SIAM, Philadelphia (1999) [3] Bjo¨rk, T.: Finite dimensional optimal filters for a class of Itoˆ-processes with jumping parameters. Stochastics 4(2), 167–183 (1980) [4] Boukas, E.K., Zhang, Q., Yin, G.: Robust production and maintenance planning in stochastic manufacturing systems. IEEE Trans. Autom. Control 40(6), 1098–1102 (1995)

123

Stochastic Nash Games

497

[5] Chen, S., Li, X., Zhou, X.Y.: Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim. 36(5), 1685–1702 (1998) [6] Chen, S., Zhou, X.Y.: Stochastic linear quadratic regulators with indefinite control weight costs. II. SIAM J. Control Optim. 39(4), 1065–1081 (2000) [7] Chen, B.S., Zhang, W.: Stochastic H2/H? control with state-dependent noise. IEEE Trans. Autom. Control 49(1), 45–57 (2004) [8] Costa, O.L.V., Oliveira, A.: Optimal mean-variance control for discrete-time linear systems with Markovian jumps and multiplicative noises. Automatica 48(2), 304–315 (2012) [9] Dockner, E., Jørgensen, S., Van Long, N., Sorger, G.: Differential Games in Economics and Management Science. Cambridge University Press, Cambridge (2000) [10] Dragan, V., Morozan, T.: Game-theoretic coupled Riccati equations associated to controlled linear differential systems with jump Markov perturbations. Stoch. Anal. Appl. 19(5), 715–751 (2001) [11] Dragan, V., Morozan, T.: The linear quadratic optimization problems for a class of linear stochastic systems with multiplicative white noise and Markovian jumping. IEEE Trans. Autom. Control 49(5), 665–675 (2004) [12] Dragan, V., Morozan, T., Stoica, A.M.: Mathematical Methods in Robust Control of Linear Stochastic Systems. Springer, New York (2006) [13] Elliott, R.J., Siu, T.K.: A stochastic differential game for optimal investment of an insurer with regime switching. Quant. Financ. 11(3), 365–380 (2011) [14] Ghosh, M.K., Arapostathis, A., Marcus, S.I.: Optimal control of switching diffusions with application to flexible manufacturing systems. SIAM J. Control Optim. 31(5), 1183–1204 (1993) [15] Huang, Y., Zhang, W., Feng, G.: Infinite horizon H2/H? control for stochastic systems with Markovian jumps. Automatica 44(3), 857–863 (2008) [16] Hui, E.C.M., Xiao, H.: Maximum principle for differential games of forwardbackward stochastic systems with applications. J. Math. Anal. Appl. 386(1), 412–427 (2012) [17] Li, X., Zhou, X.Y.: Indefinite stochastic LQ controls with Markovian jumps in a finite time horizon. Commun. Inf. Syst. 2(3), 265–282 (2002) [18] Li, X., Zhou, X.Y., Rami, M.A.: Indefinite stochastic LQ control with Markovian jumps in infinite time horizon. J. Global Optim. 27(2–3), 149–175 (2003) [19] Limebeer, D.J.N., Anderson, B.D.O., Hendel, B.: A Nash game approach to mixed H2/H? control. IEEE Trans. Autom. Control 39(1), 69–82 (1994) [20] Lin, X., Zhang, C., Siu, T.K.: Stochastic differential portfolio games for an insurer in a jumpdiffusion risk process. Math. Methods Oper. Res. 75(1), 83–100 (2012) [21] Loparo, K.A., Blankenship, G.L.: A probabilistic mechanism for small disturbance instabilities in electric power systems. IEEE Trans. Autom. Control 32(2), 177–184 (1985) [22] Mariton, M.: Jump Linear Systems in Automatic Control. Springer, New York (1990) [23] McAsey, M., Mou, L.: Generalized Riccati equations arising in stochastic games. Linear Algebra Appl. 416(2–3), 710–723 (2006) [24] Mou, L., Yong, J.: Two-person zero-sum linear quadratic stochastic differential games by a Hibert space method. J. Indus. Manag. Optim. 2(1), 93–115 (2006) [25] Øksendal, B.: Stochastic Differential Equations: An Introduction with Application. Springer, New York (1998) [26] Pan, Z., Basar, T.: H1 Control of Markovian jump systems and solutions to associated piecewisedeterministic differential games. In: Olsder, G.J. (ed.) Annals of the International Society of Dynamic Games, Birkhauser, Boston (1995) [27] Petkovski, D.: Multivariable control system design: a case study of robust control of nuclear power plants. Fault Detect. Reliab. 9(2), 239–246 (1987) [28] Qian, L., Gajic, Z.: Variance minimization stochastic power control in CDMA systems. IEEE Trans. Wirel. Commun. 5(1), 193–202 (2006) [29] Song, Q.S., Yin, G., Zhang, Z.M.: Numerical solutions for stochastic differential games with regime switching. IEEE Trans. Autom. Control 53(2), 509–520 (2008) [30] Starr, A.W., Ho, Y.C.: Nonzero-sum differential games. J. Optim. Theory Appl. 3(3), 184–206 (1969) [31] Wang, G., Yu, Z.: A Pontryagin’s maximum principle for non-zero sum differential games of BSDEs with applications. IEEE Trans. Autom. Control 55(7), 1742–1747 (2010) [32] Wang, G., Yu, Z.: A partial information non-zero sum differential game of backward stochastic differential equations with applications. Automatica 48(2), 342–352 (2012)

123

498

H.-N. Zhu et al.

[33] Xu, R., Zhang, L.: Stochastic Maximum Principle for Mean-field Controls and Non-Zero Sum Mean-field Game Problems for Forward–Backward Systems, arXiv:1207.4326v1 [math.OC] (2012) [34] Yeung, D.W.K., Petrosyan, L.A.: Cooperative Stochastic Differential Games. Springer, New York (2006) [35] Yong, J.: A leader-follower stochastic linear quadratic differential game. SIAM J. Control Optim. 41(4), 1015–1041 (2002) [36] Yu, Z.: Linear-quadratic optimal control and nonzero-sum differential game of forward-backward stochastic system. Asian J. Control 14(1), 173–185 (2012) [37] Zhang, Q.: Stock trading: an optimal selling rule. SIAM J. Control Optim. 40(1), 64–87 (2001) [38] Zhou, X.Y., Yin, G.: Markowitz mean-variance portfolio selection with regime switching: a continuous-time model. SIAM J. Control Optim. 42(4), 1466–1482 (2003) [39] Zhu, H., Zhang, C.: Infinite time horizon nonzero-sum linear quadratic stochastic differential games with state and control-dependent noise. J. Control Theory Appl. 11(4), 629–633 (2013) [40] Zhu, H., Zhang, C., Bin, N.: Infinite horizon linear quadratic stochastic Nash differential games of Markov jump linear systems with its application. Int. J. Syst. Sci. 45(5), 1196–1201 (2014)

123