Journal of Marine Science and Application, Vol. 4, No. 2, June 2005

Ship motion extreme short time prediction of ship pitch based on diagonal recurrent neural network SHEN Yan I and XIE Mei-ping 2

1. School of Science, Harbin Engineering University, Harbin 150001, China 2. School of Information Management and Engineering, Shanghai University of Finance and Economics, Shanghai 200433, China

Abstract: A DRNN (diagonal recurrent neural network) and its RPE (recurrent prediction error) learning algorithm are proposed in

this paper . Using of the simple structure of DRNN can reduce the capacity of calculation. The principle of RPE learning algorithm is to adjust weights along the direction of Gauss-Newton. Meanwhile, it is unnecessary to calculate the second local derivative and the inverse matrixes, whose unbiasedness is proved. With application to the extremely short time prediction of large ship pitch, satisfactory results are obtained. Prediction effect of this algorithm is compared with that of auto-regression and periodical diagram method, and comparison results show that the proposed algorithm is feasible. Keywords: extreme short time prediction ; diagonal recursive neural network; recurrent prediction error learning algorithm;unbiasedness

CLC number: 034

1

Document code: A

Article ID:1671 -9433(2005)02 -0056 -05

predict it [14] But all these traditional methods are on-

INTRODUCTION

ly suitable for linear systems, not the nonlinear systems A lot of research results about predictors of linear system have been obtained, but there are a large number of nonlinear systems in practice.

that should be considered in prediction of large-scale ships.

It is inevitable

Neural network method has been used successfully

that the research on predictors of linear systems should

in complex process recently. Theorems and practices

be transferred to that of nonlinear systems. Nonlinear

both indicate that the neural network method fits many

system modeling is the base of research on predictors of

complex-mapping problems. Recurrent neural network

nonlinear problem. It is very important for nonlinear

is a typical neural network, which is based on BP net-

system models to increase the precision of prediction

work and has dynamic character mapping function by

and lengthen the prediction time.

saving inner conditions. Then the system can fit real

Due to wave force and other disturbing forces, the dynamics of ships is of strong nonlinear.

time character well and can be widely applied to some

In these

complex process. Its foundational principle is to add

years, researchers have used the AR, ARMA and MA

some accessional interior feedback into the former feed-

models for extremely short time prediction of ship atti-

back net in order to enhance the n e t ' s capability in

tude. The methods are based on the data obtained from

processing dynamic information E5~61 Based on the

last period observation on ship motion. Using time se-

time sequence prediction method, this paper builds a

ries analysis method to build the motion model, then

well-known

diagonal

recun'ent

neural

network

(DRNN) for extreme short time prediction model and Received date :2004 - 11 - 17.

applies it to predict the motion of large-scale ships

SHEN Yan,et al :The ship motionextreme short time prediction of ship pitch based on diagonal recurrent neural network

• 57 •

based on the instantaneous values. The simulation re-

ordering weights, theirs dimensions are all weights val-

suits have shown that the prediction results and predic-

ue of the neural network , and the definition perform-

tion precision are improved.

ance index is A

2

THE STRUCTURE OF DRNN AND ITS RPE LEARNING A L G O R I T H M

2.1

1 J( O) = 2N -

-

.

y . eT(k,O) k=i

"

e(k,O),

where e(k,O) is the prediction error vector,and N the length of data.

The structure of DRNN DRNN (diagonal recurrent neural network ) is

The principle of RPE learning algorithm is adjus-

composed of input layer, hidden layer and output layer

ting weights along the direction of Gauss-Newton, such

(Fig 1 ). The input layer and output layer are both

that J ( O )

static networks without any regressive cell.

unnecessary to calculate second-order partial differenti-

reaches minimum value, Meanwhile, it is

al and the inverse matrixes, The adjustment of parame-

cl w

x~(k)

'

ter vector mathematical formula is

w

O(k) = O ( k - 1) + r / ( k ) . o ' [ O ( k - 1 ) 1 ,

. o(k)

x,(k)

in which r/(k) is the study rate, o ' ( 0 ) is the Gaussx,,(k)

Fig. 1

Newton direction, i.e. , tr(O) = - [t~(O) ]-' V J ( O ) .

Structure of the diagonal recurrent neural

The V J ( O ) is the gradient of J ( O ) respect to 0 , ~ r

network Suppose that at every discrete time k,x~(k) is the network input, i = 1 , 2 , -.., n ; O ( k ) the network output; S j ( k ) , Z i ( k ) , ~ ( k ) ( j = l , 2 , . . . , m )

( 0 ) is the Hessian matrix,. They are the first-order and second-order differential of 0 respectively

the input

VJ(O)

IV

1 = ~-

~tl*(k,O) .e(k,O), k=l

and output in hidden layer respectively; Wy the con1

~,(o) = ~ .

nection weight from hidden layer to output layer" We J the connection weight of recurrent neural network; Wjih the connection weights from the j-th input neural cell to the i-th regressive neural cell ,where ~( • ) is Sigmoid

N

Y, q,(~,0) • e z ( k , o ) . k=l

The qt(O) = [ d O ( k , O ) / d O ] T is the first-order of differential of the prediction. RPE algorithm based on the above principle is

Function. Then, we can get

e(k) = O(k) - O ( k ) ,

(4)

n

s~tk) : g ~ • z, tk - l) + ~ ~

• ~ , ( k ) , (1)

M(k) =

g=I

<(k)

= ~(sj(k)),

(2)

1 a(k) • [ I + M ( k ) q t r ( k ) l

P(k) =--

m

o ( k ) = y~ ~ • z , ( k )

P ( K - 1) • qt(K) A ( k ) + q-}r(k)P(k - 1 ) q t ( k ) '

(3)

.P(k-

RPE learning algorithm of DRNN

1), (6)

j=l

2.2

(5)

O(k) = O ( k - 1 )

+n(k)P(k)tlF(k)e(k).

(7)

RPE (recurrent prediction error) algorithm is one kind of method that obtains the parameter estimation through the minimizing prediction error. Suppose that all the parameter vector 0 are all neural network' s

The A (k) is the forget factor, a constant A is simply. If the dimension of output in DRNN is one, we obtain

• 58 •

Journal of Marine Science and Application, Vol. 4, No. 2, June 2005

Ws -W~j,I <~j<~l;

us(k),

= w~,l < . i < . t ;

w°~q~, ( v~( k ) ) • H;( k - 1 ) , [ "P],

dO

dWs

W°;q~,(V;(k)) • H~+~(k - 1 ) ,

W s --

W~,(

W s ~- W~/i,i_l

V;( k) ) • H,_,( k - 1 ) ,

~,(V,(k)

The s = 1 , 2 , . . - , n

) . X~(k),

(8)

Wtti+l ,1 ~< i ~< 1 - 1; ,2 ~< i ~< 1;

W = W~j,1 < . i < _ l , 1 <~j<.n.

x l + 1 x 3 is the dimension of

0. Therefore, the RPE learning algorithm of DRNN is

The DRNN network input vector x ( k - 1 ) as follows :

x(k - l) = Ex,(k - 1),x2(k - 1),...,Xo,(k - 1)1 T,

as follows : 1 ) Initialize weight value and other parameters: W(0) =random(

• ),P(0)

where :

[y(k-i),(1

and A are suitable val-

~< i ~< n ) ;

xi(k)

tu(k

ues; 2) Calculate the hide layer point H and the output layer point O of DRNN according to the Eqs.

( 1) -

(3);

d +n + l ) , ( n

+ l <.i <_n;).

The output prediction ))(k) could be calculated based on the network parameters obtained by the training algorithm mentioned above :)),~ ( k ) = O ( k ) .

3) Compose the matrix q z ( 0 )

according to the

3, 2

Eq. ( 8 ) ;

The parameter estimation 0 ( k )

4 ) Calculate the e ( k ) , P ( k ) , and 0( k ) according to t h e E q .

Unbiassedness of RPE learning algorithm

(4)

-

can be written as

follows :

(7);

O(k + l) = O(k) - R ( k ) g r a d [ J ( 0 ) ]

I o(k),

0

5) return to 2 ) until the following inequality is satisfied :

where the step l should be selected such that the covar-

Ir O ( k )

3

(lo)

-

O(k

-

1) II < ~.

iance of input vector x ( k )

is a constant matrix and is

APPLICATION OF DRNN TO NONLINEAR

independent of 0 ( k ) , R ( k )

is a N-dimension symmet-

SYSTEM PREDICTION

ric matrix so-called weight matrix; grad [ J ( 0 ) ] o

3.1

sents the graduate of lost function with respect to 0, i.

Prediction process Suppose the model of nonlinear dynamic system as

e°

grad[J(0)

follows :

y(k)

0

] I o(A/ = - [ y ( k )

- x~(k)O(k)]x(k). (11)

= F(y(k-1),...,y(k-n),y(k-d-1), •. . , u ( k - d - m ) ) ,

repre-

(9)

Substituting Eq.

( 11 ) into Eq. ( 1 0 )

yields the ex-

where m and n are the order of output and input re-

pression of 0( k + l) as follows :

spectively, d is the systematic time d e l a y , a n d F ( • )

O(k +;) : O(k) + R ( k ) x ( k ) I y ( k ) +xT(k)0(k). (12)

is an unknown nonlinear function. The structure prediction model of DRNN network is shown as Fig 1. The number of input node is n; = n + m + 1. The output node is L = 1. y,,, ( k ) is the output value as prediction of system output.

Theorem: In the case of output vector without noise, Eq.

(12)

gives the unbiased hess asymptotic

estimation of the parameter, i. e. , limE/0(k)}

= 00.

SHEN Yan,et al:The ship motionextreme short time prediction of ship pitch based on diagonal recurrent neural network Proof: Taking the mathematical expectation of

Step2: : Let

the both sides of Eq. ( 12), we have E{0(k+l)/

x(k - 1)

=

E{0(k) +R(k)x(k)[y(k)

[Xl(k

-xT(k)O(k)]l

El0(k) l +R(k)E{x(k)y(k)}

•

- l),x2(k

-

=

1),,°°,xn(k

- I)]T

=

[ u ( k - 1) , u ( k - 2) , - - . , u ( k - n) ] T as the actual input of network

-

R(k)Etx(k)xT(k)O(k)

• 59 •

]1.

Step 3: Prediction: The network parameters ob-

Because the step l is selected such that the covariance

tained in 2.2 can be used for prediction. Suppose that

of vector x ( k ) is a constant matrix and is independent

the vector parameter 0 is composed of weights of the

of O(k) , we obtain

network sequentially. The dimension of 0 is the num-

E[x(k)O(k)

~ = O.

ber of the network weights. Usually, the output of pre-

the third term of Eq. (12) can be written as

E{x(k)xm(k)O(k)t E{Etx(k)xT(k)

diction value at time k can be represented by the following expression which is a function of model parame-

=

I O ( k ) I I O ( k ) t,

ter O, the input and output data D (~-1) at the time be-

Because of output vector without noise, the above equation is O. Hence

fore k and k-1 ,i. e.

O( k ) = f ( O , k , D (~-') ) = y( k ).

E l 0 ( k + l ) t = E t O ( k ) l,

where ,f( t ) -

Due to E[XIYI = E t E [ X I Y I IYt,

1

l+e

,.

Therefore, the prediction of the system output is

and limE/0(k) t

k~

=

~(k)

00.

: o(k).

Step 4 : Inverse-normalization : ¢)(k + 1) --I ~0..... [" ; ( k ) .

4

PREDICTION

O F S H I P P I T C H IN E X T R E M -

ELY SHORT TIME

the prediction of large ship pitch. The DRNN needs

For the ship pitch prediction, similar to Eq.

two kinds of inputs, which are currently measured value and preceding pitch prediction value of network, the

(9) , we have ~9(k) = f [ 0 ( k -

Now, we apply the above prediction algorithm to

output of network O (k) is the pitch prediction value.

1),.-.,~0(k-n,),

u ( k - 1) , - - . , u ( k - nu) ] ,

(13)

where $ ( k ) is the pitch angle of ship, u ( k ) is random disturbing generally the Gauss white noise sequence. Specially let Eq ( 13 ) as : ¢,(k) = F [ q , ( k - 1 ) , - . . , ~ , ( k - , ~ ) ~ . As the enabled function of nerve network is sigmoid function, the input samples need to be normalized. The process of network prediction is shown be-

The hidden layer chooses 15 nodes. Based on the real data of pitch of a large ship, i. e. , take 100 pitch values as the inputs of the network. After the parameters training, the output prediction could be preceded. The prediction result is shown in Fig 2. The curve of absolute prediction error is shown in Fig 3. In figures, the real line describes the real curve of pitch and the broken line describes the prediction curve of pitch. In Fig 2, the time interval of (420 - 4 7 0 ) r e p r e s e n t s the

lOW :

Step 1: Normalization of input sample, i.e. y ( k )

learning period and the time interval of (470 - 4 8 0 ) r e presents the prediction period.

= o(k)

10o,oXI '

where 0 ..... I =maxl~O(i) l; l~i~k

We have made three algorithm methods of ship

• 60 •

Journal of Marine Science and Application, Vol. 4, No. 2, June 2005 [2] BATES M R, BOOK D H, and Powell F D. Analog com-

1.5 1.0 0.5

puter applications in predictor design [ J ]. IRE Trans On

04}

Elec Com,1957(3) :144 - 148.

-0.5 ~--1.0 -1.5 -2.0 -2.5 -3.0 Fig. 2

[ 3 I ZHAO Xiren, PENG Xiuyan, SHEN Yan, XIE Mei-ping. Study status quo of extremely short time modeling and predicting of ship motion [ J 1. Ships Engineering. 2002,3 : t/s

4-8.

The curve of real and prediction values of ship pitch

measuring systems mouted on ships in motion at sea [ A ].

0.20 [

presented as 4 'h National symp of Marine Science Inst ISA

°011;F\

o.o,f\J \

~,-0"05 0

4

V

-0.15 ~0 20 k Fig. 3

[4 ] KAPLAN P. ROSS D. Comparative performance of wave

[ C ]. Coco Beach, Florida : 1968. [5 ] KU C C, LEE K Y. Diagonal recurrent neural networks for 76

47

\___j ~

~

t/s

The curve of absolute prediction error

pitch predicting, and compared the errors. The mean

dynamic systems control [ J ]. IEEE Trans on Neural Network, 1995,6( 1 ) :144 - 156. [ 6 ] WEI W. A new on-line recursive learning algorithm for recurrent neural network [ J 1- Acta Automatica Sinica, 1998,24 (5) :616 -621.

squar error of each algorithm is: the error of Auto-reSHEN Yan,born in 1965. She received gressive model is 0 . 0 3 4 , the error of Periodic map is 0. bachelor degree of computational mathe029, and the error of DRNN is 0 . 0 2 4 . Therefore, the proposed DRNN prediction algorithm is more feasible than the others which are mentioned above. 5

CONCLUSIONS

matics in 1987,Jilin University, and Master degree of navigation, control and guide in 1996, Harbin Engineering University. Research interests are non-linear system modeling and predicting, neural network and computational mathematics.

Prediction of system output is proposed, and the unbiassedness of the prediction is proved. Then we ap-

XIE Mei-ping, born in 1974, received

ply the algorithm of diagonal recursive neural network

ph. D degree of Engineering in control the-

to extremely short time prediction of ship pitch. The

ory and engineering, 2001, Harbin Engi-

mean square prediction error of proposed algorithm,

neering University, master degree of sci-

auto-regressive model and periodic map algorithms are compared. The results indicate that the proposed method is more feasible to short time prediction of ship pitch

ence in applied

mathematics,

Northwestern Polytechnical

1998,

University,

and bachelor degree of Engineering in applied mathematics, 1994, Northwestern Polytechnical University. Research interests

than others. are supply chain modeling, data mining and data analysis, nonREFERENCES [ 1 ] FLECK J T. Short time prediction of the motion of a ship in waves[A]. Proe Ist Conf on ships and waves[C]. USA: Council on Wave Research and SNAME, 1954.

linear system modeling and predicting.