Vo1.12 No.1
J O U R N A L O F ELECTRON]~CS
TIME
SERIES
NEURAL
FORECASTING
Wen X i n h u i ( ~ : ~ )
Jo.n. 1995
NETWORK
METHODS
Chen K a i z h o u ( ~ j ~ : ~ )
(The CentraJ of Neural Network, Xi'dian University, XJ'an 710071) Abstract
This paper has discussed the possibility and key problem to construct the neural
network time series model, and three time series neural network forecasting methods has been proposed, i. e. a neural network nonlinear time series model, a neural network multi-dimension time series model and a neural network combining predictive model. These three m e t h o d s are applied to real problems. The results show t h a t these m e t h o d s are better t h a n the traditional one. Furthermore, the neural network methods axe compared with the traditional method, and the constructed model of intellectual information forecasting system is given. Key words
Information theory; Information processing; Neural network forecasting m e t h o d
I. I n t r o d u c t i o n Along with the development of science and technology many things should be recognized and studied.
From the point of philosophy, there are two methods for constructing the
forecasting models, one is the structural m e t h o d and the other is the processing method. The structural method will be used if we know the change rule about the problem. But if we only know input-output d a t a for the predictive thing we will analyze the relation between input and output. In fact, time series analysis m e t h o d is a kind of signal processing method [1}. We can summarize the traditional time series model by Principle 1. Principle 1
Suppose we have a time series {x~) = xt E
R'
(t = 1 , 2 , - - . , T ) , then
the traditional time series model can be written with nonlinear mapping: Xt+k--1 = ~O(Xt,~t--l,''',Xt--l)
(1)
Note 1
In Eq.(1), if we replace xt by a vector variable, ~ is a vector-valued function.
Note 2
If ~ is a linear function, Eq.(1) is a linear time series model, such as AR or
ARIMA; if 9 is a nonlinear function, Eq.(1) is a nonlinear time series model, such as TAR or SETAR. As we have mentioned above, if we want to get more fitting precision of the time series model, the key problem is to increase the measure of approximation between mapping and actual values. So we perhaps propose a problem, t h a t is, would we have used other technique to construct the mapping ~?
2
JOURNAL OF ELECTRONICS
Vol.12
Artificial neural network is the physical model to simulate the brain neural network [2]. In this paper we utilize neural network information processing skill to realize the nonlinear mapping ~. We have set up three new neural network forecasting models, t h a t is, a neural network nonlinear time series model, a neural network multi-dimension time series model and a neural network combining predictive model. These three m e t h o d s are applied to real problems. The results show t h a t these methods are better t h a n the traditional one. II. A Neural
Network
Nonlinear
Time Series Model
Nowadays nonlinear time series analysis is a new direction which has been developed rapidly because many problems can not be solved by linear model. In this section we utilize neural network technique to make nonlinear signal processing and we have formed a new neural network nonlinear time series model. 1. T h e s t r u c t u r e o f m o d e l We use {x~} indicate a real r a n d o m variable when t E Z, where Z express a set of integers.
Dec
Xt -! X( r
X( t
§
k
1)
L
-121
Fig.1 A neural network nonlinear time series model We construct a topologic structure a b o u t the neural network as Fig.1 shows. It is a backpropagation neural network whose unit function is a sigmoid excitation function. (1)
Input units
At arbitrary time t, L is the time delay, t h a t is, {x(~_~), 1 < i < L}
give L real input units. The network's k steps predictor x(i+k-a) can be defined by X(*t.i_k_l) =
Fw(x(t-1), x(t-2),
" " ", x(t+k-1)-12,""", X(,-{-k--1)--12p)
(2)
where F~ is the transformation function of the neural network. If the time series were a strong seasonal sequence we would have given more information. We use a group of month code to indicate the present month, such as U1 = 1, U2 = 0 = Ua . . . . etc.
U12 indicate January, U1 = 0, U2 = 1, Ua = 0 = U4 . . . . .
U12 indicate F e b r u a r y
No.1
T I M E SERIES NEURAL N E T W O R K F O R E C A S T I N G M E T H O D S
(2) Output units
3
Because we only study one dimensional nonlinear time series, k-th
steps predictive value has been represented by one o u t p u t unit z(t+k-D. (3) Hidden units
At the neural network we set up one or more hidden layers but the
number of hidden units at each layer depends on the real problem's complexity. The execution of neural network has two stages: learning and forecasting. T h e network's learning use past d a t a and the results will be stored in the network. The stage of forecasting is to input the predicted data to the network then the neural network will give the forecasting values. Learning process has used backpropagation algorithm 131. 2. A p p H e a t l o n We have applied neural network nonlinear time series model to predicate the Wolf's sunspot number.
X1-311~ Xl2
Xt44
X t --
Fig.2 Wolf's sunspot number predictive model
When L = 5 and k = 5 we get a forecasting neural network for predicting the Wolf's sunspot number as Fig.2 shows. The fitting data is between 1968 and 1978. We have 100 data and get 100 samples. Suppose that {tk} is primary record, we convert {t~) to Vk = log(tk + 1) ] xk
Vk/log(200) J~
(3)
The sample defines as (xt-5, zt-a, xt-3, xt-2, x t - , , xt+4),
(t = 6 , . . . , 106)
The neural network has 1 hidden layer, whose units are 28. The total learning error of neural network is 0.00174 which has been iterated by 18,000 times. We use the network to forecast the Wolf's sunspot number from 1980 to 1987, the results are shown in Tab.1.
4
JOURNAL OF ELECTRONICS
Vol.12
From Tab.l we can see that the neural network is better than the traditionalforecasting methods, because the neural network method has more nonlinear mapping feature. Tab.1 Wolf's sunspot number predictive results
1980--1987)
Error
SETAR predictor
Error
Neural network predictor
Error
162.8
8.1
i74.7
20.0
143,0
-11,7
133.5
-7.0
147.5
7.0
128,1
-12,5
115.9
91.5
-24.4
92.4
-23.5
104.8
-11A
1983
66.6
53.3
-13.3
62.9
-3.7
60.1
-6.6
1984
45.9
23.1
-22.8
24.1
-21.8
43.3
-2.6
1985
17.9
4.6
-13.3
12.6
-5.3
20.5
2.6
1986
13.4
2.0
-11.4
0.2
-13.2
13.3
~).1
1987
29.2
21.1
8.1
10.8
18.4
18.6
-10.6
Observe value
Linear" predictor
1980
154.7
1981
140.5
1982
Year
222.4
MES
III. A Neural Network
253.9
Multi-Dimension
73.23
Time Series Model
The multi-dimension time series problem is very difScult in time series analysis because if the dimension is more than I the computation is much complex. If we use a neural network multi-dimension time series model we not only can avoid the computing complexity of model but also can get good numerical results. 1. T h e s t r u c t u r e o f m o d e l Suppose x ( t ) = ( x l ( t ) , - . . , Xr(t)) T is r dimension time series, then the predictive model can be show
xCt)
=
-
1),..., x(t p)) -
(4)
where 9 is a linear or nonlinear function. We constructed a p order r dimension time series network as Fig.3 shows. nonlinear mapping form
x(t
-
1) to x(t).
We have given Theorem 1 which ensure the
precision of neural network mapping.
~t(I-- 1) x~(t- p) xl(t-t- 1) 9 xt ( t + I1 x , { ~ - 11 9 , ( t - p)
It is a
.
Fig.3 A neural network multi-dimension time series model
No.1
T I M E SERIES N E U R A L
Theorem
Set ~(t) : ( ~ l ( t ) , ' ' ' ,
1
FORECASTING METHODS
NETWORK
9 (t) =
:~r(t))T.
5
If we have relation
- 1))
(5)
where x(t), x(t - 1) E E, then the mapping ~' of three neural network can approximate at arbitrary precision. Proof
This conclusion can get directly from Kolmogorov theorem [4].
2. Application W e have applied the neural network multi-dimension time seriesmodel to predict insects of Manasi cotton field in Xinjiang. The structure of network is shown as Fig.4. It has learned 10,000 times by backpropagation algorithm and the total error E = 0.020088. Then we use this model to forecast the density of insects (Yanjima) in 1982. The result has been given by Fig.5. IV. A Neural
Network
Combining
Predictive
Model
1. T h e s t r u c t u r e o f m o d e l Bates and Granger {5] put forward the idea of combining forecasting in 1969 and the combining method has being developed rapidly in recently years. But the traditional combining m e t h o d has many limitations, so we have extend the traditional combining idea to the generalized combining principle.
9
Fig.4 Forecasting insects model Principle 2 x E X C Rnl
Suppose we have m predictive methods for event F , t h e n mapping y E Y C R indicates the i-th predictive method.
combining function of different forecasting methods
~Oi(i :
We have nonlinear
1 , ' " , Wt).
~-~ W(~Ol, ~o2, 99 9 ~om) where W is a nonlinear mapping. Set z E X , f ( z ) is an actual value function,
(6)
I I ][ is a
6
J O U R N A L OF E L E C T R O N I C S
Vol.12
800.0 600.0 9~
400.
o
200.0
0.0 0.0
5.0
10.0
15.0
20.0
25.0
Data (every 5 days to samp]iag)
Fig.5 Numerical results
measure, then the generalized combining Principle 2 says t h a t IIf(~) - ~11 ~
~-
IIf(m) - ~,(m)ll
(U
In Principle 2 it is difficult to determine the nonlinear mapping W. We have adopted neural network techniques to realize a mapping W as Fig.6 shows. In fact, every forecasting m e t h o d has corresponded to one information channel, so the action of hidden layer is to synthesize the opinions of channels and it will get b e t t e r combining predictive values. 2. A p p l i c a t i o n
Fig.6 A neural network combining predictive model
We have used the neural network combining model to predict Chinese village savings deposit balance from 1983 to 1986. The combining m e t h o d compares with the least square
No.1
TIME SERIES N E U R A L N E T W O R K F O R E C A S T I N G M E T H O D S
7
method and cubic exponent smooth method, the numerical results have been given by Tab.2. Tab.2 Methods
compared
fitting
(Unit: Year
Savings
1971
11.49
1972
10 s y u a n )
Least square
Cubic exponent
Neural network
method
Smooth method
18.47
10.03
18.41
13.06
14.54
11.23
16.52
1973
15.34
12.84
15.24
17.22
1974
20.58
13.88
18.67
18.26
1975
23.28
16.15
27.78
21.72
1976
26.46
21.16
26.36
23.11
1977
27.33
28.40
29.67
27.08
1978
34.22
37.87
27.40
30.37
1979
40.19
49.58
42.73
42.94 53.54
combining m e t h o d
1980
53.37
63.53
47.36
1981
77.79
79.71
69.84
76.79
1982
118.63
98.12
109.32
111.89
553.97
188.87
107.49
MES
Name
Tab.3 Results compared r Traditional method
Neural network
Characteristic Numerical computation
9
o
Processing operation
9
o
Samples learning
o
9
Pattern recognition
o
9
F u z z y data
o
9
Maintain
9
9 o
Flexibility
9
Adaptation
o
9
Parallel process
o
9
L a r g e scale
o
9
Maximal fitting
o
9
High precision
9
o
Tolerance
o
9
Explained results
o
Note: o - - T o h a v e p r o p e r t y ; o - - T o h a v e no p r o p e r t y
V. N e u r a l N e t w o r k F o r e c a s t i n g M e t h o d s Intellectual Information
and
Processing System
At present, neural network techniques can solve many problems which traditional method can not do in economics and business.
But still we have other problem to be
solved urgently, such as the research of fundamental theory about neural network forecast-
8
JOURNAL OF ELECTRONICS
Vol.12
ing method, the d a t a structure and how to combine traditional m e t h o d , expert s y s t e m and neural network. We have c o m p a r e d traditional m e t h o d s with network m e t h o d s , Tab.3 gives the results. We can see t h a t the traditional m e t h o d has suited a b s t r a c t thought and the neural network has suited thinking in t e r m s of images. A new constructed model of intellectual information forecasting s y s t e m is given by Fig.7. We will s t u d y more new intellectual forecasting m e t h o d s and this will be bring benefit to develop social economy. Neural network Data processing [ ~ral
network
Fig.7 The constructed model of intellectual information forecasting system
References [1] Wong Wenbo, The Base of Forecasting Theory, Petroleum Industry Publishing Co., Beijing, 1984. [2] T. Poggio, F. Giros, Networks for Approximation and Learning, Proc. IEEE, 78(1990), 1481-1497. [3] R. Hicht-Nielsen, Theory of the back propagation neural network, Proceedings of the International Conference on Neural Networks, San Diego: SOS Printing, 1(1987), 593--608. [4] A. K. Kolmogorov, On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition, Doklady Akademii Nauk SSR, 114(1957), 953-956. [5] J. M. Bates, C. W. J. Graner, The combination of forecasts, Operations Research Quarterly, 20(1969)2, 319-325.
