Genshiro Kitagawa. Introduction to Time Series Modeling (Введение в моделирование временных рядов)

Подождите немного. Документ загружается.

NON-GAUSSIAN TREND MODEL 209

= ∆t

∑

j=1

i−j

. (14.10)

[ Filtering ]

For i = 1,... ,d, compute

f (t

) =

r(y

−t

) ˜p(t

)

r(y

−t

, (14.11)

where the normalizing constant C is obtained by

C =

r(y

−t) ˜p(t)dt =

∑

j=1

j−1

r(y

−t) ˜p(t)dt

= ∆t

∑

j=1

r(y

−t

. (14.12)

[ Smoothing ]

For i = 1,... ,d, compute

= ˜s(t

) =

f (t

)

˜q(t

−u)˜s(u)

˜p(u)

f (t

)

∑

j=1

j−1

˜q(t

−u)˜s(u)

˜p(u)

= ∆t · f (t

)

∑

j=1

i−j

. (14.13)

It should be noted that, in practical computation, even after the pre-

diction step (14.10) and the smoothing (14.13), the density functions

should be normalized so that the value of the integral over the whole

interval becomes 1. This can be ach ieved, for example, by modifying f

to f

/I( f ), where I( f ) is deﬁned by

I( f ) =

f (t)dt = ∆t ( f

+ ···+ f

). (14.14)

14.4 Non-Gaussian Trend Mo del

In the trend model considered in Chapter 11, in cases where the distribu-

tion of the system noise v

or the observation noise w

is non-Gaussian,

210 NON-GAUSSIAN STATE-SPACE MODEL

Figure 14.1: Data generated by the model of (14.15).

we obtain a non-Gaussian trend model. In this section, numerical ex-

amples are given in order to explain the features of the non-Ga ussian

model.

Example (Non-Gaussian trend estimation) Figure 14.1 shows the

data generate d by the following models:

∼ N(

,1),











0, 1 ≤ n ≤ 100

1, 1 01 ≤ n ≤ 200

−1, 201 ≤ n ≤300

0, 3 01 ≤ n ≤ 400

. ( 14.15)

In the plot, it can be seen that the mean value function,

, abruptly

changes three times. For estimation of a changing mean value function

with the structural changes shown in Figure 14.1, we consider the fol-

lowing ﬁrst order tr end model:

= t

n−1

+ v

= t

+ w

, (14.16)

where the observation noise w

is assumed to be Gaussian and the system

noise v

follows the ty pe IV Pearson family of distributions:

q(v

) =

(

+ v

)

< b < ∞. (14.17)

Here, b and

denote the shape parameter and the dispersion parameter,

NON-GAUSSIAN TREND MODEL 211

Table 14.2 Non-Gaussian model with Pearson family of distributions with vari-

ous values of shape parameter b.

log-likelihood AIC

0.60 0.211 ×10

−9

1.042 −597.1 9 1198.38

0.75 0.299 ×10

−7

1.043 −597.3 9 1198.78

1.00 0.353 ×10

−4

1.045 −597.9 9 1198.98

1.50 0.303 ×10

−2

1.045 −599.1 3 1202.26

3.00 0.406 ×10

−1

1.046 −600.4 0 1204.80

∞ 0.140 ×1 0

−1

1.048 −600.6 9 1205.38

respectively, and c is a no rmalizing co nstant, which makes the value of

the integral of q(v) over the whole interval equal to 1 and is given by

c =

2b−1

Γ(b)/Γ(

)Γ(b −

) (Johnson and Kotz (1970)). The Pearson

family of distributions can express various symmetric probability density

functions with heavier-tailed distributions than the normal distributions.

Here, the Pearson family of distributions yields th e Cauchy distribution

for b = 1, the t distribution with k degrees of freedom for b = (k + 1)/2

and the normal distribution as b → ∞.

Table 14.2 summarizes the values of the ma ximum likelihoods and

the AI Cs of the Pearson family of distributions with b = 3/k, (k =

0,.. .,5). Here, it is shown that the AIC is minimized at b = 0.60, a nd

the AIC of the normal distribution model (b = ∞) is the maximum.

The plot on the left-hand side of Figure 14.2 depicts th e change over

time of the smoothed distribution of the trend p(t

), obtained from

the Gaussian model (for b = ∞), and the corresponding plot obtain e d

from the non-Gaussian model with b = 1.0 is sh own on the right-hand

side. The left plot (a) shows that the distribution of the trend estimated

by the Gaussian mode l g radually shifts left or rig ht with the progress of

time n. On the other hand, for the case of the non-Ga ussian m odel shown

in the r ight plot (b), the estimated density is seen to be very stable with

abrupt changes at only three time points.

In Figure 14.3, the plot on the left-hand side shows the mean and the

±1,2 , 3 standard deviation intervals of the estimated distribution at e ach

time point for the Gaussian model. On the other hand, the plot on the

right-ha nd side shows the 0.13, 2.27, 15.87,50.0, 84.13, 97.73 and 99.8 7

percentile points of the estimated trend distribution that correspond to

the mean and ±1, 2,3 intervals of the Gaussian distribution. Comparing

212 NON-GAUSSIAN STATE-SPACE MODEL

plot (a) with (b) of Figure 14.3, it can be seen that the non-Ga ussian

model yields a smoother estimate than the Gaussian model a nd clearly

detects jumps in the trend.

Figure 14.2 Changes over time of the smoothed distribution of the trend. Left:

Gaussian model, right: non-Gaussian model.

Figure 14.3 Esti mation of trend by Gaussian (left plot) and non-Gaussian (right

plot) models.

A TIME-VARYING VARIANCE MODEL 213

14.5 Non-Symmetric Distribution–A Time-Va rying Variance

Model

The distribution of data observed from the real world sometimes reveals

non-symmetry. This might be due to certain characteristics or nonlin-

earity of the d ata generating mechanism. If we apply the least squares

method to such a data set, we may obtain a biased estimate of the trend.

Thus, if the distribution of the observations is known, we ma y o btain a

better e stima te s by using a non-Gaussian distribution r(w) for ﬁltering

and smoothing. In this section, for the estimation of the tim e-varying

variance, we rec onsider the model treated in Section 1 3.1;

= t

m−1

+ v

= t

+ w

. (14.18)

Example In Chapter 13, the double exponential distribution for

the observation noise w

is approximated by a Gaussian distribution

N(−

/6). Here, we shall compare the distribution obtained by the

Gaussian approximation with the exact distribution obtained by nu-

merical computa tion. Figure 14.4 shows the transformed seismic data

that are derived from the original data of Figure 1.1(f ). The p lotted

data set was obtained by sampling every two points after ﬁlterin g by

˜y

= y

−0.5y

n−1

and transformed by the method discussed in Sectio n

13.1. In this chapter, sin ce it is assumed that the time series is a white

noise, this transformation was applied in order to weaken the stron g au-

tocorrelation that can be ob served in the ﬁrst part of the data.

Fitting the Gaussian model with

/6 to the data, we obtain the

maximum likeliho od estimate

= 0.04909 and AIC = 2165.10. Figur e

14.5 shows the wigg ly tr end estimated by th is model. This is due to the

outliers in Figure 14.4, creating a large deviation that strongly a ffects the

estimated trend.

On the other hand, the maximum likelihood estimates o f the non-

Gaussian m odel with a Cauchy distribution fo r the system noise and a

double exponential distribution for the obser vation noise

q(x) =

+ x

(14.19)

r(x) = exp{x −e

}

are

= 0.000260 and AIC = 2056.97, respectively.

From the big differences between the AIC values of the two mod-

els, it is evident that the non-Gaussian model is much be tter than the

214 NON-GAUSSIAN STATE-SPACE MODEL

Figure 14.4: Transformed seismic data.

Figure 14.5: Estimated trend (log-variance) by Gaussian model.

Gaussian m odel. Actually, as shown in Figure 14. 6, the trend obtained

with the non-Gaussian mode l is smoother by far than the one obtained

by the Gaussian model and clearly detected sud den increases in the vari-

ance due to the arrivals of the P-wave and the S- wave. Moreover, the

plot of this m odel depicts that the observations with a large deviation

downward inﬂuenc e only the curve of 0.13 percent points but not that

of the 50 percent p oints. In sum mary, by non-Gaussian smoothing with

an appropriate model, we can obtain an estimate of the trend which is

A TIME-VARYING VARIANCE MODEL 215

Figure 14.6: Estimated t rend (log-variance) by a non-Gaussian model.

able to automatically detect a jump of the p arameter but is not unduly

inﬂuenced by a non-symmetric distribution. This estimation me thod of

a time-varying variance can be used for the estimation of th e stochastic

volatility of a ﬁnancial time series.

Example ( Stock price index data) Figure 14.7 (a) shows the series

obtained by applying the method shown in Section 13.1 to the difference

of the logarithm of Nikkei 225 stock price index data. On the other hand,

Figure 14. 7(b) shows the posterio r distribution of the estimate of log

obtained as the trend of this series estimated by the non-Gaussian model

of (14.18). In addition, an estimate of the volatility is obtained from the

exponent of 1/2 of the central curve (the 50 percent curve) in plot (b),

i.e., by

exp



log



This method of estimating time-varying variance can be imm ediately

applied to the smoothing of the periodogram. Since the periodogram fol-

lows the

distribution with 2 degre e s of f reedom, th e series obtained

by taking the logarithm of the periodogram thus follows distributions,

which are approximately equivalent to (14.19). Therefore, the method

treated in this chapter can be ap plied to smoothing of the logarithm of

the periodogram as well. Moreover, using a non-Gaussian model, it is

216 NON-GAUSSIAN STATE-SPACE MODEL

Figure 14.7 (a) Transformed Nikkei 225 stock price index data, (b) posterior

distribution of log

, (c) estimates of volatility.

possible to estimate the time-varying variance from the transformed data

= log y

. (14.20)

In this case, for the density function of the observation noise, we use

the density fu nction obtained as the logarithm of the

distribution w ith

1 degree of freedom, i.e.,

r(w) =

√

exp



w −e



. (14.21)

Here the time-varying variance can be estimated w ithout making the as-

sumption that log

2m−1

= log

. Figure 14.8 shows the estimate of

log

obtained by applying this meth od to the logarithm of the differ-

ences of the Nikkei 225 stock price index data.

APPLICATIONS OF NON-GAUSSIAN STATE-SPACE MODEL 217

Figure 14.8 The smoothed distribution obtained by the model of (14.21) for the

Nikkei 225 stock price index data.

14.6 Applications of the Non-Gaussian State-Space Model

In this section, several applications of non-Gaussian state- space mod el

are presented. Certain typical cases for th e non-Gaussian state-sp a ce

model are dealt with in d etail in this section.

14.6.1 Processing of the outliers by a mixture of G aussian

distributions

Here, we consider outliers to follow a distribution different from the

usual noise distribution of o bservations. To process the outliers using

a time series, assuming th at the outliers appea r with probability

≪ 1,

we may use a mixture of Gaussian distributions for the observation noise

r(w) = (1 −

)

(w;

) +

αϕ

(w;

). (14.22)

The Gaussian-sum ﬁlter and smoother for this Gaussian-mixture noise

model is given in Alspach and Sorenson (1 972) and Kitag awa (1989,

1994). Here

(x;

) denotes the density function of the no rmal dis-

tribution with mean

and variance

. The ﬁrst and second terms on

the right- hand side express the distribution of the normal observations,

and the distribution of the outliers, respectively. In particu la r, by putting

= 0 and

≫

, the observation noise of the outliers is assumed

to have zero mean but possess a large variance. The Cauch y distribution

may b e considered as an ap proximation of the above-mentioned mixture

distribution. Therefore, the outliers can be treated for example using the

Cauchy distribution or the Pearson family of distributions, which can

be used to generate a density function of the observation noise. On the

218 NON-GAUSSIAN STATE-SPACE MODEL

other hand, if outliers always appear on the positive side of the time se-

ries, as shown in Figure 1.1(h) of the groundwater data, we can estimate

the trend ignoring the o utliers by setting

≫

and

14.6.2 A nonstationary discrete process

Non-Gaussian models can also be applied to the smoothing of discrete

processes, for instance, nonstationary binomial distributions and Poisson

distributions. Consider the time series m

,.. ., m

that takes integer val-

ues. Assume that the probability of taking value m

at time n is given by

the Poisson distribution

P(m

) =

−p

. (14.23)

Here, p

denotes the expected f requency of the occurrences of a certain

event at a unit time. The mean o f the Poisson distribution is not a constant

and gradually changes with time. Such a model is called a nonstationary

Po isson process model. As a model for the cha nge over time of the Pois-

son mean, p

, we may use the tre nd component model p

= p

n−1

+ v

Here, since p

is required to be positive, 0 < p

< ∞, we deﬁne a n ew

variable

= log p

and assume a tr end component model. Therefore,

the model for smoothing the nonstation ary Poisson process becomes

n−1

+ v

, (14.24)

P(m

) =

−p

, (14.25)

where p

is deﬁned by p

= e

. The ﬁrst and second models correspond

to the system model and the observation model and yield conditional

distributions of

and m

, respectively. There fore, the non-Gaussian ﬁl-

tering method pr esented in this chapter can be applied to the discrete

process, and we can estimate the time-varying Poisson mean value from

the data.

Similarly, the non-Gaussian state-space model can be ap plied to the

smoothing of an inhomogeneous binary proce ss. Assume that the proba-

bility of a certain event occurring m

times in th e ℓ

observations at time

n is given by the binomial distribution

P(m

, ℓ

) =



ℓ



(1 − p

)

ℓ

−m

. (14.26)

Here, p

is called the binomial probability a nd expresses the expe c te d

number of occurrences of a certain event. In this case, if p

varies with