Genshiro Kitagawa. Introduction to Time Series Modeling (Введение в моделирование временных рядов)
Подождите немного. Документ загружается.
List of Figures
1.1 Examples of various tim e series. 2
1.1 Examples of various tim e series (co ntinued). 3
1.1 Examples of various tim e series (co ntinued). 4
1.1 Examples of various tim e series (co ntinued). 5
1.2 Difference o f the logarithm of the Nikkei 225 data. 10
1.3 Change from previous month and year-over-year change
of WHARD data. 11
1.4 Maximum tem perature data and its m oving average. Top
left: orig inal data, to p right: moving average with k = 5,
bottom left: k = 17, bottom right: k = 29. 13
2.1 Histograms and scatterplots of the yaw rate and rolling
of a ship. 18
2.2 Sample autocorrelation functions. 23
2.3 Histograms and scatterplots of ship ’s data. 24
2.4 Histograms and scatterplot of the gro undwater level and
barometric pressure data. 25
2.5 Autocorrelation fun ctions and cross-correlation func-
tions of ship’s data. 28
2.6 Autocorrelation fun ctions and cross-correlation func-
tions of the groundwater level an d barometric pressure
data. 29
3.1 Autocorrelation function, power spectrum, and realiza-
tion of a white noise with variance
σ
2
= 1. 32
3.2 Autocorrelation function, power spectrum, and realiza-
tion of a first-order AR model with a = 0.9. 33
3.3 Autocorrelation function, power spectrum, and realiza-
tion of a first-order AR model with a = −0.9. 34
xi
xii LIST OF FIGURES
3.4 Autocorrelation function, power spectrum, and realiza-
tion of a second-order AR model with a
1
= 0.9
√
3 and
a
2
= −0.81. 35
3.5 Periodograms of the data of Figure 1.1 (on a logarithmic
scale). 37
3.6 Sample autocorrelation functions and periodograms in a
logarithm ic scale: sample sizes n = 200, 800, 3200. 39
3.7 Raw spectra of a white noise. 40
3.8 Smoothed periodograms of the data shown in Figu re 1.1.
Horizontal axis: frequency f , vertical axis: periodogram
on a logarithmic scale, log p( f ). 43
3.9 Data with line spectr um and its periodo grams obtained
by (3.8) and by FFT after adding 11 2 zeros. 46
3.10 Yaw rate data, periodogram and FFT spec trum. 47
4.1 Density functio ns of various probability distributions. 52
4.2 Realizations of various probability distributions. 53
4.3 Difference between th e expected log-likelihood and the
log-likeli-hood. 63
4.4 Log-transformation of the sunspot number data and the
WHARD data. 67
5.1 Original data and the minimum AIC regression curve
for the maximum temperature data. 77
6.1 Impulse response fu nctions of four models. 86
6.2 Autocovariance functions of the fo ur models. 87
6.3 PARCORs of the four models. 89
6.4 Logarithm of the power spectra of the four models. 90
6.5 Line-like spectrum of ARMA(2.2). 91
6.6 Characteristic roots. (a) AR model of order 1, (b) AR
model of order 2, (c) MA mo del with order 2 and (d)
ARMA mode l with order (2,2). 93
6.7 Spectra (diagonal), amplitude spectra (above diagonal)
and phase spectra (below diagona l) for the ship data. 97
6.8 Power spectra (3 in diagonal) and cohere ncies. 98
6.9 Power co ntributions. 99
7.1 Changes of PARCOR and AIC and estimated spectrum
by the AIC best AR model for the sunspot number data. 113
7.2 Change in AIC values, as the order varies. 114
LIST OF FIGURES xiii
7.3 Estimated spectra by AR models with minimum AIC
orders. 115
8.1 The east-west com ponent record of seismic wave and
estimated spectra obtained by a locally stationary AR
model. 129
8.2 Estimation of the arrival times of the P-wave and the
S-wave. 133
9.1 Recursive computation by the Kalman filter and smooth-
ing algorithm. ⇒: prediction, ⇓: filter, ←: smoo thing,
→: increasing horizon prediction. 139
9.2 Increasing horizon predictive distributions (bold line:
mean, thin line: ± (standard deviation) and ◦: observed
value). Ord e rs o f the AR models are 1, 5, 10 and 15,
respectively. 143
9.3 Interpolation of missing values (b old line: mean, thin
line: ± (standard deviation) and ◦: observed value).
Orders of the AR m odels are 0, 1, 5, 10 and 15 . 150
11.1 Trends of maximum temperature da ta and WHARD
data estimated by the polynomial regression model. 162
11.2 Trend of the temperature da ta obtained by the first order
trend models. 167
11.3 Trend of temperature data obtained by the second o rder
trend models. 168
11.4 Temperature data and the tren d and residuals obta ined
by the trend model with k = 2 and
τ
2
= 0.321 ×10
−3
. 170
12.1 Seasonal adjustment of WHARD data by a standard
seasonal adjustment mode l. 177
12.2 Increasing horizo n prediction using a seasonal adjust-
ment mode l. 178
12.3 Seasonal adjustment of BLSALLFOOD data by the
standard seasonal adjustment model. 180
12.4 Increasing horizo n predic tion of BLSALLFOOD data
(prediction starting point: n = 132). 181
12.5 Increasing horizo n predic tion of BLSALLFOOD data
with floating starting points (prediction starting point:
n =126, . .. , 138). 181
xiv LIST OF FIGURES
12.6 Seasonal adjustment of BLSALLFOOD data by the
model, including stationary AR components. 183
12.7 Increasing horizon pre diction of the BLSALLFOOD
data by the model, including stationary AR components. 184
12.8 Trading-day adjustment for WHARD data. 187
13.1 Estimation of time-varying variance and standardized
time serie s. From top to bottom: transformed data,
estimated time- varying log-variance and the normalized
time series. 191
13.2 Estimated time-varying PARCOR for the normalized
seismic data. Only the first four PARCOR’s are shown.
Left plots: k = 1, right plots: k = 2. Horizontal axis:
time point, vertical axis: value of PA RCOR. 196
13.3 Time-varying spectr a of a seismic data. 198
13.4 Time-varying spectrum estimated by assuming d isconti-
nuities in the parameters. 200
14.1 Data gene rated by the model of (14.15). 210
14.2 Changes over time of the smoothed distribution of the
trend. Left: Gaussian m odel, right: non-Gaussian model. 212
14.3 Estimation of trend by Gaussian (left plot) and non-
Gaussian (right plot) mo dels. 212
14.4 Transformed seismic data. 214
14.5 Estimated trend (log-variance) by Ga ussian model. 214
14.6 Estimated trend (log-variance) by a non-Gaussian
model. 215
14.7 (a) Transformed Nikkei 225 stock price index data,
(b) posterior distribution of log
σ
2
n
, (c) estimates of
volatility. 216
14.8 The smoothed distribution obtained by the model of
(14.21) for the Nikkei 225 stock price index data. 217
15.1 Various approximation methods for a non-Gaussian
distribution: (a) assumed true distribution, (b) normal
approximation, (c) piecewise linear function approxi-
mation, (d) step function approximation, (e) Gaussian
mixture approximation and (f) particle realizations. 222
LIST OF FIGURES xv
15.2 Comparison between the empirical distribution func-
tions and the true cumulative distribution functions for
various numbers of particles: (a) m = 10, (b) m = 100
and (c) m = 1000. 224
15.3 One step of the Monte Carlo filter. The figures in the left-
hand column illustrate the probability density functions,
100 realizations and the histogram, respectively, and the
figures in the right-hand colu mn depict the distribution
functions and the empirical distribution functions
obtained from the realizations, respectively. (a) and (b):
the initial state distributions. (c) and (d): the system noise
distribution. (e) and (f): the one-step-ahead pr edictive
distributions. (g) and (h): the filter distributions. (i) and
(j): the filter distributions after re-sampling. 230
15.4 The results o f the Monte Carlo filter: (a) the exact
filter distribution using a Kalman filter. (b) – (d) the
fixed-lag (L = 20) smoothed densities ((b) m = 100, (c)
m = 1000, (d) m = 10,000) with a Monte Carlo filter. 232
15.5 Smoothing with Cauchy distribution model: (a) The
exact distribution obtained from the non-Gaussian
smoothing algorithm. (b) The results of Monte Carlo
smoothing (m = 10,00 0). 233
15.6 Nonlinear smoothing: (a) Data y
n
. (b) Unknown State
x
n
. (c) Smoothed distribution of x
n
obtained by the
Monte Carlo smoo ther. 234
15.7 Fixed-point smoothing for t = 30 (left) and t = 48
(right). From top to bottom, pre dictive distributions,
filter distributions, 1-lag smoothers and 2-lag smoothers. 235
16.1 Realizations, histograms and autocorrelation functions
of uniform rand om numbers and w hite noise. 240
16.2 Simulation of a random walk model. 242
16.3 Simulation of an AR model. 242
16.4 Simulation of a seasonal adjustment model. 243
16.5 Density functions of system noise and the results of the
simulation. 245
16.6 Simulation of an AR model with different noise distri-
butions: (a) normal distribution, (b) Cauchy distribution. 246
List of Tables
3.1 Variances of th e periodogram and the logarithm of the
periodogram obtained from (3.8) a nd (3.14) . 41
3.2 Hanning and Hamming windows. 42
4.1 K-L information for various values of x
n
and k. (g:
normal distribution and f : normal distribution) 56
4.2 Numerical integration for K-L information with various
values of k when g(y) is the standard normal distribution
and f (y) is a Cauchy distribution. 57
4.3 Estimation of the pa rameters of the Cauchy distribution
by the quasi-Newton method. 61
4.4 Estimation of the parameters of the Cauchy distribu-
tion by a quasi-Newton m ethod th a t uses numerical
differentiation. 62
4.5 Various Box-Cox transform s and their AICs. 69
5.1 Residual variances and AICs of regression models of
various orders. p: number of regression coefficients of
the model,
ˆ
σ
2
p
: residual variance, D-AIC: d ifference of
AIC 77
7.1 Innovation variances a nd AIC values of the AR mo dels
of various o rders fitted to the sunspot n umber data. 1 12
7.2 AICs of multivariate AR models fitted to ship’s data. 117
10.1 AIC values for ARMA models with various orders. 155
10.2 Log-likelihood values for ARMA models with various
order. 155
10.3 Estimation of ARMA models with modified initial
values. 156
xvii
xviii LIST OF TABLES
11.1 Maximum tem perature data: The resid ual variances and
AIC values of the polynomial regression models. 161
11.2 (Log) WHARD data: The residual variances and AIC
values of the polynomial regression models. 161
11.3 AIC values for trend models. 169
13.1 AIC’s of time-varying c oefficients AR models fitted to
normalized seismic data. 196
14.1 Approximation of density functions. 208
14.2 Non-Gaussian model with Pearson family of distribu-
tions with various values of shape parameter b. 211
15.1 Approximations of distributions used in the sequential
Monte Carlo filter and the smoother. 224
Contents
1 Introduction and Preparatory Analysis 1
1.1 Time Series Data 1
1.2 Classification of Time Series 6
1.3 Objectives of Time Series Analy sis 8
1.4 Pre-proc essing of Time Series 8
1.4.1 Transformation of variables 9
1.4.2 Differencing 9
1.4.3 Change from the previous month (quarter) and
annual change 10
1.4.4 Moving average 11
1.5 Organization of This Book 13
2 The Covariance Function 17
2.1 The Distribution of Time Series and Stationarity 17
2.2 The Autocovariance Function of Station a ry Time Series 20
2.3 Estimation of the Autocovariance Functio n 21
2.4 Multivariate Time Ser ie s and Scatterplots 24
2.5 Cross-Covariance Function and Cross-Correlation
Function 26
3 The Power Spectrum and the Periodogram 31
3.1 The Power Spectrum 31
3.2 The Periodogram 36
3.3 Averaging a nd Smoothing of the Periodogram 40
3.4 Computational Method of Periodogram 44
3.5 Computation of the Periodo gram by Fast Four ie r Trans-
form 44
4 Statistical Modeling 49
4.1 Probability Distributions and Statistical Models 49
xix
xx CONTENTS
4.2 K-L Infor mation and the Entro py Maximization Princi-
ple 54
4.3 Estimation of the K-L I nformation and Log-Likelihood 56
4.4 Estimation of Parameters by the Maximum Likelihood
Method 58
4.5 AIC (Akaike Information Criterion ) 62
4.5.1 Evaluation of C
1
64
4.5.2 Evaluation of C
3
65
4.5.3 Evaluation of C
2
66
4.5.4 Evaluation of C and AIC 66
4.6 Transformation of Data 66
5 The Least Squares Method 71
5.1 Regression Models and the Least Squares Me thod 71
5.2 Householder Transformation 73
5.3 Selection of Orde r by AIC 75
5.4 Addition of Data and Suc cessive Househ older Reduction 78
5.5 Variable Selection by AIC 79
6 Analysis of Time Series Using ARMA Models 83
6.1 ARMA Model 83
6.2 The Impulse Response Function 84
6.3 The Autocovariance Function 85
6.4 The Relation Between AR Coefficients and th e PAR-
COR 88
6.5 The Power Spectrum of the ARMA Process 88
6.6 The Characteristic Equation 92
6.7 The Multivariate AR Model 93
7 Estimation of a n AR Model 103
7.1 Fitting an AR Model 103
7.2 Yule-Walker Method and Levinson’s Algo rithm 105
7.3 Estimation of an A R Model by the Least Squares
Method 106
7.4 Estimation of an AR Model by the PARCOR Meth od 108
7.5 Large Sample Distribution of the Estimates 111
7.6 Estimation of a Multivariate AR Model by the Yule-
Walker Method 113
7.7 Estimation of a Multivariate AR Model by the Least
Squares Method 117
CONTENTS xxi
8 The Locally Stationary AR Model 123
8.1 Locally Stationary AR Model 123
8.2 Automatic Partitioning of the Time Interval into an
Arbitrary Numb er of Subintervals 125
8.3 Precise Estimation of a Change Point 130
9 Analysis of Time Series with a State-Space Model 135
9.1 The State-Space Model 135
9.2 State Estimation via the Kalman Filter 138
9.3 Smoothing Algorithms 140
9.4 Increasing Horizon Prediction of the State 140
9.5 Prediction of Time Series 141
9.6 Likelihood Computation and Parameter Estimation for a
Time Series Model 144
9.7 Interpolation of Missing Observations 147
10 Estimation of the ARMA Model 151
10.1 State-Space Representatio n of the ARMA Model 151
10.2 Initial State of an ARMA Model 152
10.3 Maximum Likelihood Estimate of an ARMA Model 1 53
10.4 Initial Estimates of Parameters 154
11 Estimation of Trends 159
11.1 The Polynomial Trend Model 159
11.2 Trend Comp onent Model–Model for Probabilistic
Structural Changes 162
11.3 Trend Model 165
12 The Seasonal Adjustment Model 173
12.1 Seasonal Component Model 173
12.2 Standard Seasonal Adjustme nt Model 176
12.3 Decomposition Including an AR Compon e nt 179
12.4 Decomposition Including a Trading-Day Effect 184
13 Time-Varying Coefficient AR Model 189
13.1 Time-Varying Variance Model 189
13.2 Time-Varying Coefficient AR Model 192
13.3 Estimation of the Time-Varying Spectrum 197
13.4 The Assumption on System Noise for the Time-Var ying
Coefficient AR M odel 198
13.5 Abrupt Changes of Coefficients 199