Skiadas C.H., Dimotikalis I. (editors) Chaotic Systems: Theory and Applications
Подождите немного. Документ загружается.
278
G.
P.
Pavlos et al.
surrogate (thick line), as well as m=6-1O, r=6, w=6 for the ECG time
series and m=lO,
r=6, w=6 for its surrogate (thick line).
The results show in normal conditions, namely absence
of
epileptic seizure, the brain activity corresponds to high dimensional -
stochastic dynamics as indicated by the high values
of
the slopes
of
the
correlation integral, shown in Fig. 4(b).
On the contrary, the cardiac
activity corresponds to low dimensional and periodic dynamics, as
indicated from the low value saturation slopes profile. However, during
the epileptic seizure crucial changes occur both in brain and cardiac
dynamics, as shown in Figure Sea-d).
4. Summary
of
Data Analysis and Results
The results presented previously showed:
a)
Coexistence
of
a SOC (stochastic) process and a hidden low
dimensional chaotic process, revealed after the exclusion
of
high
frequency component, underlying the solar activity.
b) Evidence for low and high dimensional local plasma processes
undelying the magnetosphere, that reveal the turbulent character
of
the
plasma sheet, during Superstorm-substorm processes.
c) Coexistence
of
low dimensional chaotic spatiotemporal processes
and a strong stochastic component corresponding to magnitude,
revealing the nonlinear and complex character
of
the Hellenic
lithospheric dynamics.
d) During healthy states, the brain activity is high dimensional
corresponding to SOC dynamics, while during epileptic episodes
corresponds to local and low dimensional chaotic dynamics. As for the
cardiac dynamics our results indicate that during healthy states low
dimensional and periodic dynamics prevail, while during epilepsy the
dynamics become high dimensional and chaotic.
Thus, our results showed two different and significant chaotic
processes underlying all the distributed systems that were studied. The
first corresponds to the development
of
local low dimensional chaos in
a high dimensional
SOC environment and was observed during super-
substorms in the magnetospheric system or epilepsy events in the
human brain. The second one corresponds to the chaotic spreading
or
wandering
of
local avalanche spatiotemporal events and was observed
in the solar flares and earthquake processes. The development
of
cardio
chaos during epilepsy could be explained also by the spatial spreading
of
desynchronization local events or spreading
of
local defects,
as
we
show in the next section.
Spatiotemporal Chaos
in
Distributed Systems 279
S.
Conclusions and Theoretical Expanations
Although solar flares, magnetospheric superstorms or space plasma
turbulence, earthquakes, epilepsy and cardiac arrhythmias are processes
in different physical systems, they reveal significant common features.
They include local structures in distributed physical systems far from
equilibrium. Moreover, these phenomena are related to far from
equilibrium, first or second order, phase transition processes, with low
chaotic or high dimensional chaotic-SOC spatiotemporal profile.
The common theoretical presupposition underlying the above
spatiotemporal phenomena is one
of
the main concepts supported in
this study. This point
of
view was further supported by previous studies
concerning the character
of
universality in solar flares and substorms or
earthquake occurrence (Watkins,
2001[14], Archangelis et ai., 2006
[1]). This concept
of
universality is based on the experimental
observation
of
power laws which were observed also in the brain
dynamics. Generally, the universality
of
power law processes
historically was used
as
the trace
of
SOC theory which was against
chaos.
In this study
we
presented strong experimental evidence for a
wider theoretical framework which can compromise
SOC and Chaos as
two different, between others, manifestations
of
nonlinear dynamics
of
a distributed system. In the following we introduce and try to correlate
the results
of
data analysis with some significant phenomena included
in the far from equilibrium nonlinear dynamics.
5.1. Critical points
and
spinodal nucleation
Although critical theory and spinodal decomposition are well known
processes near equilibrium, their essential features can be also observed
at far from equilibrium phenomena. In particular, nucleation from
metastables states with deep quenches
of
their free energy reveal
noncompact, dominant fluctuations, while the nucleating droplets are
formed by the coalescence
of
clusters and are ramified with fractal
dimension. Fluctations near a spinodal are generated by random walk
of
the number
of
fundamental clusters while there exist a raming onto
random bond percolation. These concepts can be extended in the far
from equilibrium processes including long-range interactions (mean or
near-mean field). Spinodal decomposition processes and their emerging
spatiotemporally correlated fluctuations can be used for the modeling
of
driven threshold systems where the avalanches can be explained as
nucleation near the spinodal and percolation states. These theoretical
280
G.
P.
Pavlos et al.
concepts concerning the far from equilibrium spinodal nucleation, can
be used to explain the low dimensional chaos or high dimensional
chaos or
SOC states, observed in the systems we studied, as critical
dynamics and changes
of
critical states in the space
of
the relevant
parameters
of
the nonlinear stochastic system (see also Seq.
2).
5.2. Local structures, turbulence, intermittency, anomalous diffusion
and synchronization in extended systems
Solar flares, super-substorms, earthquakes, epileptic episodes and
cardiac arrhythmia are some examples
of
the macroscopic
manifestation
of
local and dynamical structures
of
the underlying
complex dynamical systems.
On the other hand, complex dynamical
systems with many degrees
of
freedom, composed
of
many parts
interrelated in nontrivial way, exhibit a wealth
of
collective phenomena
such as: ordered (coherent) and random (incoherent) behavior,
hierarchy
of
structures over a wide range
of
lengths.
The variety
of
complex phenomena in distributed systems with
many (practically infinite) degrees
of
freedom has been revealed until
now by studying different systems such as: deterministic or
probabilistic cellular automata (CA), coupled map lattices (CML),
PDE
systems (Kuramoto - Shivashinsky (KS) equation), complex
Ginzbourg-Landau equation, reaction-diffusion (RD) equation, Navier-
Stokes equation, stochastic Kardar-Parisi-Zuang equation and other
Langevin type equations).
Synchronization has been one
of
the collective phenomena
as
interacting chaotic systems, in spite
of
their natural instability, can
synchronize their trajectories. Synchronization or de synchronization in
the system is produced by the competition between intrinsic disorder
and the diffusive effect provoked by the coupling. At critical points
toward synchronization we observe intermittent behavior where the
synchronized state is interrupted by chaotic burst.
The chaotic bursting accompanying the synchronization
transition gives the behavior
of
on-off intermittency. This behavior
is
accompanied with dimension variability as it is characterized by the
breakdown
of
the continuous splitting between stable and unstable
manifold in the high-dimensional phase space
of
the coupled system.
The synchronization or loss
of
synchronization process with on-off
intermittency burst can be used as a theoretical explanation
of
the
correlation dimension changes, observed in solar flares,
magnetospheric substorms or space plasma turbulence, earthquakes,
Spatiotemporal Chaos
in
Distributed Systems
281
epilepsy and cardiac arrhythmias, according to our data analysis
presented in
Sec.
3.
In general, our data analysis results reveal bursts
of
low dimensionality in a high dimensional environment with SOC or
chaotic characteristics, in cases
of
solar flares, substorms, earthquakes,
epilepsy, or oppositely high dimensional
(SOC or chaos) bursts in a
low dimensional environment in the case
of
cardiac arrhythmia.
Moreover, chaotic bursts with positive Lyapunov exponents in a high
dimensional environment, as we have observed in our study, indicate
the existence
of
a strongly intermittent process known as sporadic
chaos,
as
well
as
the existence
of
directed percolation process (Wang,
1995[13]).
On the other hand, intermittent phenomena and sporadic chaos
can be the manifestation
of
anomalous diffusion and non-Gaussian
fluctuations with Levy motion profile (Wang, 1992[12]). Anomalous
diffusion and Levy flight distributions indicate the nonextensive
generalization
of
Boltzmann-Gibbs statistical mechanics for the
nonequilibrium systems described here. In particular, the fluctuating
dynamics at the onset (edge)
of
chaos and intermittent behavior is
connected with Tsallis q-statistics and Mori's q-phase thermodynamics
[Tsallis,
2005].
The physical systems that were analyzed in this study are
driven and spatially extended systems. For such systems with many
degrees
of
freedom and with chaotic fluctuations in space and time, as
we have described previously, it is possible
to
observe continual
creation, annihilation and interaction
of
topological defects. This
behavior is known as weak turbulence, defect chaos and defect
turbulence.
Topological defects destroy the coherence
of
spatial structures
as
their appearance is linked to the onset
of
defect, mediated turbulence
and onset
of
spatiotemporal chaos and spatial dislocations. The defects
can move in space with chaotic wandering following diffusion,
according to a stochastic Langevin process. This process
of
the chaotic
defect dynamics can map the far from equilibrium spinodal nucleation
process and the droplet dynamics, as well as the cluster dynamics in the
directed percolation process (Miller
& Huse, 1993[8]). Topological
defects,
as
local structures in distributed systems, can be related also to
anomalous diffusion and Levy flight processes according
to
the
mapping
of
deterministic processes
of
the Kuramoto-Sivashinsky type
to stochastic and diffusive processes
of
Kardar-Parisi-Zhang processes
(Jayaprakash
et at., 1993[6]).
282
G.
P.
Pavlos et at.
5.3. Chaotic itinerancy
Chaotic itinerancy can be equivalent to the defect-turbulence or
spinodal nucleation and directed percolation processes,
as
it constitutes
a universal character
of
itinerant motion among varieties
of
low
dimensional dynamical states interrupted by high dimensional
dynamics (Kaneko
& Tsuda, 2003[7]).
The chaotic spatiotemporal spreading
of
local events as well
as
the development
of
local chaos events in a high dimensional
environment
as
we have observed for the solar flares and earthquakes
(first case) and for the magnetospheric super-substorms, the brain
epilepsy and the heart arrhythmias (second case) can also be explained
as a chaotic itinerancy event in the high dimensional phase space
of
the
distributed dynamics.
5.4. Toward a theoretical unification
The universe is a far from equilibrium process, creating fluctuations
and correlations at any scale. The principle
of
scale invariance,
expressed by the renormalization group theory, indicates the possibility
for unification
of
physical processes at any level, while nonlinearity,
chaos and dissipativity are present at any level (Castro,
2005[2])
creating spatiotemporal coherence and structures at microscopic,
mesoscopic and the macroscopic level. Moreover, these characteristics
indicate the unification
of
the deterministic nonlinear chaotic processes
with stochastic diffusive processes (Jayaprakash
et
al., 1993[6],
G't
Hooft, 1999[5]). In the direction
of
such global unification
of
the far
from equilibrium dynamics, the generalized Liouville equation as a
master equation
d
-Ip
>=-Llp
>
dt
r r
(3)
where L is generally non-Hermitean, can be used
as
a prototype
of
unified non-equilibrium phenomena (Tsallis, 2005[ 11], Hinrichen,
2006[4]), including generalized statistics and far from equilibrium
dissipative structures.
References
[1]
Archangelis et al. Universality in Solar Flare and Earthquake Occurrence.
Phys. Rev. Lett. 96: 051102,2006.
[2]
Castro
C.
On non-extensive statistics, chaos and fractal strings. Physica A
347:184-204,2005.
Spatiotemporai Chaos
in
Distributed Systems 283
[3]
Chang,
T.
Low-Dimensional Behavior and Symmetry Braking
of
Stochastic
Systems near Criticality - Can these Effects be observed in Space and in
the Laboratory. IEEE 20(6): 691-694,1992.
[4]
Hinrichen
H.
Non-equilibrium Phase Transitions. Physica A 369:1-28,
2006.
[5]
Hooft
G.
't. The Holographic Principle. Class. Quant. Grav. 16:3283, 1999.
[6]
Jayaprakash
C.
et al. Universal Properties
of
the Two-Dimensional
Kuramoto-Sivashinsky Equation.
Phys. Rev. Lett. 71(1): 12-15, 1993.
[7]
Kaneko
K.
& Tsuda I. Chaotic Itinerancy. Chaos 13(3):926-936,2003.
[8]
Miller
J.
and Huse
D.
A., Macroscopic equilibrium from microscopic
irreversibility in a chaotic coupled-map lattice. Phys. Rev. E 48(4): 2528-
2535, 1993.
[9]
Pavlos, G.P. et
al.
Evidence for Chaotic Dynamics in the Jovian
Magnetosphere. Planetary and Space Science 52:513-541,
2003.
[10] Pavlos,
G.
P. et al. "Self Organized Criticality or / and Low Dimensional
Chaos in Earthquake Processes. Theory and Practice in Hellenic
Region,"
in Nonlinear Dynamics
in
Geosciences, eds. Tsonis
A.
& Elsner
J.
(Springer) pp. 235-259,2007.
[11] Tsallis
C.
Nonextensive Statistical Mechanics, Anomalous Diffusion and
Central Theorems. Milan}. math. 73:145-176,
2005.
[12] Wang X.J. Dynamical sporadicity and anomalous diffusion in the Levy
motion.
Phys. Rev. A 45(12):8407-8417, 1992.
[13] Wang X.J.Sporadic Chaos in space-time processes.
Phys. Rev. E
55(2):1318-1324,1995.
[14] Watkins
W.
et
al.
Testing the SOC Hypothesis for the magnetosphere.
1.
Atm. Solar-Terrestrial Phys. 63:1435-1445,2001.
[15] Wilson K.G. Problems in physics with many scales
of
length. Scientific
American 241:158-179,1979.
284
An
Enhanced
Tree-Shaped
Adachi-Like
Chaotic
Neural
Network
Requiring
Linear-
Time
Computations
Ke Qinl,*
and
B.
John
Oommen
2
,**
1 School
of
Computer
Science
and
Engineering
University
of
Electronic
Science &
Technology
of
China
610054,
Chengdu,
China
(e-mail:
thinkeLqk@hotmail.com)
2 School
of
Computer
Science
Carleton
University
K1S 5B6,
Ottawa,
ON,
Canada
(e-mail:
oommen@scs.carleton.ca)
Abstract:
The
Adachi
Neural
Network
(AdNN)
[1-5],
is
a
fascinating
Neural
Net-
work
(NN)
which
has
been
shown
to
possess
chaotic
properties,
and
to
also
demon-
strate
Associative
Memory
(AM)
and
Pattern
Recognition
(PR)
characteristics.
Variants
of
the
AdNN
[6,7J
have
also
been
used
to
obtain
other
PR
phenomena,
and
even
blurring.
A significant
problem
associated
with
the
AdNN
and
its
vari-
ants,
is
that
all
of
them
require
a
quadratic
number
of
computations.
This
is
essentially
because
all
their
NNs
are
completely
connected
graphs.
In
this
paper
we
consider
how
the
computations
can
be
significantly
reduced
by
merely
using
a
linear
number
of
computations.
To
do
this,
we
extract
from
the
original
complete
graph,
one
of
its
spanning
trees.
We
then
compute
the
weights
for
this
spanning
tree
in
such
a
manner
that
the
modified
tree-based
NN
has
approximately
the
same
input-output
characteristics,
and
thus
the
new
weights
are
themselves
calculated
using
a
gradient-based
algorithm.
By
a
detailed
experimental
analysis, we
show
that
the
new
linear-time
AdNN-like
network
possesses
chaotic
and
PR
properties
for different
settings.
As far as we know,
such
a
tree-based
AdNN
has
not
been
reported,
and
the
results
given
here
are
novel.
Keywords:
Chaotic
Neural
Networks,
chaotic
pattern
recognition
,
Adachi
Neural
Network.
1
Introduction
Artificial
Neural
Network
(ANN) is one
of
the
four
best
approaches
for PR.
ANN
attempts
to
use some
organizational
principles
such
as learning, gen-
*
The
work
of
this
author
was
done
while
he
was
in
Canada
as a
Visiting
Scholar
at
Carleton
University
in
2008.
The
author
gratefully
acknowledges
the
support
of
the
K.
C.
Wong
Education
Foundation
of
Hong
Kong.
H Chancellor's
Professor;
Fellow:
IEEE
and
Fellow:
IAPR.
This
author
is also
an
Adjunct
Professor
with
the
University
of
Agder
in
Grimstad,
Norway.
The
work
of
this
author
was
partially
supported
by
NSERC,
the
Natural
Sciences
and
Engineering
Research
Council
of
Canada.
Enhanced Tree-Shaped Adachi-Like Chaotic Neural Network 285
eralization,
adaptivity,
fault tolerance,
distributed
representation,
and
com-
putation
in
order
to
achieve
the
recognition.
One
of
the
limitations
of
most
ANN
models
is
the
dependency
on
an
external
stimulation.
Once
an
output
pattern
has
been
identified,
the
ANN
remains
in
that
state
until
the
arrival
of
a new
external
input.
This
is in
contrast
to
real biological NNs which
exhibit
sequential
memory
characteristics.
To
be
more
specific, once a
pattern
is
recalled from a
memory
location,
the
brain
is
not
"stuck"
to
it,
it
is also ca-
pable
of
recalling
other
associated
memory
patterns
without
being
prompted
by
any
additional
external
inputs.
This
ability
to
"jump"
from one
memory
state
to
another
in
the absence
oj
a stirnulus is one of
the
hallmarks
of
the
brain,
and
th'ls is one phenornenon
that
a chaot'ic P R systern has to ern·ulate.
The
goal
of
the
field
of
Chaotic
PR
systems
can
be
expressed as follows:
We do
not
intend
a
chaotic
PR
system
to
report
the
identity
of
a
testing
pattern
with
such
a
"proclamation".
Rather,
what
we
want
to
achieve,
on
one
hand,
is
to
have
the
chaotic
PR
system
give a
strong
peTiodic
or
rnoTe
jTequent
signal
when
a
pattern
is
to
be
recognized.
Further,
between
two consecutive
recognized
patterns,
none
of
the
trained
patterns
must
be
recalled. Finally,
and
most
importantly,
if
an
untrained
pattern
is
presented,
the
system
must
give a
chaotic
signal.
This
paper
deals
with
the
Adachi
Neural
Network
AdNN
[1-5],
which
possesses a
spectrum
of very
interesting
chaotic,
AM
and
PR
properties,
as
described in [6,7].
The
fundamental
"problem"
associated
with
the
AdNN
and
its
variants
is
its
quadratic
computational
requirement.
We shall show
that
by using
spanning
tree
concepts
and
an
effective
gradient
search
strategy,
this
burden
can
be
reduced
to
be
linear,
and
yet
be
almost
as effective
with
regard
to
the
chaotic
and
PR
characteristics.
2
Limitations
of
the
Current
Schemes
Although
the
works of
Adachi
et
al
and
Calitoiu
et
al
were
ground-breaking,
it
turns
out
that,
as
stated
in
[9],
the
results
claimed
in
the
prior
art
l
were
not
as precise as
stated.
Apart
from
this
limitation,
the
computational
burden
is
excessive,
rendering
it
as
an
impractical
machine.
More
precisely, we
mention
the
following "drawbacks"
of
the
state-of-the-art:
1.
The
AdNN's
power as a
PR
system
requires excessive
computations
-
of
the
order
of
O(N2). For
the
examples
cited
by
Adachi
et
al
and
Calitoiu
et
al
[1-7], which use 10 x 10 pixel
arrays,
this
involves la, 000
computations
peT tirne step.
2.
The
PR
capabilities
of
the
AdNN
are
limited, as will
be
explained
presently.
Our
intention
is
to
enhance
these.
1
The
state
of
the
art
of
the
AdNN
and
its
variants
can
be
found
in [6,7,9J.
286
K.
Qin and
B.
1.
Dammen
3.
It
is also
appropriate
that
the
PR
limitations
of
the
work of
Calitoiu
et
al
[7]
are
mentioned.
Although
the
experimental
results
cited
in
[7]
dis-
playa
reduction
in
the
transient
phase
(when
compared
to
the
AdNN)
,
unfortunately,
the
M-AdNN
does
not
achieve
PR
for
untrained
patterns.
It
turns
out
that
although
unreported
in
[7],
the
M-AdNN
always repro-
duces
the
input
patterns
periodically
independent
of
whether
the
input
is a
trained
or
an
untmined
pattern.
This
has
also
been
illustrated
in
[9].
3
Designing
the
L-AdNN
3.1
The
Topology
of
the
L-AdNN
To minimize
the
computational
burden
of
the
AdNN, we shall first
arrive
at
a
topology
with
a
linear
number
of
edges. We
arrive
at
the
modified
linear version in two steps.
First
of all, we reduce
the
number
of
edges
to
be
linear by using
spanning
tree.
The
second
step
will involve
the
computation
of
the
weights
associated
with
this
new
structure,
which we will
address
subsequently.
Of
course,
the
details
of
these
steps
cannot
be
presented
here
due
to
space
limitations,
but
they
can
be
found
in
[8].
Since
the
network
topology
has
been
changed
and
we
want
the
L-AdNN
to
maximally
approximate
the
original
AdNN,
we
propose
to
use
the
Maximum
Spanning
Tree
(MST)
of
the
AdNN
(with
the
edge weights given
by
their
absolute
values)
to
be
the
initial
linear
approximation
to
the
completely-
connected
AdNN.
This
is formalized
by
Algorithm
Topology-.1-AdNN below.
Unlike
the
AdNN
and
its
variants,
this
initial
approximation
of
the
L-AdNN
Algorithm
1 Topology
-.1-
AdNN
Input:
N,
the
number
of
neurons
in
the
network,
and
a
set
of
p
patterns
which
the
network
has
to
"memorize".
Output:
The
topology
and
initial
weights
of
the
L-AdNN.
Method:
1:
Create
a
completely
connected
graph
9 which is
to
represent
the
AdNN.
2:
Compute
the
weights
of
the
edges
of
g,
{Wij},
by
the
following:
Wij
=
~
L~=l
(2xi -
1)
(2xj
-1),
where
xi
is
the
ith
component
of
the
sth
stored
pattern.
3:
Create
a
non-negative
modified
set
of
edge-weights for
the
graph
as:
W;j
=
IWijl
Vi,]
4:
Compute
the
MST
of
9
with
the
associated
weights
of
{w;j}
as
the
initial
approximation
for
the
L-AdNN.
End
Algorithm
Topology
_L-AdNN
is
tree-shaped,
since we have
pruned
the
edges which we believe
are
the
most
"redundant"
.
In
this
regard,
since
that
the
original
AdNN
has
N
neurons
and
N
2
edges, we
recommend
the
use
of
the
Kruskal
algorithm
which is
much
more
efficient
than
the
Prim
algorithm
in such a
setting.
Enhanced Tree-Shaped Adachi-Like Chaotic Neural Network 287
3.2
The
Weights
of
the
L-AdNN:
Gradient
Search
Since we have removed
most
of
the
"redundant"
edges from
the
completely-
connected
graph
by
using a
MST,
it
is clear
that
the
NN
at
hand
will
not
adequately
compare
with
the
original AdNN.
Thus,
our
next
task
is
to
de-
termine
a new
set
of weights so as
to
force
the
L-
AdNN
to
retain
some of
its
PR
properties,
namely
those
corresponding
to
the
trained
patterns.
We
briefly
explain
below (for
more
details
the
reader
is
requested
to
refer
to
[8])
the
process for achieving this.
The
L-AdNN
is defined
by
following equations:
xf(t
+
1)
= !(TJf(t + 1) +
~f(t
+ 1)),
TJf(t
+
1)
= kfTJf(t) + L
wf
x;(t),
eijET
(1)
(2)
(3)
where
{wf},
xf,
~f
and
TJf
are
the
weights,
outputs,
and
state
variables
of
the
L-AdNN
respectively,
and
have similar meanings
to
{Wij},
Xi,
~i
and
Tli
of
the
AdNN.
In
order
to
find
the
optimal
values
of
{wf},
we
define
the
square
error
between
the
original
output
of
the
AdNN
and
new
output
at
the
nth
step:
N
1""
A L 2
Ep
=
2"
~(Xi
,p
-
Xi
,P(n)) ,
(4)
k=l
where
xt,p
and
.y;'P
imply
the
outputs
of
the
ith
neuron
when
the
pth
pattern
is
presented
to
the
AdNN
network
and
the
L-AdNN
network
respectively.
The
overall global
error
is defined
by
E =
"L:=l
Ep
where P is
the
number
of
trained
patterns.
In
order
to
adjust
wfj
to
obtain
the
least
global
error
E,
we consider
the
gradient,
Llwfj,
and
move
wfj
by
an
amount
which equals
Llwfj
in
the
direction
where
the
error
is minimized.
This
can
be formalized as below:
8x;'P(n)
owfj
p
=
(3
L(xt'P
- x;,P(n)) .
~
. x;'P(n) .
(1
- xf,p(n)) . xf,p(n),
(5)
p=l
where
(3
is
the
learning
rate
of
the
gradient
search.
The
formal
algorithm
which achieves
the
update
is given in
Algorithm
2.
The
results
of
a typical
numerical
experiment
which proceeds along
the
above
gradient
search
are
shown in
Figure
1.
In
these
simulations,
we
have chosen
the
learning
rate
(3
to
be
0.05.
The
reader
will observe
that
the
global
error,
E,
converges
very