Glossary of Nonlinear Terms
Compiled by Terry Marks-Tarlow, Keith Clayton, and Stephen Guastello
The following is a
glossary of important concepts in nonlinear dynamics that we believe are the
most central and elementary for understanding the field of nonlinear dynamics.
Like the other material in this Resources
area, it should be regarded as another work in progress. Underlined terms that
appear within the glossary entries below appear elsewhere in the glossary.
There are several references to figures that appear in the main Resources page
as well. For a comprehensive reference resource for these and many other
nonlinear concepts, we recommend Allwyn Scott’s Encyclopedia of Nonlinear Dynamics (2005),
published by Routledge.
Agent-based modeling. A computer
simulation technique that studies the interactions of a large number of
entities known as “agents,” which are units of programming that perceive
situations and make decisions, usually of a specific or local nature. The
objective is to observe global patterns that emerge from these myriad
interactions. See the main Resources
page for an example of an agent-based outcome. Also see emergence, self-organization,
and cellular automata.
Attractor. An attractor is the end-state of a dynamic
system as it moves over time. Once the object or data point goes into the basin
of attraction, it does not leave unless a strong force is applied. The set
of one or more attractors of a dynamic system can be represented visually or
graphically as trajectories in state space, where state space represents
the multidimensional, abstract space of all possible system behavior. There are
four types of possible attractors: fixed points, limit cycles, toroidal
attractors, and chaotic (or strange) attractors. Point attractors are regular,
terminating in a single point in state space. Cycle attractors are also
regular, sometimes oscillating between two or more fixed points, or exhibiting
a sinusoidal pattern over time. Toroidal attractors are semi-regular,
representing coupled rhythms whose ratio of periodicities terminates in an
irrational rather than a rational number, and appearing in state space as a
donut. Chaotic attractors are fully irregular, represented by an aperiodic
trajectory in state space that never repeats or settles to a stable pattern,
whose basin of attraction is often fractal in shape; see chaos.
Regular point and cycle attractors are characteristic of relatively simple
systems. Irregular toroidal and chaotic attractors are more characteristic of
complex systems.
Autopoiesis.
Autopoiesis is the tendency for complex, dynamical
systems, especially biological ones, to self-organize so as to maintain
cohesion and identity over time. Whether existing at the level of a cell,
organ, organism, or group of organisms, all autopoetic systems dissipate energy
in order to remain a bounded unity. Autopoietic
systems maintain operational closure, which allows them to conserve their
internal organization. At the same time, such systems remain structurally
coupled to their context. This enables the exchange of matter, information, and
energy across open borders, plus the adaptation of such systems to the external
environment.
Basin of attraction. A region in
phase space associated with a given attractor. The basin of attraction of an attractor
is the set of all (initial) points that go to that attractor.
Bifurcation. A bifurcation is a pattern of instability that
often manifests as a sudden, spontaneous change in the attractor pattern
of a dynamical system. Within nonlinear states, as control parameters are
increased or decreased smoothly, bifurcations often arise abruptly at
transition zones in response to tiny changes in a control parameter.
Within graphical depictions of state space, bifurcations appear as
crossroads in a system’s trajectory, such as the switch from a fixed point to a
limit cycle attractor or the progression from order to chaos,
whose bifurcation sequence reveals fractal structure. In the reverse situation,
where order self-organizes spontaneously out of chaotic bases, complexity
builds as bifurcations reduce entropy in local areas. When applying nonlinear theory to living organisms,
bifurcations can be inherent in either discrete state changes that occur within
real time or discrete stage changes that occur within developmental time. In
the human infant unevenness in the emergence of new capacities means that
different bifurcations exist for different developmental functions, such as
speech or motor coordination, both within and between individuals. Within the
experience-dependent, self-organizing right brain, transitions in development,
both towards greater order or the breakdown of previous order, can be graphed
as one or more bifurcations in state space. Bifurcations are inherent in all catastrophe
models.
Bifurcation diagram. Visual
summary of the succession of period-doubling produced as a control parameter is
changed. Also see logistic map.
Catastrophe. A
catastrophe is a discontinuous change of events, which is produced by a process
that involves an underlying continuity. According to catastrophe theory, all
discontinuous changes of events can be modeled by one of seven elementary
topological models (with qualifications). The models vary in complexity, which
is illustrated by the number and type of attractors, order parameters,
control parameters, and bifurcations that are involved in the
process. Catastrophe models are useful for describing the global changes that
result from self-organizing events. The cusp catastrophe model, which is one of
the most widely used of the elementary seven models, is shown on the main Resources page.
Cellular
automata. Cellular automata are one of the earliest forms of agent-based
modeling wherein agents are depicted as cells on a field of graph paper.
Each cell interacts with, and produces an effect upon adjacent cells according
to some pre-programmed rules. The objective is to observe patterns of cell
behaviors after the process has run for a sufficiently long time. Also see emergence,
self-organization.
Chaos. Chaos describes the behavior of a system that appears
random, but is actually produced by deep order underneath. Chaos can be
characterized by simple deterministic equations. The hallmark of a system in
chaos is sensitive dependence on initial conditions, which means that slight
changes in starting places dramatically alter the dynamical system’s course.
Chaotic systems are deterministic, in that current behavior is based precisely
upon past states, even though future states are fundamentally unpredictable.
Numerical sequences that are generated by chaotic equations are also bounded
and non-repeating; both of these principles are matters of degree. The basin or
outer rim of a chaotic attractor is a fractal pattern. Chaos has been
identified in physiological, human social, and economic phenomena.
Closed system. Also known as a Hamiltonian system, a closed
system in one in which the entities inside the system have no interaction with
entities outside the system. Closed systems are conservative of energy, unlike dissipative
systems. For real-world systems, the designation of open or closed is more
of a matter of degree. A system containing water, vapor, a sealed container,
and a heat source would be closed. A loose social network where members of the
network can join or leave regularly is relatively open.
Complexity theory. Complexity theory involves the study of
nonlinear dynamical systems containing “order for free,” which means that no a
priori order exists until its spontaneous emergence without importation from
outside the system; also see self-organization. In complex systems,
order emerges at a global level, often the outcome of many interactions
following simple rules at local levels. In complex adaptive systems that
characterize most life forms, complexity can structuralize as internal maps of the
organism’s own behavior or internal states in relation to the external physical
or social environment. When dynamical systems exist close to equilibrium, there
is a minimal exchange of matter, energy or information across open borders, and
system behavior is often simple and stable. By contrast, when dynamical systems
exist in conditions far from equilibrium, high flows of matter, energy, or
information across open borders lead to unstable and nonlinear behavior. Under
extreme non-equilibrium conditions, system order can break down. Under optimal
conditions, at the edge of chaos, nonlinear systems self-organize to higher
complexity spontaneously and unpredictably, according to intrinsic dynamics.
Complex system. A system
that has multiple parts that interact to produce results that cannot be
explained by simply specifying the roles of the various parts. A complex adaptive system is a complex system that
changes its internal structure to meet the demands that arise from places outside
the system or from changes within the system.
Control parameter. In a
nonlinear dynamical system a control parameter affects the behavior of the
system in any of a number of ways, such as increasing the variability of a
response, or triggering a discontinuous change or qualitative difference in the
system’s state. It is akin to an “independent variable” in conventional
research except that control parameters have more specific roles in the
dynamics of a system than a simple additive effect. Also see order parameter.
Correlation dimension. A
calculation for the fractal dimension that is usually applied to time
series data. It is similar in principle to the Hausdoff
dimension, except that it covers the time series with circles of fixed
radii instead of boxes. The “correlation” aspect of the computation is based on
the degree of similarity between one observation and others later on in the
time series. The Grassberger-Procaccia algorithm for
calculating a correlation is a widely-used computation for the correlation
dimension, although its limitations when used with real data are now well
known.
Coupled dynamics. Coupled dynamics occur when two dynamical
systems become highly interdependent, and they as a single complex system.
Coupled dynamics extend from primitive, neural and physiological levels to
higher order psychological and social levels. Coupled linkages are one
important way that emotional complexes, knowledge, personal history and culture
become transmitted, both implicitly and explicitly. Coupled dynamics provide
the engine for the neurobiology of attachment.
Difference equation. A function
specifying the change in a variable from one discrete point in time to another.
Difference equations are discretized differential equations.
Differential equation. A function
that specifies the rate of change in a continuous variable over changes in
another variable. The other variable is usually time in nonlinear dynamical systems. Differential equations can be
linear or nonlinear, although the nonlinear varieties are far more frequent and
relevant to nonlinear dynamics.
Dimension. See embedding
dimension, Hausdorff dimension, correlation dimension, information
dimension.
Dissipative
systems. A system that is characterized by semi-permeable boundaries and
which leaks energy into the environment. Dissipative symptoms were first
thought to be symptomatic of a system that would eventually suffer from “heat
death.” It is now known that dissipative systems self-organize to
maintain their functionality. Also see closed system.
Dynamic system. A set of
equations specifying how certain variables change over time. The equations
specify how to determine (compute) the new values as a function of their current
values and control parameters. The functions, when explicit, are either
difference equations or differential equations. Dynamic systems may be
stochastic or deterministic. In a stochastic system, new values come from a
probability distribution. In a deterministic system, a single new value is
associated with any current value.
Embedding dimension. Successive
N-tuples of points in a time series are treated as
points in N dimensional space. The points are said to reside in embedding
dimensions of size N, for N = 1, 2, 3, 4 ... etc.
Emergence. Emergence is the hallmark of complex dynamical
systems, by which novel and unexpected structure, pattern or process arises
spontaneously in self-organizing systems. Emergence represents a “bottom-up”
process of evolution and change, whereby complexity at a higher level of description
arises from lower levels in nonlinear fashion out of a myriad of local
interactions. With emergence, the global outcome cannot be predicted, even with
a thorough understanding of constituent elements and local rules of
interaction. In contrast to “top-down,” models of development and change, with
linear chains of cause-effect, emergence arises out of multi-directional,
circular, reciprocal feedback loops that operate in parallel across multiple
size or time scales or levels of description. The concept of emergence
pre-dates most of nonlinear dynamical systems theory. One of its earliest
objectives was to explain how a social group was more than the result of
actions of individuals.
Entropy. Entropy is a measure of unpredictability in a system as it
changes state. In
Far from equilibrium. Under far from equilibrium conditions, a
dynamical system exhibits the continual exchange of matter, energy and
information across open boundaries. Close to equilibrium, dynamical systems
maintain homeostasis. Here they may fluctuate to some degree, but are unlikely
to exhibit large-scale change. By contrast, far from equilibrium, dynamical
systems operate under pre-requisite conditions to self-organize out of chaotic
bases into higher levels of complexity. When water streams in conditions close to
equilibrium, it maintains a smooth, laminar flow. When existing in conditions
far from equilibrium, water becomes turbulent, and its molecules self-organize
into a complex series of vortexes that exhibit fractal structure.
Fractals. Fractals are defined technically as geometrical
structures displaying fractional dimensionality and more loosely as complex
shapes displaying detail on multiple size or time scales. Fractal geometry is
sometimes called the “geometry of nature,” because of its ability to model
irregular, recursive, rough, and discontinuous patterns that are characteristic
of both organic and inorganic processes. The hallmark of fractals is
self-similarity, meaning that the pattern of the whole is repeated within its
parts, either exactly or approximately. Fractals also display the related
property of scale-invariance, by which pattern holds across different spatial
or temporal scales. Fractals can manifest either as spatial structures, such as
those observed in the shape of plants or branching patterns in lungs, or can
appear statistically as power laws or mathematical order observed in
time series data, particularly when chaotic processes are involved.
Fractal dimension. A measure of
a geometric object that can take on fractional values. Fractal dimension is
often used as a measure of how fast length, area, or volume increases with
decrease in scale, or as a measure of complexity of a system. Also see Hausdorff
dimension and correlation dimension.
General
linear model. See linear function.
Genetic
algorithm. A computer simulation technique that emulates genetic processes
as agents interact and “reproduce” according to known or hypothetical rules of
genetics. These techniques were first introduced to study genetic processes
literally, but have evolved into a more general class of evolutionary computations that are useful for developing scenarios
for the future of systems.
Hausdorff dimension. A measure of
a geometric object that can take on fractional values; see fractal dimension).
It is also known as the box-counting dimension because it relies on the concept
of placing boxes of equal size over an irregular geometric shape and counting
the number of boxes that are required to cover the target object.
Hysteresis. A shift
between two or more stable states that is usually rapid, repeated and
reversible. Hysteresis effects are signatures of catastrophe models and
typically occur around a bifurcation manifold.
Information. Information
is what is needed to predict the state of a system, given that the system can
take on multiple states, which are usually characterized as discrete or
categorical. Also see entropy.
Information
dimension. A calculation of the fractal dimension that is based on
Initial condition. The starting
point of a dynamic system. See sensitive dependence on initial conditions.
Iteration. Iteration can be understood computationally as
a technique of beginning with X1
run through a function of X to
produce X2. X2 is then run through a
function of f(X), to produce X3, and so on. Iteration is a
quality of nonlinear dynamical systems, by which their future states are
deterministically linked with the history of all past states. Through
iteration, system output at each moment becomes input for processing the next
moment. Within neurobiological structures iteration of underlying algorithms is
important for understanding system dynamics precisely as they move, change,
evolve or devolve over time. Iteration contrasts with the concept of
repetition, where the dynamics of future states can operate independent of past
states. Iterative structures are
inherent in a wide range of nonlinear dynamical processes.
Iterative function. A function
used to calculate the new state of a dynamic system.
Iterative system. A system in
which one or more functions are iterated to define the system.
Limit cycle. An attractor
that is periodic in time, that is, that cycles periodically through an ordered
sequence of states. For continuously-valued variables it is often characterized
by sinusoidal functions.
Linear function. The equation
of a straight line. A linear equation is of the form y = mx
+ b, in which y varies
"linearly" with x. In this
equation, m determines the slope of
the line and b reflects the y-intercept, which is the value that y obtains when x equals zero. Note that the proportionality between x and y is consistent for all values of x, unlike situations involving nonlinear functions. Linear
functions can be expanded or complexified as weighted
combinations of two or more variables xi,
e.g., y = b0 + b1x1 + b2x2 +… bnxn, which is the
common form of multiple linear regression, also known as the general linear
model.
Logistic difference equation. See logistic
map.
Logistic map. xt+1 = rxn[1-
xn].
A concave-down parabolic function that 0<x<1)
can produce a time series of fixed points, oscillations, oscillations within
oscillations, or chaos, depending on the value of the control parameter r (r
> 0). The logistic map diagram is shown on the main Resources page. The logistic map has a substantial history of use
for ecological and population dynamics. It is also an easy means of generating
chaotic data when r > 4.
Lorenz attractor. A
butterfly-shaped strange attractor. It came from a meteorological model developed
by Edward Lorenz with three equations and three variables. It was one of the
first strange attractors studied.
Lyapunov exponent. (Liapunov number). The value of an exponent is a coefficient
of time that reflects the rate of departure of dynamic orbits. It is a measure
of sensitivity to initial conditions and a measure of turbulence in a dynamical
system.
Nonlinear dynamical systems theory. A dynamical system is
any system that moves and changes over time. Nonlinear dynamics is the study of
dynamical systems whose behavioral output is disproportionate to their input.
Relevant concepts include: attractors, bifurcations, chaos,
fractals, self-organization, and sensitive dependence on
initial conditions. When dynamical systems exist in nonlinear states, small
perturbations can carry the capability to trigger substantial changes in the
system’s trajectory, or conversely, large perturbations can alter the system’s
trajectory only slightly, if at all. When nonlinearity characterizes a
dynamical system, its output may be multiplicative or exponential, may be
subject to threshold effects, hysteresis, or sensitive to amplifying or
damping effects from other system components.
Nonlinear function. Any of a wide variety of relationships between
two or more variables such that the dependent measure y is not proportional to the input variable x, e.g. y = x2,
y = sin(x), y = ex.
Note that systems structured as y = b0 + b1x + b2x2 +… bnxn are
sometimes regarded as “linear” because it is possible to substitute a nonlinear
component for one or more linear components in the general linear model.
Open system. A system that has a great deal of information
transmission across its boundaries. It is the opposite of a closed system.
Open systems have a tendency to be dissipative systems as well.
Orbit (trajectory). A sequence
of positions (path) of a system in its phase space.
Order
parameter. A variable that exhibits nonlinear behavior. Order parameters are
closely akin to dependent measures in conventional research. One important
distinction, however, is that order parameters can be studied in their own
right with or without the involvement of control parameters. Another is that
when systems involve two or more order parameters, the two order parameters
influence each other to some extent.
Period-doubling. The change
in dynamics in which an N-point attractor
is replaced by a 2N-point attractor.
Phase portrait. The collection
of all trajectories from all possible starting points in the phase space
of a dynamic system. It is often used to visualize a chaotic or other
deterministic process in the data.
Phase space. An abstract
space used to represent the behavior of a system. Its dimensions are the variables
of the system. Thus a point in the phase space defines a potential state of the
system. The points actually achieved by a system depend on its iterative
function and initial condition (starting
point). It is graphed showing a subsequent DX as a function of X at
each point in time, i.e., a plot of position versus velocity. Also see state
space.
Power laws. A power law is a statistical distribution
where Frequency(X) = aXb;
in dynamical processes b < 0. Power
laws, or 1/f b distributions,
are ubiquitous in nature, representing self-organized criticality, or
the broad tendency of nature to self-organize asymmetrically at the complex
edge of chaos. The distribution of a power law reveals many small-scale events,
a medium number of mid-size events, and relatively few large-scale events. When
a power law is present, there exists a nonlinear, log-log relationship between
the frequency of X (f) and the value of X. For example, the ratio of frequency versus magnitude in
earthquakes as measured by the Richter scale reveals a power law, as does the
ratio between word rank and frequency in English as well as most other natural
languages, which is known in linguistics as Zipf’s
law. Sometimes the exponent associated with f is an integer, as is the case with earthquakes. Sometimes the
exponent is a fraction, representing fractal processes in nature, such as
white, pink, and brown noises, whose distributions are uncorrelated, partially
correlated, and highly correlated, respectively.
Recursive process For our
purposes, "recursive" and "iterative" are synonyms. Thus
recursive processes are iterative processes, and recursive functions are
iterative functions.
Repellors. One type of
limit point. A point in phase space that a system moves away from.
Return map. Plot of a
time series values Xt vs. X
t+1.
Saddle point. A point,
usually in three-space, that both attracts and repels, attracting in one
dimension and repelling to another.
Self-similarity. An infinite
nesting of structure on all scales. Strict self- similarity refers to a
characteristic of a form exhibited when a substructure resembles a
superstructure in the same form.
Self-organization. Self-organization refers to the emergence of
novelty, new levels of integration, and higher levels of order or complexity
within a dynamical system. Self-organization arises spontaneously, often
unpredictably from nonlinear interactions among simple system components. The
concept of self-organization applies to multiple levels of neural,
psychological, social, cultural and historical description.
Self-organized criticality. A critical
point in the life of a system where it suddenly self-organizes into a new
structure. An illustrative example is where a sand pile suddenly avalanches and
becomes a distribution of smaller piles of various sizes. See power law.
Sensitive dependence on initial conditions. Sensitive dependence
means that small differences in starting conditions, as well as tiny
perturbations to a system’s trajectory, can carry the capacity to greatly alter
its future course. Sensitive dependence on initial conditions is the hallmark
of chaotic states and nonlinear systems as they near transition points.
Informally, the quality of sensitive dependence, known as the “butterfly
effect,” means an event as seemingly trivial as a butterfly flapping its wings
can, at critical times, completely alter how a weather system develops. Due to
this quality, it becomes extremely difficult to predict the precise trajectory
of chaotic states and nonlinear systems over the long range.
State. A point in
state space designating the current location (status) of a dynamic system.
State space. An abstract
space used to represent the behavior of a system. Its dimensions are the variables
of the system. Thus a point in the phase space defines a potential state of the
system. For a one-dimensional system the graphic plot of a state space is the
same as a return map. For systems involving two or more dimensions, however,
each point on the plot is a pair of points X,
Y… for the two variables at each
point in time, or sometimes DX versus DY.
Strange attractor. N-point attractor in which N equals infinity. Usually (perhaps
always) self-similar in form. Trajectories within the strange attractor are
sensitive to initial conditions, and are often chaotic, although chaos is not
guaranteed.
Time series. A set of
measures of behavior over time.
Torus. An attractor
consisting of N independent
oscillations. Plotted in phase space, a 2-oscillation torus resembles a donut.
Trajectory (orbit). A sequence
of positions (path) of a system in its phase space. The path from its starting
point (initial condition) to and within its attractor.
Vector. A two-valued
measure associated with a point in the phase space of a dynamic system. Its direction
shows where the system is headed from the current point, and its length
indicates velocity.
Vector field. The set of
all vectors in the phase space of a dynamic system. For a given continuous
system, the vector field is specified by its set of differential equations