Glossary of Nonlinear Terms

Compiled by Terry Marks-Tarlow, Keith Clayton, and Stephen Guastello

 

The following is a glossary of important concepts in nonlinear dynamics that we believe are the most central and elementary for understanding the field of nonlinear dynamics. Like the other material in this Resources area, it should be regarded as another work in progress. Underlined terms that appear within the glossary entries below appear elsewhere in the glossary. There are several references to figures that appear in the main Resources page as well. For a comprehensive reference resource for these and many other nonlinear concepts, we recommend Allwyn Scott’s Encyclopedia of Nonlinear Dynamics (2005), published by Routledge.

 

Agent-based modeling. A computer simulation technique that studies the interactions of a large number of entities known as “agents,” which are units of programming that perceive situations and make decisions, usually of a specific or local nature. The objective is to observe global patterns that emerge from these myriad interactions. See the main Resources page for an example of an agent-based outcome. Also see emergence, self-organization, and cellular automata.

 

Attractor. An attractor is the end-state of a dynamic system as it moves over time. Once the object or data point goes into the basin of attraction, it does not leave unless a strong force is applied. The set of one or more attractors of a dynamic system can be represented visually or graphically as trajectories in state space, where state space represents the multidimensional, abstract space of all possible system behavior. There are four types of possible attractors: fixed points, limit cycles, toroidal attractors, and chaotic (or strange) attractors. Point attractors are regular, terminating in a single point in state space. Cycle attractors are also regular, sometimes oscillating between two or more fixed points, or exhibiting a sinusoidal pattern over time. Toroidal attractors are semi-regular, representing coupled rhythms whose ratio of periodicities terminates in an irrational rather than a rational number, and appearing in state space as a donut. Chaotic attractors are fully irregular, represented by an aperiodic trajectory in state space that never repeats or settles to a stable pattern, whose basin of attraction is often fractal in shape; see chaos. Regular point and cycle attractors are characteristic of relatively simple systems. Irregular toroidal and chaotic attractors are more characteristic of complex systems.

 

Autopoiesis. Autopoiesis is the tendency for complex, dynamical systems, especially biological ones, to self-organize so as to maintain cohesion and identity over time. Whether existing at the level of a cell, organ, organism, or group of organisms, all autopoetic systems dissipate energy in order to remain a bounded unity. Autopoietic systems maintain operational closure, which allows them to conserve their internal organization. At the same time, such systems remain structurally coupled to their context. This enables the exchange of matter, information, and energy across open borders, plus the adaptation of such systems to the external environment.

 

Basin of attraction. A region in phase space associated with a given attractor. The basin of attraction of an attractor is the set of all (initial) points that go to that attractor.

 

Bifurcation. A bifurcation is a pattern of instability that often manifests as a sudden, spontaneous change in the attractor pattern of a dynamical system. Within nonlinear states, as control parameters are increased or decreased smoothly, bifurcations often arise abruptly at transition zones in response to tiny changes in a control parameter. Within graphical depictions of state space, bifurcations appear as crossroads in a system’s trajectory, such as the switch from a fixed point to a limit cycle attractor or the progression from order to chaos, whose bifurcation sequence reveals fractal structure. In the reverse situation, where order self-organizes spontaneously out of chaotic bases, complexity builds as bifurcations reduce entropy in local areas.  When applying nonlinear theory to living organisms, bifurcations can be inherent in either discrete state changes that occur within real time or discrete stage changes that occur within developmental time. In the human infant unevenness in the emergence of new capacities means that different bifurcations exist for different developmental functions, such as speech or motor coordination, both within and between individuals. Within the experience-dependent, self-organizing right brain, transitions in development, both towards greater order or the breakdown of previous order, can be graphed as one or more bifurcations in state space. Bifurcations are inherent in all catastrophe models.

 

Bifurcation diagram. Visual summary of the succession of period-doubling produced as a control parameter is changed. Also see logistic map.

 

Catastrophe. A catastrophe is a discontinuous change of events, which is produced by a process that involves an underlying continuity. According to catastrophe theory, all discontinuous changes of events can be modeled by one of seven elementary topological models (with qualifications). The models vary in complexity, which is illustrated by the number and type of attractors, order parameters, control parameters, and bifurcations that are involved in the process. Catastrophe models are useful for describing the global changes that result from self-organizing events. The cusp catastrophe model, which is one of the most widely used of the elementary seven models, is shown on the main Resources page.

 

Cellular automata. Cellular automata are one of the earliest forms of agent-based modeling wherein agents are depicted as cells on a field of graph paper. Each cell interacts with, and produces an effect upon adjacent cells according to some pre-programmed rules. The objective is to observe patterns of cell behaviors after the process has run for a sufficiently long time. Also see emergence, self-organization.

 

Chaos. Chaos describes the behavior of a system that appears random, but is actually produced by deep order underneath. Chaos can be characterized by simple deterministic equations. The hallmark of a system in chaos is sensitive dependence on initial conditions, which means that slight changes in starting places dramatically alter the dynamical system’s course. Chaotic systems are deterministic, in that current behavior is based precisely upon past states, even though future states are fundamentally unpredictable. Numerical sequences that are generated by chaotic equations are also bounded and non-repeating; both of these principles are matters of degree. The basin or outer rim of a chaotic attractor is a fractal pattern. Chaos has been identified in physiological, human social, and economic phenomena.

 

Closed system. Also known as a Hamiltonian system, a closed system in one in which the entities inside the system have no interaction with entities outside the system. Closed systems are conservative of energy, unlike dissipative systems. For real-world systems, the designation of open or closed is more of a matter of degree. A system containing water, vapor, a sealed container, and a heat source would be closed. A loose social network where members of the network can join or leave regularly is relatively open.

 

Complexity theory. Complexity theory involves the study of nonlinear dynamical systems containing “order for free,” which means that no a priori order exists until its spontaneous emergence without importation from outside the system; also see self-organization. In complex systems, order emerges at a global level, often the outcome of many interactions following simple rules at local levels. In complex adaptive systems that characterize most life forms, complexity can structuralize as internal maps of the organism’s own behavior or internal states in relation to the external physical or social environment. When dynamical systems exist close to equilibrium, there is a minimal exchange of matter, energy or information across open borders, and system behavior is often simple and stable. By contrast, when dynamical systems exist in conditions far from equilibrium, high flows of matter, energy, or information across open borders lead to unstable and nonlinear behavior. Under extreme non-equilibrium conditions, system order can break down. Under optimal conditions, at the edge of chaos, nonlinear systems self-organize to higher complexity spontaneously and unpredictably, according to intrinsic dynamics.

 

Complex system.  A system that has multiple parts that interact to produce results that cannot be explained by simply specifying the roles of the various parts. A complex adaptive system is a complex system that changes its internal structure to meet the demands that arise from places outside the system or from changes within the system.

 

Control parameter. In a nonlinear dynamical system a control parameter affects the behavior of the system in any of a number of ways, such as increasing the variability of a response, or triggering a discontinuous change or qualitative difference in the system’s state. It is akin to an “independent variable” in conventional research except that control parameters have more specific roles in the dynamics of a system than a simple additive effect. Also see order parameter.

 

Correlation dimension. A calculation for the fractal dimension that is usually applied to time series data. It is similar in principle to the Hausdoff dimension, except that it covers the time series with circles of fixed radii instead of boxes. The “correlation” aspect of the computation is based on the degree of similarity between one observation and others later on in the time series. The Grassberger-Procaccia algorithm for calculating a correlation is a widely-used computation for the correlation dimension, although its limitations when used with real data are now well known.

 

Coupled dynamics. Coupled dynamics occur when two dynamical systems become highly interdependent, and they as a single complex system. Coupled dynamics extend from primitive, neural and physiological levels to higher order psychological and social levels. Coupled linkages are one important way that emotional complexes, knowledge, personal history and culture become transmitted, both implicitly and explicitly. Coupled dynamics provide the engine for the neurobiology of attachment.

 

Difference equation. A function specifying the change in a variable from one discrete point in time to another. Difference equations are discretized differential equations.

 

Differential equation. A function that specifies the rate of change in a continuous variable over changes in another variable. The other variable is usually time in nonlinear dynamical systems. Differential equations can be linear or nonlinear, although the nonlinear varieties are far more frequent and relevant to nonlinear dynamics.

 

Dimension. See embedding dimension, Hausdorff dimension, correlation dimension, information dimension.

 

Dissipative systems. A system that is characterized by semi-permeable boundaries and which leaks energy into the environment. Dissipative symptoms were first thought to be symptomatic of a system that would eventually suffer from “heat death.” It is now known that dissipative systems self-organize to maintain their functionality. Also see closed system.

 

Dynamic system. A set of equations specifying how certain variables change over time. The equations specify how to determine (compute) the new values as a function of their current values and control parameters. The functions, when explicit, are either difference equations or differential equations. Dynamic systems may be stochastic or deterministic. In a stochastic system, new values come from a probability distribution. In a deterministic system, a single new value is associated with any current value.

 

Embedding dimension. Successive N-tuples of points in a time series are treated as points in N dimensional space. The points are said to reside in embedding dimensions of size N, for N = 1, 2, 3, 4 ... etc.

 

Emergence. Emergence is the hallmark of complex dynamical systems, by which novel and unexpected structure, pattern or process arises spontaneously in self-organizing systems. Emergence represents a “bottom-up” process of evolution and change, whereby complexity at a higher level of description arises from lower levels in nonlinear fashion out of a myriad of local interactions. With emergence, the global outcome cannot be predicted, even with a thorough understanding of constituent elements and local rules of interaction. In contrast to “top-down,” models of development and change, with linear chains of cause-effect, emergence arises out of multi-directional, circular, reciprocal feedback loops that operate in parallel across multiple size or time scales or levels of description. The concept of emergence pre-dates most of nonlinear dynamical systems theory. One of its earliest objectives was to explain how a social group was more than the result of actions of individuals.

 

Entropy. Entropy is a measure of unpredictability in a system as it changes state. In Shannon’s original conceptualization, information and entropy added up to maximum information, which was the maximum information that was needed to predict the changes in a system. In Prigogine’s revision of the concept, entropy and information were the same entity, because information was generated by a system in motion. Topological entropy, or Komolgorov-Sinai entropy, is the amount of information that is gained or lost as the system evolves, unfolds, or iterates over time.

 

Far from equilibrium. Under far from equilibrium conditions, a dynamical system exhibits the continual exchange of matter, energy and information across open boundaries. Close to equilibrium, dynamical systems maintain homeostasis. Here they may fluctuate to some degree, but are unlikely to exhibit large-scale change. By contrast, far from equilibrium, dynamical systems operate under pre-requisite conditions to self-organize out of chaotic bases into higher levels of complexity. When water streams in conditions close to equilibrium, it maintains a smooth, laminar flow. When existing in conditions far from equilibrium, water becomes turbulent, and its molecules self-organize into a complex series of vortexes that exhibit fractal structure.

 

Fractals. Fractals are defined technically as geometrical structures displaying fractional dimensionality and more loosely as complex shapes displaying detail on multiple size or time scales. Fractal geometry is sometimes called the “geometry of nature,” because of its ability to model irregular, recursive, rough, and discontinuous patterns that are characteristic of both organic and inorganic processes. The hallmark of fractals is self-similarity, meaning that the pattern of the whole is repeated within its parts, either exactly or approximately. Fractals also display the related property of scale-invariance, by which pattern holds across different spatial or temporal scales. Fractals can manifest either as spatial structures, such as those observed in the shape of plants or branching patterns in lungs, or can appear statistically as power laws or mathematical order observed in time series data, particularly when chaotic processes are involved.

 

Fractal dimension. A measure of a geometric object that can take on fractional values. Fractal dimension is often used as a measure of how fast length, area, or volume increases with decrease in scale, or as a measure of complexity of a system. Also see Hausdorff dimension and correlation dimension.

 

General linear model. See linear function.

 

Genetic algorithm. A computer simulation technique that emulates genetic processes as agents interact and “reproduce” according to known or hypothetical rules of genetics. These techniques were first introduced to study genetic processes literally, but have evolved into a more general class of evolutionary computations that are useful for developing scenarios for the future of systems.

 

Hausdorff dimension. A measure of a geometric object that can take on fractional values; see fractal dimension). It is also known as the box-counting dimension because it relies on the concept of placing boxes of equal size over an irregular geometric shape and counting the number of boxes that are required to cover the target object.

 

Hysteresis. A shift between two or more stable states that is usually rapid, repeated and reversible. Hysteresis effects are signatures of catastrophe models and typically occur around a bifurcation manifold.

 

Information. Information is what is needed to predict the state of a system, given that the system can take on multiple states, which are usually characterized as discrete or categorical. Also see entropy.

 

Information dimension. A calculation of the fractal dimension that is based on Shannon’s information function.

 

Initial condition. The starting point of a dynamic system. See sensitive dependence on initial conditions.

 

Iteration. Iteration can be understood computationally as a technique of beginning with X1 run through a function of X to produce X2. X2 is then run through a function of f(X), to produce X3, and so on. Iteration is a quality of nonlinear dynamical systems, by which their future states are deterministically linked with the history of all past states. Through iteration, system output at each moment becomes input for processing the next moment. Within neurobiological structures iteration of underlying algorithms is important for understanding system dynamics precisely as they move, change, evolve or devolve over time. Iteration contrasts with the concept of repetition, where the dynamics of future states can operate independent of past states.  Iterative structures are inherent in a wide range of nonlinear dynamical processes.   

 

Iterative function. A function used to calculate the new state of a dynamic system.

 

Iterative system. A system in which one or more functions are iterated to define the system.

 

Limit cycle. An attractor that is periodic in time, that is, that cycles periodically through an ordered sequence of states. For continuously-valued variables it is often characterized by sinusoidal functions.

 

Linear function. The equation of a straight line. A linear equation is of the form y = mx + b, in which y varies "linearly" with x. In this equation, m determines the slope of the line and b reflects the y-intercept, which is the value that y obtains when x equals zero. Note that the proportionality between x and y is consistent for all values of x, unlike situations involving nonlinear functions. Linear functions can be expanded or complexified as weighted combinations of two or more variables xi, e.g., y = b0 + b1x1 + b2x2 +… bnxn, which is the common form of multiple linear regression, also known as the general linear model.

 

Logistic difference equation. See logistic map.

 

Logistic map. xt+1 = rxn[1- xn]. A concave-down parabolic function that 0<x<1) can produce a time series of fixed points, oscillations, oscillations within oscillations, or chaos, depending on the value of the control parameter r (r > 0). The logistic map diagram is shown on the main Resources page. The logistic map has a substantial history of use for ecological and population dynamics. It is also an easy means of generating chaotic data when r > 4.

 

Lorenz attractor. A butterfly-shaped strange attractor. It came from a meteorological model developed by Edward Lorenz with three equations and three variables. It was one of the first strange attractors studied.

 

Lyapunov exponent. (Liapunov number). The value of an exponent is a coefficient of time that reflects the rate of departure of dynamic orbits. It is a measure of sensitivity to initial conditions and a measure of turbulence in a dynamical system.

 

Nonlinear dynamical systems theory. A dynamical system is any system that moves and changes over time. Nonlinear dynamics is the study of dynamical systems whose behavioral output is disproportionate to their input. Relevant concepts include: attractors, bifurcations, chaos, fractals, self-organization, and sensitive dependence on initial conditions. When dynamical systems exist in nonlinear states, small perturbations can carry the capability to trigger substantial changes in the system’s trajectory, or conversely, large perturbations can alter the system’s trajectory only slightly, if at all. When nonlinearity characterizes a dynamical system, its output may be multiplicative or exponential, may be subject to threshold effects, hysteresis, or sensitive to amplifying or damping effects from other system components.

 

Nonlinear function. Any of a wide variety of relationships between two or more variables such that the dependent measure y is not proportional to the input variable x, e.g. y = x2, y = sin(x), y = ex. Note that systems structured as y = b0 + b1x + b2x2 +… bnxn are sometimes regarded as “linear” because it is possible to substitute a nonlinear component for one or more linear components in the general linear model.

 

Open system. A system that has a great deal of information transmission across its boundaries. It is the opposite of a closed system. Open systems have a tendency to be dissipative systems as well.

 

Orbit (trajectory). A sequence of positions (path) of a system in its phase space.

 

Order parameter. A variable that exhibits nonlinear behavior. Order parameters are closely akin to dependent measures in conventional research. One important distinction, however, is that order parameters can be studied in their own right with or without the involvement of control parameters. Another is that when systems involve two or more order parameters, the two order parameters influence each other to some extent.

 

Period-doubling. The change in dynamics in which an N-point attractor is replaced by a 2N-point attractor.

 

Phase portrait. The collection of all trajectories from all possible starting points in the phase space of a dynamic system. It is often used to visualize a chaotic or other deterministic process in the data.

 

Phase space. An abstract space used to represent the behavior of a system. Its dimensions are the variables of the system. Thus a point in the phase space defines a potential state of the system. The points actually achieved by a system depend on its iterative function and initial condition (starting point). It is graphed showing a subsequent DX as a function of X at each point in time, i.e., a plot of position versus velocity. Also see state space.

 

Power laws. A power law is a statistical distribution where Frequency(X) = aXb; in dynamical processes b < 0. Power laws, or 1/f b distributions, are ubiquitous in nature, representing self-organized criticality, or the broad tendency of nature to self-organize asymmetrically at the complex edge of chaos. The distribution of a power law reveals many small-scale events, a medium number of mid-size events, and relatively few large-scale events. When a power law is present, there exists a nonlinear, log-log relationship between the frequency of X (f) and the value of X. For example, the ratio of frequency versus magnitude in earthquakes as measured by the Richter scale reveals a power law, as does the ratio between word rank and frequency in English as well as most other natural languages, which is known in linguistics as Zipf’s law. Sometimes the exponent associated with f is an integer, as is the case with earthquakes. Sometimes the exponent is a fraction, representing fractal processes in nature, such as white, pink, and brown noises, whose distributions are uncorrelated, partially correlated, and highly correlated, respectively.

 

Recursive process For our purposes, "recursive" and "iterative" are synonyms. Thus recursive processes are iterative processes, and recursive functions are iterative functions.

 

Repellors. One type of limit point. A point in phase space that a system moves away from.

 

Return map. Plot of a time series values Xt vs. X t+1.

 

Saddle point. A point, usually in three-space, that both attracts and repels, attracting in one dimension and repelling to another.

 

Self-similarity. An infinite nesting of structure on all scales. Strict self- similarity refers to a characteristic of a form exhibited when a substructure resembles a superstructure in the same form.

 

Self-organization. Self-organization refers to the emergence of novelty, new levels of integration, and higher levels of order or complexity within a dynamical system. Self-organization arises spontaneously, often unpredictably from nonlinear interactions among simple system components. The concept of self-organization applies to multiple levels of neural, psychological, social, cultural and historical description.

 

Self-organized criticality. A critical point in the life of a system where it suddenly self-organizes into a new structure. An illustrative example is where a sand pile suddenly avalanches and becomes a distribution of smaller piles of various sizes. See power law.

 

Sensitive dependence on initial conditions. Sensitive dependence means that small differences in starting conditions, as well as tiny perturbations to a system’s trajectory, can carry the capacity to greatly alter its future course. Sensitive dependence on initial conditions is the hallmark of chaotic states and nonlinear systems as they near transition points. Informally, the quality of sensitive dependence, known as the “butterfly effect,” means an event as seemingly trivial as a butterfly flapping its wings can, at critical times, completely alter how a weather system develops. Due to this quality, it becomes extremely difficult to predict the precise trajectory of chaotic states and nonlinear systems over the long range.

 

State. A point in state space designating the current location (status) of a dynamic system.

 

State space. An abstract space used to represent the behavior of a system. Its dimensions are the variables of the system. Thus a point in the phase space defines a potential state of the system. For a one-dimensional system the graphic plot of a state space is the same as a return map. For systems involving two or more dimensions, however, each point on the plot is a pair of points X, Y… for the two variables at each point in time, or sometimes DX versus DY.

 

Strange attractor. N-point attractor in which N equals infinity. Usually (perhaps always) self-similar in form. Trajectories within the strange attractor are sensitive to initial conditions, and are often chaotic, although chaos is not guaranteed.

 

Time series. A set of measures of behavior over time.

 

Torus. An attractor consisting of N independent oscillations. Plotted in phase space, a 2-oscillation torus resembles a donut.

 

Trajectory (orbit). A sequence of positions (path) of a system in its phase space. The path from its starting point (initial condition) to and within its attractor.

 

Vector. A two-valued measure associated with a point in the phase space of a dynamic system. Its direction shows where the system is headed from the current point, and its length indicates velocity.

 

Vector field. The set of all vectors in the phase space of a dynamic system. For a given continuous system, the vector field is specified by its set of differential equations