Empirical dynamic modeling (EDM) is a framework for analysis and prediction of
nonlinear
dynamical systems. Applications include
population dynamics,
[1]
[2]
[3]
[4]
[5]
[6]
ecosystem service,
[7]
medicine,
[8]
neuroscience,
[9]
[10]
[11]
dynamical systems,
[12]
[13]
[14]
geophysics,
[15]
[16]
[17] and
human-computer interaction.
[18] EDM was originally developed by
Robert May and
George Sugihara. It can be considered a methodology for
data modeling,
predictive analytics, dynamical system analysis,
machine learning and
time series analysis.
Mathematical models have tremendous power to describe observations of real-world systems. They are routinely used to test
hypothesis, explain mechanisms and predict future outcomes. However, real-world systems are often nonlinear and multidimensional, in some instances rendering explicit equation-based modeling problematic. Empirical models, which infer patterns and associations from the data instead of using hypothesized equations, represent a natural and flexible framework for modeling complex dynamics.
Donald DeAngelis and Simeon Yurek illustrated that canonical
statistical models are ill-posed when applied to
nonlinear dynamical systems.
[19] A hallmark of nonlinear dynamics is state-dependence: system states are related to previous states governing transition from one state to another. EDM operates in this space, the multidimensional state-space of system dynamics rather than on one-dimensional observational time series. EDM does not presume relationships among states, for example, a functional dependence, but projects future states from localised, neighboring states. EDM is thus a
state-space,
nearest-neighbors paradigm where system dynamics are inferred from states derived from observational time series. This provides a model-free representation of the system naturally encompassing nonlinear dynamics.
A cornerstone of EDM is recognition that time series observed from a dynamical system can be transformed into higher-dimensional state-spaces by time-delay
embedding with
Takens's theorem. The state-space models are evaluated based on in-sample fidelity to observations, conventionally with
Pearson correlation between predictions and observations.
EDM is continuing to evolve. As of 2022, the main algorithms are
Simplex projection,
[20] Sequential locally weighted global
linear maps (S-Map) projection,
[21] Multivariate embedding in Simplex or S-Map,
[1]
Convergent cross mapping (CCM),
[22]
and Multiview Embeding,
[23] described below.
Nomenclature
Parameter |
Description
|
![{\displaystyle E}](https://wikimedia.org/api/rest_v1/media/math/render/svg/4232c9de2ee3eec0a9c0a19b15ab92daa6223f9b) |
embedding dimension
|
![{\displaystyle k}](https://wikimedia.org/api/rest_v1/media/math/render/svg/c3c9a2c7b599b37105512c5d570edc034056dd40) |
number of nearest neighbors
|
![{\displaystyle T_{p}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/11812d9eb5f2afd4b461d144c578d88dcc3c4d27) |
prediction interval
|
![{\displaystyle X\in \mathbb {R} }](https://wikimedia.org/api/rest_v1/media/math/render/svg/8fb5134661be014f6cc703c2330ed3dc9088a2bb) |
observed time series
|
![{\displaystyle y\in \mathbb {R} ^{E}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/db737b1f715095fc7aa99970d42eadbb5c91f6c3) |
vector of lagged observations
|
![{\displaystyle \theta \geq 0}](https://wikimedia.org/api/rest_v1/media/math/render/svg/598d0eca41cb40ec71c3c10473629af28cf788e2) |
S-Map localization
|
![{\displaystyle X_{t}^{E}=(X_{t},X_{t-1},\dots ,X_{t-E+1})\in \mathbb {R} ^{E}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/c9ad61f65eccc4bb7ffaf57b6c3e6068f5908380) |
lagged embedding vectors
|
![{\displaystyle \|v\|}](https://wikimedia.org/api/rest_v1/media/math/render/svg/31b9981ac84e8a1b538a23de2e99a53bc6e4766d) |
norm of v
|
![{\displaystyle N=\{N_{1},\dots ,N_{k}\}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/4a12abce130e4982e230df406d3cf40df81fdf07) |
list of nearest neighbors
|
Nearest neighbors are found according to:
Simplex projection
[20]
[24]
[25]
[26] is a nearest neighbor projection. It locates the
nearest neighbors to the location in the state-space from which a prediction is desired. To minimize the number of free parameters
is typically set to
defining an
dimensional simplex in the state-space. The prediction is computed as the average of the weighted phase-space simplex projected
points ahead. Each neighbor is weighted proportional to their distance to the projection origin vector in the state-space.
- Find
nearest neighbor: ![{\displaystyle N_{k}\gets {\text{NN}}(y,X,k)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/15322565f51e6ae3290d79e4f46aedc0d77f282f)
- Define the distance scale:
![{\displaystyle d\gets \|X_{N_{1}}^{E}-y\|}](https://wikimedia.org/api/rest_v1/media/math/render/svg/32bd2d7ca1d2c792241cf19e97543f856bb8967d)
- Compute weights: For{
} : ![{\displaystyle w_{i}\gets \exp(-\|X_{N_{i}}^{E}-y\|/d)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/248da2eec7ed6c01057d3ecd1a11f9f4e8ddf1d1)
- Average of state-space simplex:
![{\displaystyle {\hat {y}}\gets \sum _{i=1}^{k}\left(w_{i}X_{N_{i}+T_{p}}\right)/\sum _{i=1}^{k}w_{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/463d23828bcdb346a4690153da9712371e475d40)
S-Map
[21] extends the state-space prediction in Simplex from an average of the
nearest neighbors to a linear regression fit to all neighbors, but localised with an
exponential decay kernel. The exponential localisation function is
, where
is the neighbor distance and
the mean distance. In this way, depending on the value of
, neighbors close to the prediction origin point have a higher weight than those further from it, such that a local linear approximation to the nonlinear system is reasonable. This localisation ability allows one to identify an optimal local scale, in-effect quantifying the degree of state dependence, and hence nonlinearity of the system.
Another feature of S-Map is that for a properly fit model, the
regression coefficients between variables have been shown to approximate the
gradient (
directional derivative) of variables along the manifold.
[27] These
Jacobians represent the time-varying interaction strengths between system variables.
- Find
nearest neighbor: ![{\displaystyle N\gets {\text{NN}}(y,X,k)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/670bef0ba9f26bad81163a91161299acdf721db0)
- Sum of distances:
![{\displaystyle D\gets {\frac {1}{k}}\sum _{i=1}^{k}\|X_{N_{i}}^{E}-y\|}](https://wikimedia.org/api/rest_v1/media/math/render/svg/6f311608e888dcbe66ea61a25c28ed6f040f6c34)
- Compute weights: For{
} : ![{\displaystyle w_{i}\gets \exp(-\theta \|X_{N_{i}}^{E}-y\|/D)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/4bb89d1c51c8877ddd18749237608e1337bcae1b)
- Reweighting matrix:
![{\displaystyle W\gets {\text{diag}}(w_{i})}](https://wikimedia.org/api/rest_v1/media/math/render/svg/d6644a5762c476751685db462fdb8986e74f98a7)
- Design matrix:
![{\displaystyle A\gets {\begin{bmatrix}1&X_{N_{1}}&X_{N_{1}-1}&\dots &X_{N_{1}-E+1}\\1&X_{N_{2}}&X_{N_{2}-1}&\dots &X_{N_{2}-E+1}\\\vdots &\vdots &\vdots &\ddots &\vdots \\1&X_{N_{k}}&X_{N_{k}-1}&\dots &X_{N_{k}-E+1}\end{bmatrix}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/093adaebc48d51889a036fbb0b3548072406ec7b)
- Weighted design matrix:
![{\displaystyle A\gets WA}](https://wikimedia.org/api/rest_v1/media/math/render/svg/7c3308290d5ab75c2039291c6faf190e980cb5c9)
- Response vector at
: ![{\displaystyle b\gets {\begin{bmatrix}X_{N_{1}+T_{p}}\\X_{N_{2}+T_{p}}\\\vdots \\X_{N_{k}+T_{p}}\end{bmatrix}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/35df7cef735799c0f34d6dc3aee1415b4e536375)
- Weighted response vector:
![{\displaystyle b\gets Wb}](https://wikimedia.org/api/rest_v1/media/math/render/svg/f2d98a6a064719626136979434b14a5d1fba5fde)
- Least squares solution (SVD):
![{\displaystyle {\hat {c}}\gets {\text{argmin}}_{c}\|Ac-b\|_{2}^{2}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/f0ede9b4e8caea537a0280f6a4aee500fb051198)
- Local linear model
is prediction: ![{\displaystyle {\hat {y}}\gets {\hat {c}}_{0}+\sum _{i=1}^{E}{\hat {c}}_{i}y_{i}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/0817e3067ad1dce88f843de52d3935130cb31c8d)
Multivariate Embedding
[1]
[12]
[28] recognizes that time-delay embeddings are not the only valid state-space construction. In Simplex and S-Map one can generate a state-space from observational vectors, or time-delay embeddings of a single observational time series, or both.
Convergent cross mapping (CCM)
[22] leverages a corollary to the Generalized Takens Theorem
[12] that it should be possible to cross predict or cross map between variables observed from the same system. Suppose that in some dynamical system involving variables
and
,
causes
. Since
and
belong to the same dynamical system, their reconstructions (via embeddings)
, and
, also map to the same system.
The causal variable
leaves a signature on the affected variable
, and consequently, the reconstructed states based on
can be used to cross predict values of
. CCM leverages this property to infer causality by predicting
using the
library of points (or vice versa for the other direction of causality), while assessing improvements in cross map predictability as larger and larger random samplings of
are used. If the prediction skill of
increases and saturates as the entire
is used, this provides evidence that
is casually influencing
.
Multiview Embedding
[23] is a
Dimensionality reduction technique where a large number of state-space time series vectors are combitorially assessed towards maximal model predictability.
Extensions to EDM techniques include:
- Generalized Theorems for Nonlinear State Space Reconstruction
[12]
- Extended Convergent Cross Mapping
[13]
- Dynamic stability
[4]
- S-Map regularization
[29]
- Visual analytics with EDM
[30]
- Convergent Cross Sorting
[31]
- Expert system with EDM hybrid
[32]
- Sliding windows based on the extended convergent cross-mapping
[33]
- Empirical Mode Modeling
[17]
- Variable step sizes with bundle embedding
[34]
- Multiview distance regularised S-map
[35]
- ^
a
b
c
[1]Dixon, P. A., et al. 1999. Episodic fluctuations in larval supply. Science 283:1528–1530
-
^
[2]Hao Ye, Richard J. Beamish, Sarah M. Glaser, et al. 2015. Equation-free mechanistic ecosystem forecasting using empirical dynamic modeling. Proceedings of the National Academy of Sciences Mar 2015, 112 (13) E1569-E1576; DOI: 10.1073/pnas.1417063112
-
^
[3]Ethan R. Deyle, Michael Fogarty, Chih-hao Hsieh, et al. 2013. Proceedings of the National Academy of Sciences Apr 2013, 110 (16) 6430-6435; DOI: 10.1073/pnas.1215506110
- ^
a
b
[4]Ushio, M., Hsieh, Ch., Masuda, R. et al., 2018. Fluctuating interaction network and time-varying stability of a natural fish community. Nature 554, 360–363
-
^
[5]Deyle E.R., et al. 2016. Tracking and forecasting ecosystem interactions in real time. Proc. R. Soc. B 283: 20152258
-
^
[6]Tanya L. Rogers, Stephan B. Munch, Simon D. Stewart, Eric P. Palkovacs, Alfredo Giron-Nava, Shin-ichiro S. Matsuzaki, Celia C. Symons. Ecology Letters, 23 (8) August 2020, 1287-1297
-
^
[7]Park J., et al. 2021. Dynamics of Florida milk production and total phosphate in Lake Okeechobee. PLoS ONE 16(8): e0248910. doi:10.1371/journal.pone.0248910
-
^
[8]George Sugihara, Walter Allan, Daniel Sobel, and Kenneth D. Allan, 1996. Nonlinear control of heart rate variability in human infants. Proc. Natl. Acad. Sci. USA. Vol. 93, pp. 2608-2613, March 1996. Medical Sciences
-
^
[9]McBride, J. C., et al. Sugihara causality analysis of scalp EEG for detection of early Alzheimer's disease. Neuroimage-Clinical 7:258–265 (2015)
-
^
[10]Tajima S, Yanagawa T, Fujii N, Toyoizumi T (2015) Untangling Brain-Wide Dynamics in Consciousness by Cross-Embedding. PLoS Comput Biol 11(11): e1004537.
https://doi.org/10.1371/journal.pcbi.1004537
-
^
[11]W. Watanakeesuntorn et al., "Massively Parallel Causal Inference of Whole Brain Dynamics at Single Neuron Resolution," 2020 IEEE 26th International Conference on Parallel and Distributed Systems (ICPADS), 2020, pp. 196-205, doi: 10.1109/ICPADS51040.2020.00035
- ^
a
b
c
d
[12] Deyle ER, Sugihara G (2011) Generalized Theorems for Nonlinear State Space Reconstruction. PLoS ONE 6(3): e18295. doi:10.1371/journal.pone.0018295
- ^
a
b
[13]Ye, H., Deyle, E., Gilarranz, L. et al., 2015. Distinguishing time-delayed causal interactions using convergent cross mapping. Sci Rep 5, 14750 (2015). doi:10.1038/srep14750
-
^
[14]Cenci, S., Saavedra, S. Non-parametric estimation of the structural stability of non-equilibrium community dynamics. Nat Ecol Evol 3, 912–918 (2019).
https://doi.org/10.1038/s41559-019-0879-1
-
^
[15]Tsonis A. A., et al. Dynamical evidence for causality between galactic cosmic rays and interannual variation in global temperature. Proc Natl Acad Sci 112(11):3253–3256 (2015).
-
^
[16]Nes EH Van, et al. Causal feedbacks in climate change. Nat Clim Chang 5(5):445–448 (2015)
- ^
a
b
[17]Park, J., et al. Empirical mode modeling. Nonlinear Dyn (2022).
https://doi.org/10.1007/s11071-022-07311-y
-
^ van Berkel, Niels; Dennis, Simon; Zyphur, Michael; Li, Jinjing; Heathcote, Andrew; Kostakos, Vassilis (2021-07-04).
"Modeling interaction as a complex system". Human–Computer Interaction. 36 (4): 279–305.
doi:
10.1080/07370024.2020.1715221.
hdl:
11343/247884.
ISSN
0737-0024.
S2CID
211267275.
-
^
[18]Donald L. DeAngelis, Simeon Yurek, 2015, Equation-free modeling unravels the behavior of complex ecological systems. Proceedings of the National Academy of Sciences Mar 2015, 112 (13) 3856-3857; DOI: 10.1073/pnas.1503154112
- ^
a
b
[19] Sugihara G. and May R., 1990. Nonlinear forecasting as a way of distinguishing chaos from measurement error in time series. Nature, 344:734–741
- ^
a
b
[20] Sugihara G., 1994. Nonlinear forecasting for the classification of natural time series. Philosophical Transactions: Physical Sciences and Engineering, 348 (1688) : 477–495
- ^
a
b
[21] Sugihara G., May R., Ye H., et al. 2012. Detecting Causality in Complex Ecosystems. Science 338:496-500
- ^
a
b
[22] Ye H., and G. Sugihara, 2016. Information leverage in interconnected ecosystems: Overcoming the curse of dimensionality. Science 353:922–925
-
^
[23] Takens, F. (1981). Detecting strange attractors in turbulence. In D. A. Rand & L. S. Young (Eds.), Dynamical Systems and Turbulence (pp. 366–381). Springer.
-
^
[24] Casdagli, M. (1989). Nonlinear prediction of chaotic time series. Physica D: Nonlinear Phenomena, 35(3), 335–356.
-
^
[25] Judd, K., & Mees, A. (1998). Embedding as a modeling problem. Physica D: Nonlinear Phenomena, 120(3), 273–286.
-
^
[26]Deyle ER. et al. 2016. Tracking and forecasting ecosystem interactions in real time. Proc. R. Soc. B 283: 20152258
-
^
[27] Sauer, T., Yorke, J. A., & Casdagli, M. (1991). Embedology. Journal of Statistical Physics, 65(3), 579–616
-
^
[28]Cenci S, Sugihara G, Saavedra S, 2019. Regularized S-map for inference and forecasting with noisy ecological time series, METHODS IN ECOLOGY AND EVOLUTION, 10 (5), 650-660
-
^
[29] Hiroaki Natsukawa, et al. 2021. A Visual Analytics Approach for Ecosystem Dynamics based on Empirical Dynamic Modeling. IEEE Transactions on Visualization and Computer Graphics. Feb. 2021, 506-516, vol. 27
DOI: 10.1109/TVCG.2020.3028956
-
^
[30] Breston, L., Leonardis, E.J., Quinn, L.K. et al. 2021. Convergent cross sorting for estimating dynamic coupling. Sci Rep 11, 20374 (2021). doi:10.1038/s41598-021-98864-2
-
^
[31] Deyle E. R. et al. A hybrid empirical and parametric approach for managing ecosystem complexity: Water quality in Lake Geneva under nonstationary futures. PNAS Vol. 119, No. 26 (2022).
-
^
[32] Ge, X., Lin, A. Dynamic causality analysis using overlapped sliding windows based on the extended convergent cross-mapping. Nonlinear Dyn 104, 1753–1765 (2021).
https://doi.org/10.1007/s11071-021-06362-x
-
^
[33] Bethany Johnson, Stephan B. Munch. 2022. An empirical dynamic modeling framework for missing or irregular samples. Ecological Modelling, Volume 468, June 2022, 109948.
-
^
[34] Chang, C.-W., Miki, T., Ushio, M., et al. (2021) Reconstructing large interaction networks from empirical time series data. Ecology Letters, 24, 2763– 2774.
https://doi.org/10.1111/ele.13897
- Animations
- Online books or lecture notes
- Research groups
-
Sugihara Lab, Scripps Institution of Oceanography, University of California San Diego.