|
|
## <span dir="">Modelling Complex Systems</span>
|
|
|
|
|
|
_C. Ates_
|
|
|
|
|
|
<span dir="">Complexity is associated with the emergent behaviour, where the whole presents more than the summation of the individual parts / actions. Here the key word is the “summation”, that is, the linear behaviour. Complex systems, on the other hand, exhibit nonlinear relationships, such as schools of fish, art colonies. An ant by itself is a relatively simple system when isolated, yet what they can achieve as colony is amazingly complex. This synergetic effect, nonlinearity so to say, is what we observe and interpret as self-organization, which increases the possibility of realizing rare events.</span>
|
|
|
|
|
|
How can we analyse and model the behaviour of such systems? This is typically done via finding recurrent patterns in the changing states of a system. A nice analogy here is the game of chess: (i) 32 elements of 6 types, (ii) a limited number of rules and (iii) a limited space of 8x8 squares. Yet each move has about 10 possibilities – in an average game of 50 moves, this leads to 10<sup>50</sup> sequences. A number greater than (the probable) number of atoms in the universe! Hence, we end up with a practically infinite possibility space of game where no two games can be identical. Herein, “How can someone or a code can play learning it efficiently?” is a very valid question. <span dir=""> </span>The short answer is the existence of recurring patterns that offer similar possibilities and this is what we aim to capture while modelling nonlinear systems with ML.
|
|
|
|
|
|
#### Determinism and chaos
|
|
|
|
|
|
In engineering, we tend to describe the world around us with idealized behaviours, or _summation_ of set of idealized behaviours. This usually make sense if the boundaries of the analysed system are selected properly by reducing the complexity through physically consistent assumptions. If we are an interested in calculating the volume required to transport N number of apples, idealizing the shape as a perfect sphere is a reasonable assumption. Equations of motion do help us to estimate the trajectories of objects of various sizes at low speeds, and the predictions usually match with our observations.
|
|
|
|
|
|
We also love mathematics. Conservation laws are expressed in terms of PDEs, which can be used to estimate the system behaviour starting from an initial state. We should remember that such an analysis is only possible when the boundaries of the system is well defined and all the interactions between the real life agents are represented as different terms in our equations. We also know by experience that changing interactions a system is not simply additive and this nonlinearity usually eliminates the direct use of PDEs, as the mathematical tools we have, including the theory of PDEs, are based on assumptions of additivity. This habit of ours has been further challenged in the second half of the 20<sup>th</sup> century. We have learnt that even if the PDEs describing the behaviour of a system are fully deterministic such as [trajectories](https://en.wikipedia.org/wiki/Lorenz_system), very similar starting conditions can lead to very different future states, monumented as the “butterfly effect”. <span dir=""> </span>
|
|
|
|
|
|
These events now analysed under the hood of chaos theory, a specialized branch of mathematics studying the underlying patterns in dynamical systems appearing to be random at the first glance. These systems are (i) highly sensitive to initial conditions and the (ii) the apparent randomness is a result of nonlinear (inter) connections, feedback loops, self-organization and self-similarity.
|
|
|
|
|
|
_"At one point I decided to repeat some of the computations in order to examine what was happening in greater detail. I stopped the computer, typed in a line of numbers that it had printed out a while earlier, and set it running again. I went down the hall for a cup of coffee and returned after about an hour, during which time the computer had simulated about two months of weather. The numbers being printed were nothing like the old ones. I immediately suspected a weak vacuum tube or some other computer trouble, which was not uncommon, but before calling for service I decided to see just where the mistake had occurred, knowing that this could speed up the servicing process. Instead of a sudden break, I found that the new values at first repeated the old ones, but soon afterward differed by one and then several units in the last decimal place, and then began to differ in the next to the last place and then in the place before that. In fact, the differences more or less steadily doubled in size every four days or so, until all resemblance with the original output disappeared somewhere in the second month. This was enough to tell me what had happened: the numbers that I had typed in were not the exact original numbers, but were the rounded-off values that had appeared in the original printout. The initial round-off errors were the culprits; they were steadily amplifying until they dominated the solution."_
|
|
|
|
|
|
_E. N. Lorenz, <span dir="">The Essence of Chaos</span>, U. Washington Press, Seattle (1993), page 134._
|
|
|
|
|
|
Herein, self-similarity is an interesting way to represent our observations. As we discussed briefly at the beginning, we like and tend to make things simple to describe and analyse what we see around. Almost the entire history of mathematics focuses on “perfect” shapes and descriptions. This is also true for dynamical system analysis. Any textbook on engineering practices (check the books you have on fluid mechanics, heat transfer, mass transfer, reaction kinetics, signal processing or process control) derive the governing equations on continuous, perfectly shaped objects. This had become a reflex in the scientific community: we had accepted that the behaviours generated by the laws we had (e.g., Newtonian mechanics) are to be smooth and continuous. In the 19<sup>th</sup> century, “things” started to fall apart in the classical mechanics so we did replace our equations with probabilistic ones –wave functions– to patch our beloved equations. Discrete behaviour of the nature revealed in the quantum theory did little to change our mathematical interpretation of the world. Formalisms that assume continuity, such as PDEs, do little to unveil the underlying patterns we see in complex systems.
|
|
|
|
|
|
In real life nothing is perfect. We see complexity in nature; the coastlines of a map, branching of a tree, behaviour of a colony, snowflakes, mammalian circulatory systems. The survival of the species literally depends on their ability to detect these patterns. Humans are also extremely talented in pattern recognition. We can easily distinguish a naturally drawn tree from a bad, “unnaturally perfect” tree drawing. The same question arises: how can we capture these motifs if it is that complex? Could there be some simple governing laws? Is there a consistent mathematical definition leading to these complex structures? Can we somehow learn it from our observations? How can we learn them? These questions will constitute the core of our discussions in the following chapters.
|
|
|
|
|
|
\*\*\* to be continued \*\*\* |
|
|
\ No newline at end of file |