You might think it a little unusual that on an Enterprise IT website we haven't used photos of smartly dressed business executives carrying expensive briefcases or standing around the water cooler finding each other fascinating? Well, we thought about it, but decided to choose images that reflected a core attribute of our Infiniflow product suite - the dynamic, autonomic principles of Complex Adaptive Systems (CAS).

But before we start, just what has CAS to do with modern business systems and the very real problems that face enterprise IT?

All organizations are under increasing pressure to drive out costs, decrease operational complexity, increase business agility and improve service resilience. Whatever technology decisions you make, in order to succeed they must ultimately exhibit the agility and simplicity characteristics of CAS systems.

Enterprise information systems are starting to look more like 'ecologies' in which a variety of platforms and services exist. At some point (a Negroponte shift point) it is necessary to start thinking about this collectively as the environment in which software 'lives' and 'grows'. Analysts have not been slow in recognizing this analogy (see Forrester's 'Organic IT' and Gartner's 'Enterprise Nervous System'), suggesting that, over the next decade, the shift towards such environments will provide significant business opportunity.

Those who succeed in deploying such environments will be rewarded with autonomic environments that automatically adapt in response to damage or increased resource demands from the business. Changing business requirements will be rapidly addressed via service adaptation. As with human autonomic systems, some conscious (in this case external) control over normally self-regulating functions will be possible, however most behaviors will result from a subtle mix of self-controlling and self co-ordinating responses of system components to their environment; realization of these emergent behaviors is the 'nirvana' of organic information systems design.

In contrast, those that attempt to scale legacy 'stove-piped' systems in a conventional manner, face the prospect of managing increasing costly and fragile environments. Instead of providing the foundations for efficiency and flexibility, such environments will act as the straightjacket within which these businesses must attempt to compete.

That said, there is little real guidance on how one should achieve the desired endpoint; the road ahead is littered with marketing obfuscations and tactical, short-term compromises. We're told that to achieve the desired results we need a service-oriented architecture that leverages grid, ESB and caching, with a liberal quantity of XML thrown in, not to mention a plethora of other buzz words and technologies, all sat on a distributed blade or consolidated mainframe utility infrastructure!? In reality, such technologies used without the correct

Complexity

The realization that classical reductionism (i.e. dissect until the basic elements are found) was no longer a valid approach for analysis of systems, and this resulted in the birth of complexity science. Complexity science is the study of interactions and resultant high-order effects, these discernible as semi-persistent patterns that are not explained from the individual composite elements of the system.

An example of such behavior can be found in flocking computer simulations. Here, sets of intricate patterns emerge that seem to steer a group of agents, as if it were a single organism, avoiding obstacles or just bouncing around. This intricate behavior emerges from the following simple rules applied to each individual agent:

• Separation: steer to avoid crowding local flock mates,
• Alignment: steer towards the average heading of local flock mates,
• Cohesion: steer to move toward the average position of local flock mates.

The results of these three rules can be seen in every modern animation movie that features flocking animals.

Meanwhile, research suggests that complexity resides in a region bounded by Order and Chaos (ref: "Creation, Life and How to Make It", Steve Grand - ISBN: 0674011139). In general, orderly systems (structured and change resistant) tend to get less orderly, and chaotic systems (unstructured and rapidly changing) tend to get less chaotic, both evolving towards complex adaptive systems. Hence, complex adaptive systems (CAS) are said to live "on the boundary between order and chaos", being neither linear nor chaotic in nature. As elements of a CAS react to each other, and because elements of the system never stabilize, the whole system is constantly changing, with elements adapting to environmental changes as well as to their counterparts. In this way adaptive systems develop behaviors that enable them to experiment and fail without killing the entire system. Perhaps the most profound conclusion is that the most effective systems (survivable) are those that pursue agility. In addition, many complex adaptive systems exhibit scale-free behaviors, so allowing extremely sophisticated federated systems to be built up by the repetition of simple patterns.

From the chemical autocatalytic networks that ultimately drive biological life, to the ecosphere and the global financial system, all ultimately may be interpreted as software systems executing on various types of computers.

Hence, from flocking behavior to sentience, an emergent, bottom-up complex adaptive system design seems to be how nature puts things together. Software systems that resist such principles are ultimately doomed to limited adaptability to changing environments, and so ultimately, a foreshortened service lifetime; such systems cannot help being highly sensitivity to component failure. Meanwhile systems that embrace these principles should display emergent autonomic properties. Note the inference that autonomic systems cannot be built, true autonomic behavior being fundamentally, an emergent property of a distributed system.

The Characteristics of Complex Adaptive Systems

Whilst such discussions are of interest, they provide little guidance as to how such behaviours should be harnessed in the designs of adaptive business solutions. However, certain key characteristics have been identified in all complex adaptive systems ("Hidden Order", John Holland - ISBN: 0201442302), these providing us with a start point. These characteristics include:

•  Hierarchies: The tendency to form hierarchies in which agents aggregate to create higher order meta-agents or services , whilst remaining loosely coupled with the environment and agents of the same order.

•  Tagging: The exploitation of tagging mechanisms to facilitate the formation of aggregates. A trivial example is a flag or banner used to rally members of an army or a political persuasion. Tagging is used to break resource symmetry.

•  Stigmergy: Individuals are not controlled directly, rather their behavior responds to emergent properties of the group or system to which they belong. Hence such systems many exhibit non-linear behaviors, this a natural result of component aggregation and the response of individuals to emergent properties.

•  Flow optimization: Complex adaptive systems evolve in a manner that optimizes flow. Flow may relate to energy flowing through a biological system, an ecosystem, wealth through the finances markets, or given suitable cost metrics, information flowing through a complex business process.

More Complex Systems in the Natural World

We conclude this discussion with some of the complex structures found in nature. From the Cells <> Software components to the Earth's Biosphere <> World Wide Web, the analogies between structure of natural systems and economic/technology systems are intriguing and profound.

Summary

It appears that Complex Adaptive Systems are fundamental to every system we take for granted in our everyday lives - the world, natural ecosystems, the human body, the financial markets, politics ... you name it!

Armed with this knowledge it is clear that taking advantage of CAS principles provides a powerful and compelling basis for a new era in IT architecture and the software that we use to knit this all together.

The Infiniflow Service Fabric is a realization of these principles and delivers a platform that transforms the way the world runs applications.