Paul Cilliers
University of Stellenbosch, ZAF
A central philosophical problem, one that has concerned scientists as much as philosophers, is the relationship between our descriptions of the world and the world itself. This problem is present in one way or another in many different theoretical discourses: in discussions of the status of models and theories in science (instrumentalism, reductionism, realism, etc.), in theories of representation, theories of meaning, and in the realm of law and ethics.1 I do not want to propose a final solution to this problem, but in order to clarify the issue, I want to analyze one of the central notions in most of the discourses mentioned, namely, that of rules. More specifically, I want to investigate the use and the status of rules when we deal with complex phenomena like the brain, language, or social and cultural systems.2
In an extended analysis of complex systems (Cilliers, 1998), I argue that a rule-based approach is not adequate when we want to model complex systems. The argument employs poststructural perspectives (mainly those of Derrida) in order to show that the intricate and dynamic network of relationships between the components of a complex system can be understood better in terms of connectionist (or neural network) models. The practical limitations of current neural networks, specifically the severe limitations of feedforward networks, are acknowledged, but nevertheless it is proposed that distributed systems with recurrency3 can serve as general models for complex systems. I will not repeat all those arguments here; however, I do want to address an (incorrect) impression that may arise from them: that rules are not important or useful. As a matter of fact, we cannot do without them, but we need to analyze more closely what their function and status could be. It is also necessary to investigate how descriptions based on rules and descriptions based on relationships relate to each other. I will start with a first-level consideration of this last question, and return to it in more detail later.
In order to demonstrate the difference between a rule-based and a relationship-based description, one can make use of two different approaches to language. On the one hand, there is the attempt to model language in terms of a formal system of rules. This approach is exemplified best by the early work of Chomsky. On the other hand, there is a description of language in terms of a system of differences, best exemplified by Saussure. Although any sophisticated theory of language will not be limited to one of these, the distinction remains useful. The relationship-based approach attempts to deal with patterns within the system as a whole,4 while the rule-based approach attempts to find fundamental units in the system, and identify the rules that combine and transform them. This distinction remains central to many disputes in cognitive science, as well as between different philosophical schools.5
A similar kind of distinction is made in complexity theory between things that are “complicated” and things that are “complex.” Something that is complicated can have many components, and can be quite intricate, but the relationships between the components are fixed and clearly defined. We can use the analytic method to analyze complicated things, i.e., we can take them apart and put them back together again, like a jumbo jet. Something that is complex, on the other hand, is constituted through a large number of dynamic, nonlinear interactions. Therefore the important characteristics of a complex system are destroyed when it is taken apart, i.e., when the relationships between components are broken. Living things, language, cultural, and social systems are all complex. The behavior of complicated things can be described by rules; the behavior of complex systems is constituted through relationships.
This distinction can also be used to make another point: complex things have emergent properties, complicated things do not. Emergent properties are those we cannot predict merely by analyzing the components of the system. Consciousness is an emergent property of the brain that cannot be predicted by examining a neuron. The behavior of complicated things, however, is predictable—as it mostly should be. No one would fly in a jumbo jet with emergent properties.
The notion of emergent properties is a problematic one for many. They feel that it refers either to something mystical, or to something we cannot, or do not want to, analyze. One can attempt to counter these anxieties by arguing that emergent properties have nothing to do with mysticism, that emergence does not involve metaphysical components, that we are merely talking about properties that arise because of nonlinear, dynamic interactions of which there are so many that we cannot hope to contain all the information involved. According to this argument, “emergent properties” can be renamed to something like “relational properties.” Although I concur with this argument, there is a price to pay for it: it undermines the very distinction between complex and complicated. If we claim that complexity is not a metaphysical thing, that we can talk about complex systems in a materialist way, then we have to grant that the small-scale interactions in a complex system are such that basic physical principles, like causality, should hold. Although it may be impossible to describe these micro-interactions in practice, it should in principle be possible to describe them in terms of rules, and therefore, at bottom, everything is only complicated.
To grant that the distinction between complex and complicated is an analytic one—useful in theory, but not always maintainable in practice— does not change our understanding of complex systems significantly. The more pressing issue here, however, is whether this implies that the distinction between relationships and rules is also dissolved. I wish to argue that it is not. In order to do that, we need to analyze the notion of a rule in more detail.
The rule-based approach to complex systems is one we know well from mainstream artificial intelligence (AI) research in the last 30 years (see Pinker & Mehler, 1988). It has recently been restated forcefully by John Holland in his book Emergence (1998).6 I will examine this work briefly in order to develop a more differentiated understanding of the notion of a rule.
Holland develops ideas from mainstream AI, influenced by the work done at the Santa Fe Institute.7 He is quite clear about his intentions at the start of the book. He acknowledges that “emergent phenomena also occur in domains for which we presently [sic] have few accepted rules,” like ethics and the evolution of nations, but that he will restrict his study to “systems for which we have useful descriptions in terms of rules and laws” (1998: 3). Although he adheres for the most part to these intentions, one gets the impression that ultimately he thinks that all complex systems are underpinned by a precise set of rules. Indeed, he ends the book talking about “life” and “consciousness” (1998: 246-8). The approach to complexity he suggests is a familiar one, based on formal models. Such models consist of atomistic building blocks (1998: 24-6) whose interactions are determined by a set of formal production rules. The components of the model represent elements of reality on a one-to-one basis, i.e., each component of the model stands for a specific element of reality (1998: 29). It is usually clear that he views these rule-based models as descriptions of reality, although he does sometimes talk about rule-governed systems (1998: 6), which creates the impression that rules are somehow more fundamental to the working of the systems he describes. He claims:
A well-conceived model exhibits the complexity, and emergent phenomena, of the system being modeled, without the obscuring effects of incidentals. (Holland, 1997: 17)
His favorite examples are, predictably, board games like chess and checkers.
Although Holland is correct in arguing that models have to reduce the complexity of the phenomenon being modeled, he does not acknowledge that this is exactly the reason that we cannot have exact rule-based models of complex systems. Because of the nonlinearity of the interactions constituting a complex system, it cannot be “compressed.” Any simplifying model will have to leave out something, and because of the nonlinearity, we cannot predict the significance of what is suppressed. In order to capture all the complexity, we will have to “repeat” the system in its entirety. This is just as problematic. Since complex systems interact with their environment in intricate ways, it is never obvious where the limits of the system are. When we deal with complexity, we cannot avoid framing our description thereof in some way or another. Models can, therefore, not function in an objective way, they have to be interpreted.
I have criticized purely formal models in detail (Cilliers, 1998: 13-15, 58-74), mainly because of the shortcomings of a traditional theory of representation. Something that can be understood fully in terms of a set of rules can, at best, be complicated. It is, therefore, my contention that the formal systems that Holland describes cannot show emergent behavior. Chess may indeed have many possibilities that have not yet been realized, but all of these novelties can still be understood in terms of the basic, static, timeless rules of the game. This is not comparable to something like consciousness as an emergent property of a large number of neurons.
Holland (1998: 81-114) does, however, also employ the example of neural networks. The connectionist models I suggest as an alternative to formal, rule-based models are, according to Holland, also rule based at bottom. His argument is that the functioning of recurrent neural nets is based on the working of Hebb's rule.8 This “rule” describes the local interaction between neurons that is responsible for the organization of structure in a network. If Holland is correct, neural nets are also rulebased systems.9 This would be another way of collapsing the distinction between complex and complicated, and a vindication of his position that formal systems can have emergent properties. However, I think he is incorrect, and that his mistake is at heart a semantic one: Hebb's rule is not a rule.
The logic of the notion “rule” implies a certain generality. A rule should apply “without exception to the cases subsumed by the description incorporated in the rule” (Winston, 1971: 177). In order for a rule to apply, it has to be established that the specific case at hand is the same as the general case described by the rule. A “good” rule is one that could apply to many cases. It must say something about the system it describes, or, in Holland's terms, the model of the system must be simpler than the system it describes (Holland, 1998: 24). We formulate a rule in order to describe a pattern that we have come to recognize, to formalize a regularity. Furthermore, when working with a system of rules, the different rules should be “articulated” properly, i.e., rules must be linked in such a way that the output of one rule can be used by a next rule as an input. This characteristic of a system of rules is described by the notion of an algorithm.
Such an understanding of the notion “rule” is clearly at odds with a low level, nonalgorithmic principle like Hebb's rule. This is a principle that applies locally between the components of a complex system. It describes a process that has no ideational content, it operates on contingent, low-level information, is not selective, and provides us with no general information about the system as a whole, or even parts of it. The same rule—perhaps it is better to start talking of something like a principle, rather than a rule—operates everywhere in the system. More importantly, it operates in all recurrent neural systems, irrespective of what the function of that specific system is. In the brain, for example, the same principle can be used to organize the motor cortex and the visual cortex; it does not tell us anything about the differences between perception and action. In Derridean terms, we can say that Hebb's rule functions on the level of the trace. There is no similarity between this and a rule that is supposed to capture some essential or general aspect of a system.
To summarize, we can say that Hebb's rule (or the use principle) and rules in a rule-based model of a system do not have the same status. The use principle describes a mechanism for organization in complex systems. It does not tell us anything about the system. Rule-based models generate descriptions of what such systems do, and perhaps how they are supposed to do it. They are not “wrong” or useless, they are all we have when we want to develop an understanding of complex systems. We must merely be clear about the limitations of our models. In order to do this, we cannot just talk of rules in a blanket fashion, we need to differentiate between different kinds of rules.
A distinction is often made between descriptive and prescriptive rules. This distinction relies on the existence of a clear differentiation between facts and values, one that is problematic to maintain,10 especially in the context of complex systems. Since I do not want to move into the normative domain directly,11 I will focus on another distinction, that between regulative and constitutive rules.12
Constitutive rules provide the framework within which one can understand a set of facts. The best way to describe these rules is in terms of the rules of a game. The game does not exist if the framework is not accepted. Regulative rules, on the other hand, are those rules that determine or constrain permissible moves within that framework. A regulative rule thus only makes sense in the context generated by the set of constitutive rules. To serve a double fault only makes sense when one is playing tennis. One is playing tennis only when those participating agree to submit themselves to the rules that constitute the game. There is often no compelling reason to do this other than to play the game.
This distinction can, I believe, be extended to scientific models. In the Newtonian framework, for example, one would use the regulative rule that states “if a force is applied to a mass it will accelerate.” For this rule to make sense, the Newtonian framework must already be constituted. This may bear some relationship to a Kuhnian understanding of science, but my point is a little more specific: If one wants to generate a formal model of a complex system in terms of a set of rules, a model that can be simulated on a computer, the distinction between regulative and constitutive rule should be clearly understood. The nitty-gritty rules of the model, those that determine or constrain its behavior, are regulative rules. One can spend a lot of time developing and refining them, but one should not forget that they only have meaning in terms of the framework constituting that meaning. When dealing with complicated things, the constitutive framework can normally be determined precisely, at least in principle. When modeling complex systems, however, the constitutive framework is not given, nor is it self-evident in a straightforward fashion. In order to generate some general understanding, the framework has to reduce the complexity. A framework is selected in terms of the aims of our description of the system. The quality and usefulness of the model are primarily determined by this selection.
One should also bear in mind that although the constitutive framework may reduce complexity from a certain perspective, it may increase it from another perspective. This is especially the case, ironically perhaps, when we have developed a good framework, one that allows us to see things that were not apparent from previous models. Thus, our models can both conceal and reveal complexity. How we employ our models often has a lot to do with prevalent values in a discipline, something that, without too much exaggeration, we can call style.
From the claim that the constitutive framework for a complex system is not naturally determinable, it may be inferred, incorrectly, that the framework is arbitrary. This mistake is exactly equivalent to the claim, also mistaken, that deconstruction implies relativism.13 However, at this point we are returned to the problem with which this article opened: What is the relationship between our descriptions, or models, of the world and the world itself?
A brief recapitulation of the claims made so far may be helpful. There is nothing mystical about the workings of a complex system. However, since the nature of the system is the result of countless, local, nonlinear, nonalgorithmic, dynamic interactions, it cannot be described completely and accurately in terms of a set of rules. These local interactions themselves can also not be understood in terms of the notion “rule.” Yet, we cannot avoid rules when trying to describe complex systems. We must, nevertheless, always bear in mind that these rules only make sense in terms of a framework that is not naturally given, but that is generated by the description itself. This precludes the possibility that any model of a complex system can be a perfect or exact one. We cannot avoid the reduction of complexity in the process of modeling. However, if we want to argue that models are merely linguistic entities, we are caught in an instrumentalist or relativist position, a position that cannot be coherently maintained. A possible answer begins to suggest itself when we move away from the foundationalist/relativist dichotomy. In order to do this, we have to acknowledge that the language we use to describe the world is not completely independent from it. There is two-way communication between the two, but since both are complex, there is also no perfect fit between them.
There is no reason to conclude from this position that our models are arbitrary. The rules of our descriptions, I would argue, are attempts to say something about the structure of the patterns of interaction in a complex system. This structure is not rule following or rule governed in itself (in the same sense that a stone does not solve differential equations when falling to the ground), nor is it timeless or fixed. A specific system of rules, especially if it is sensitive to the historical nature of complexity, may at times give a very accurate description of this structure. A specific structure may be fairly stable over time and contained within clear limits. The rules used to describe this may be so accurate that we are quite happy to say that they are “true.” Other structures may be much more open, volatile, and interdependent, and therefore not amenable to a rule-based description. If we want to describe such structures, we have no choice but to impose limits on it in order to make the description possible. In the latter case, the distortion caused by the model will be more significant.
Two problems compound the issue even further. The first is that patterns of all kinds are possible in a complex system, from the stable to the ephemeral and everything in between. We have no a priori method able to provide a guarantee that we are dealing with an aspect of the system that is stable, and likely to remain so. The second problem has to do with the algorithmic nature of a system of rules, i.e., that they have to articulate cleanly. The patterns of structure in a complex system are much more messy. Their borders are not clearly defined. They overlap and interpenetrate each other. If a complex system is critically organized, it will also have structural components on all scales of magnitude, and will therefore be maximally sensitive to influences that can change its structure.14 These considerations lead me to conclude that rule-based models will not be able to provide general and accurate descriptions of complex systems, particularly not of things like human beings.
The final question to consider, then, is whether connectionist models can do so.
Can an accurate model of a complex system be implemented in a neural net? For now, my answer is a qualified “no.” All networks used in practice have a specific function and clear limits. Although their internal logic does differ substantially from traditional formal systems, and despite the fact that they can solve certain pattern-recognition problems with greater ease than formal, rule-based models, ultimately they are well framed, receive specific inputs, and produce specific outputs. It may be somewhat tedious, but I can see no reason that they cannot be reduced to a system of rules. Call this Holland's revenge, if you wish.
What, then, becomes of the argument that neural networks can help us in our understanding of complex systems? I still maintain that large-scale, highly connected recurrent networks can serve as a general model of complex systems, but there are two important qualifications. In the first place, I am not aware of any real practical applications of such nets. They are just too cumbersome to work with. In the second place, even if it would become possible to build these models, they would cease to be “models” in the sense that they reduce complexity and thereby improve our understanding of the system.15 From the argument for the conservation of complexity—the claim that complexity cannot be compressed (Cilliers, 1998: 9-10)—it follows that a proper model of a complex system would have to be as complex as the system itself. As a result, the behavior of the model will be as complex—and unpredictable—as that of the system itself.
Is this conclusion a desperate one? It may be for those scientists and managers who still dream of a perfect grip on reality, usually in order to control it. For the rest of us, it serves as a reminder that our capabilities are limited, that there are limits to our understanding of the world.16 Nothing of what I said can be construed as an argument not to engage with those limits enthusiastically.
A previous version of this paper was published in TLE (1999, vol. 17: 39-50) under the title “Règles et systèmes complexes.” Permission to rework the material is gratefully acknowledged.
Bak, P. (1996) How Nature Works, New York: Springer.
Cilliers, F.P. (1991) “Rules and relations: Some connectionist implications for cognitive science and language,” South African Journal of Philosophy, 10(2, May): 49-55.
Cilliers, P (1998) Complexity and Postmodernism: Understanding Complex Systems, London: Routledge.
Cilliers, P (2000) “What can we learn from a theory of complexity?”, Emergence, 2(1): 23-33. Cornell, D. (1992) The Philosophy of the Limit, London: Routledge.
Derrida, J. (1976) Of Grammatology, Baltimore: John Hopkins University Press.
Derrida, J. (1988) Limited Inc., Evanston: North-Western University Press.
Hebb, D.O. (1949) The Organization of Behavior, New York: Wiley.
Holland, J.H. (1997) “Emergence,” Philosophica, 59(1): 1-40.
Holland, J.H. (1998) Emergence: From Chaos to Order, Reading, MA: Addison-Wesley (Helix Books).
Lyotard, J.F. (1986) “Rules and paradoxes and svelte appendix,” Cultural Critique, 5: 209-19. Pinker, S. & Mehler, J. (eds) (1988) Connections and Symbols, Cambridge, MA: MIT Press.
Rawls, J. (1955) “Two concepts of rules,” The Philosophical Review, 64(1, January): 3-32. Reddiford, G. (1985) “Constitutions, institutions and games,” Journal of the Philosophy of Sport, XII: 41-51.
Singer, P (ed.) (1994) Ethics, Oxford, UK: Oxford University Press.
Winston, K.I. (1971) “Justice and rules: A criticism,” Logical Analysis, 14: 177-82.