Real leaders embracing the paradigm of complexity

Introduction

The above is a quote from an interview conducted in 2007. Almost 10 years have passed since, but we find ourselves returning repeatedly to its core message. The interviewee is one of the leading Finnish experts on health policy and management, who, at the time, was one of the central figures in the planning of the national health service reform. In the interview he reminded us that wicked problems are not going to go away; we just need to learn to live with them. He assured us that as a result of this growing acceptance and the increased understanding of complex systems the escalating levels of pain in our organizations and societies would begin to subside1.

This is not a new idea. It has been around for forty years, since the seminal paper by Horst Rittel and Melvin Webber2 separating tame problems, which are easy to define and also easy to resolve, from wicked problems, that are both very hard to define in a clear and broadly acceptable way, and extremely difficult—in fact well-nigh impossible—to solve to the enduring satisfaction of the contending stakeholders. The concept has become established via a wide variety literature on wicked problems and complexity sciences, as has also arisen in the speeches of world leaders, most notably the former US Secretary of State, Hillary Clinton, when in her memoirs Hard Choices3, she adapted the concept to describe the escalation of the crisis in Syria.

Despite the increasing recognition of wicked problems, research has almost stagnated recently. As the Editor-in-Chief of Landscape and Urban Planning Wei-Ning Xiang4 points out, the bulk of the research on wicked problems seems to follow the same formula: first raising awareness, then preaching acceptance, and finally advocating creative adaptation strategies and innovative approaches. So, after more than forty years we seem to be merely repeating Rittel’s and Webber’s main message. For us this creates two dilemmas. The first relates to the isolation of wicked problems. Although we value the concept of wicked problems as such, we see significant additional value in perceiving wicked problems as part of a wider conceptual framework of the complexity sciences, including chaos theory and complexity thinking. Ideally, study of the complexity sciences will help us understand more clearly why wicked problems are actually wicked. Such understanding would reduce the likely possibility of the wicked problem becoming merely a fashionable phrase.

The second dilemma is rooted in the scarcity of empirical research. We find it disconcerting when more and more issues are defined as wicked problems and the same definite lists of bullet points for resolution are suggested. There is of course an intrinsic value in such writings, but for us this raises the question of what that all means in real life, that is, how leaders who truly understand the nature of wicked problems and who share the worldview of complexity sciences actually come to transform their leadership. In this article we begin to tackle this question. Through informal interviews we present three individuals–David Walker, Tom Hanén and Olli-Pekka Heinonen–who we label Chaos Pilots5. Each is an individual holding a senior leadership position and with a deep understanding of at least one of the following frameworks: wicked problems, chaos theory, or complexity thinking.

Resonance with the paradigm of complexity

The Newtonian worldview emphasizing the presence of a clockwork universe has long been the determining paradigm in the science and practice of leadership and management. The view leads to the world being treated as if it were a “really big machine”6. As such, it could be taken apart, and by studying the parts, the whole could be understood. The paradigm of complexity challenges this mechanistic worldview. Scholars are beginning to view the world as a dynamic and open system exhibiting self-organizing emergent behavior, like a living entity. While all our interviewees share this particular organic worldview, they have their own individual understanding of it, as well as their own path they have followed to arrive at that understanding.

Interestingly, the way David Walker began to apply the principles of chaos theory, is itself a path characterized by emergence and self-organizing. In the mid-1970s, Walker was a student at Cambridge, studying mathematics for both of his first degrees. As it is now well-known–mostly due to James Gleick’s7 bestselling book Chaos: Making a New Science–chaos theory began to emerge at that time, in the 1970s. While French mathematician Henri Poincaré made initial contributions, it is American meteorologist Edward Lorenz who is generally recognized as the discoverer of the theory. The famous butterfly effect is often considered to be the core of chaos theory. This effect describes metaphorically how the flapping of a butterfly’s wings on one side of the globe causes a hurricane on the other, that is, how small events in non-linear, dynamic systems can have significant trajectory altering consequences. More scientifically this phenomenon is known as sensitivity to initial conditions.

As a mathematician, Walker was able to witness the development of chaos theory into an actual academic discipline. Later in his life he was ordained to the priesthood and, since late 2013, he has been the Church of England’s Bishop of Manchester. Walker told us that how as he moved into positions of seniority he began to revisit the meaning and value of chaos theory. As his focus as a senior bishop is less on the present practice of his organization than on its future, chaos theory offers a novel way to approach organization’s possible future paths. Walker’s background gives him an extraordinary opportunity to apply the principles of chaos theory he learned as a mathematician in the field of leadership and management. Instead of just a cursory view, he has a deep understanding of what chaos theory is actually about.

For Tom Hanén, the path has been in a way the reverse of that of Bishop Walker. Hanén is a senior officer in the Finnish Border Guard, and as part of the role, he found himself struggling to understand the unknown. As major crises emerged in Finland, for example Estonia ship disaster and school shootings and such, he began to ponder whether such events should be foreseeable, and why they always seem to catch society by surprise. At the time he was beginning his PhD studies at the National Defence University and soon noticed that in safety/security research a new paradigm had begun to emerge. It had been recognized that these crises are actually composed of many different variables that are difficult to perceive. Hanén then began to understand complexity: how everything resembles combined lumps of modeling clay and how it is hard to know what will emerge. Hanén also acknowledges that there are several ways of comprehending complexity and instead of exclusively adopting the hard complexity science that focuses on computational modeling, or the soft complexity science that works at the level of metaphors, he uses an approach that could be termed complexity thinking. In their editorial in Emergence, Kurt Richardson and Paul Cilliers8 describe complexity thinking as focusing on “the limits of our knowledge in the light of complexity, limits that are often trivialized by contemporary scientific thinking.”

Since the beginning of 2015 Captain (Navy) Hanén has been the commander of Gulf of Finland Coast Guard. Over the years, his understanding of complexity has deepened. He now understands that there is naturally a high degree of connectivity and interdependence between the different actors and dimensions of a system and also between the system and its environment, giving rise to complex behaviors. Hanén feels that fundamentally complexity thinking offers an answer to the question of why everything is as it is. He puts it like this: “In a way it is a liberating image of reality. It just is and we do not need to feel pain about it.” This resonates well with what Professor David Byrne9 has written in his book Complexity and the Social Sciences: “Complexity/chaos offers the possibility of an engaged science not founded in pride, in the assentation of an absolute knowledge . . . but rather in a humility about the complexity of the world coupled with a hopeful belief in the potential of human beings doing something about it.” For both Hanén and Byrne, complexity is then something very natural; not something that should be made to disappear.

Even though Olli-Pekka Heinonen became familiar with complexity sciences at an early stage of his career, it is during the last few years that he has focused on the framework of wicked problems. Heinonen is a former Minister of Education and Minister of Transport and Communications. In 2012 he was appointed as the State Secretary to the Finnish Prime Minister and after the 2015 Finnish parliamentary election as the State Secretary to the National Coalition Party’s group of ministers. For Heinonen, wicked problems bring value to complexity sciences: the concept creates a concrete tool. It helps to understand the nature of the problem and to select appropriate resolution methods. The relationship between wicked problems and complexity sciences is then twofold. Where the concept of wicked problems brings something concrete to complexity sciences, complexity sciences helps improve the understanding of when and how wicked problems occur.

Heinonen considers that the concept of wicked problems has had a really strong explanatory power in his work. As State Secretary to the Prime Minister his task was to assist him in coordinating the preparation and processing of matters that fall under the responsibility of the Government of Finland. Heinonen tells us that these matters are often wicked and, most importantly, horizontal by their very nature. It would not be productive to attempt to resolve these matters by adopting an individual point of view, as such reductionist approaches rarely lead to sustainable resolutions. Heinonen feels that wicked problems faced by the Government of Finland have increased from the 1990s to the 2010s due to the increased interdependencies. He encapsulates the challenges in a following way: “These problems have no beginning and they do not have an endpoint. The question is only how we can get to grips with these problems a little better and understand that the approaches to resolving wicked problems are indeed so different...”

Transforming leadership practices

All of our interviewees feel that the paradigm of complexity has influenced their leadership. For example, State Secretary Heinonen tells us how he has begun to understand that there is no one way to manage; that different problems and different environments call for different approaches. Through this increased understanding he feels he has become more sensitive to the initial conditions. This is seen in the way Heinonen approaches wicked problems. He emphasizes the collective creation of situational awareness. He devotes time to establishing who are the relevant stakeholders in relation to the wicked problem. As these people then come together, first it is important to see how each views the problem. If differences in perceptions emerge, an attempt is made to establish a shared understanding through deliberation. Heinonen also reminds us that as wicked problems are usually symptoms of other problems, it can be useful to try to see if an upper-level problem can be defined, and through it to find a vision that links each issue to the common process. In that process of discovery, Heinonen advises the participants against retaining their own agendas, “We have to be in a way as naked as possible in our commitment”, he explains. Heinonen takes the creation of the Government Administration Department as an example of such a process. This new 500 strong department pools all the administration and specialist services functions from the various ministries under the wing of the Prime Minister’s Office. Its creation was a joint project by the ministries' Permanent Secretaries and was implemented as a direct response to the horizontal nature of the problems each faced, and serves to steer operations away from institutional silos.

For Capt. (N) Hanén, the main change required of his leadership style has been the reduction of straightforward thinking. It is apparent that for an increasing number of issues there is no right or wrong answer. Complexity creates problems that have many dimensions, some of which are the responsibility of some actor, some of which cannot be easily assigned to anyone. In a similar way as Heinonen, Hanén then sees the value in the escalation of dialogue, sharing of information, and joint action. Hanén also understands that complexity partially challenges scenario-based risk modeling and similar approaches. In the words of Professor Paul Cilliers10, to model complex systems we would have to “model life, the universe and everything”, which is of course unmanageable. Hanén, however, reminds us that the value of such approaches is not so much in their promise to predict the future, but in that they make us talk, and to imagine what the future might hold. He accepts that surprises are inevitable and understands that conflicts and surprises are often the results of the general principles of complex systems, and are therefore natural components of planning processes11. We do not know in advance the scale of the surprise, nor who will be involved. An example can be found in the Command Centre of the Gulf of Finland Coast Guard, which has been modified to meet the needs of uncertainty: as Hanén points out, “We have built desks with communication links without knowing who we have built them for. They are meant for future…for that surprising event.”

In addition, Bishop Walker feels that his understanding of chaos and complexity has allowed him to approach different processes in his organization with less control and more freedom. He told us, “My role as senior bishop in the Diocese of Manchester is not to run the system smoothly but to disturb it and then help it to find a new and different equilibrium, before repeating the exercise.” Instead of controlling from above he is enabling different groups to find their own ways of working, letting them self-organize and then potentially disturb the whole organizational culture. He operates then at the edge of chaos—between order and chaos—where non-linearity and sensitivity to initial conditions are explicit. In such a state, the system has just the right amount of both structure and looseness12. When finally a new order has been decided upon, Walker strives to ensure that it is truly implemented. While he understands that stability is not a normal state of the system, he emphasizes that this particular execution phase is important as “it avoids a situation where one never reaches a conclusion.”

Bishop Walker clearly has the characteristics of a Chaos Pilot, a person who navigates within self-organizing dynamics, believing that there is a time for contemplating and facilitating different choices and a time for spontaneous self-organization and emergence5. Being a Chaos Pilot is then to know when to allow the flapping of the butterfly’s wings to determine events and when to influence matters by one’s own actions. What makes Walker a true Chaos Pilot is the way in which he embraces complexity. He accepts that he does not know what the end result will be, but that the important thing is that the process is highly likely to lead to a good place. Lichtenstein and Plowman13 presented similar findings in their research, showing how embracing uncertainty and encouraging novelty can truly “catalyze and enhance the creating of new order.” Even though in our previous research, the Chaos Pilot designation was created as an ideal type of ‘chaos management’, it is encouraging to see that there exist real counterparts like Walker, Heinonen, and Hanén.

Challenges in embracing the complexity

It is not, however, an easy task to embrace complexity14. Sometimes the systemic structure of the organization is so rigid that acting differently is extremely difficult. Capt. (N) Hanén and State Secretary Heinonen describe how the ideas of scientific revolution are still strong in modern organizations. For example, if we do not know something, it is thought to be so just because we do not yet have enough information. There is a reluctance to accept that the presence of complexity makes a precise understanding of reality impossible. In addition, Heinonen feels one problem is that the predominant hierarchies create strong incentives to think hierarchically, whereas the problems faced are mostly horizontal in nature. He feels that it is important for us to accept that the way we act is actually part of the problem.

Professor Keith Grint15 writes how leaders have become addicted to command and are allergic to forms of leadership where a collective assumes responsibility for collective problems. Our interviewees seem to share this particular view. For example, Heinonen has many times faced situations where people express frustration at an ongoing discussion process and the connected lack of activity. In that case, somebody is likely to ask, what we are blathering about. Why do we not just do the job? This can be understood as a process of taming the wicked issue16. This seems to be a rather common practice. For example, in our research on Finnish national health reform1, the main conclusion was that even though the wickedness of the problems was acknowledged, the solutions implemented resembled solutions designed for tamer problems. Heinonen reminds us that even though solving wicked problems through truly collaborative processes is a slow process, it might actually be quicker than approaching the issue through traditional authoritative procedures. Professor Nancy Roberts17 has come to similar conclusions; usually we have to fail in pursuing authoritative or competitive strategies before finally turning to collaboration.

Hanén flags the dilemma between command and collaborative leadership. He feels that in a military organization working with complexity is sometimes difficult as everything one says is taken to be a command. It is then necessary to state explicitly that the aim is currently to mull over the issue and that suggestions that arise are just that and not orders. Hanén also emphasizes that leaders should be careful that they do not overcomplicate everything, that is, they should avoid the “complexity trap”. A leader must be able to separate routine management issues from more complex, or even wicked issues. Heinonen thus advocates meta-decision making, which Professor Zhongtuo Wang18 defines as, “the process of deciding how to make decisions.” Heinonen sees the danger going forward hastily, without perceiving what is the nature of the problem or in what kind of environment we are working. Do we perhaps tend to think too much about what to decide, but neglect to plan how to decide?

Finally, Heinonen tells us how the idea of heroic leadership19 is still strong in the realm of state governance. The leader is supposed to know the right answers. He calls this masculine leadership. Bishop Walker perceives the issue in a similar way, stating “There is a temptation for leaders to go with the projection of many organization members that the leader will come up with the right answer every time.” Walker tells us that he himself has been fortunate enough to be able to try out different methods in a rather flexible environment. Moreover, in his new position as bishop, his seniority has made things easier. Ultimately, the issue is largely about securing positive results. Walker puts it like this: “The more times I use chaotic and complex methods in leadership and get positive results from them the more confident I am using them in future situations.”

Conclusion: Implications for leadership education

Even though Walker, Hanén, and Heinonen all see a significant value in the paradigm of complexity, they are cautious about advocating its broader application in leadership education. Hanén warns us that we should not advance too fast. He is afraid that complexity would share the destiny of chaos theory in the 1990s in the hands of management consults. “We should not popularize complexity overly, as it would then consume itself. However, even though I am cautious, I am certain that something new will come to leadership through complexity sciences. What it will be, I am not yet sure.”

State Secretary Heinonen highlights similar challenges. Although he has received positive responses to his presentations—in various leadership forums— on the concept of the wicked problem, the challenge has become that the concept has been come to be used carelessly. He explains: “Awfully few people are aware that what wicked problems are really about. There is the risk of promoting misconceptions and as a result wicked problems will soon come to be used to describe all problems that are difficult… The concept could then easily lose its explanatory power and suffer from a kind of inflation.”

Capt. (N) Hanén sees it as important to strengthen the connections between the theoretical side of complexity sciences and the existing management culture. In particular, it would be important to collate the points of view of real leaders. Through theory-practice dialogue—the technique applied in the current article—a more in-depth understanding of complexity, chaos, and wickedness could be obtained. Bishop Walker shares this understanding. “There needs to be more work done among those who are naturally drawn to these ways of working before the ideas are robust enough to promote more widely in the field of leadership education,” he tells us. This is an open call to us all, academics, and real leaders.