Self-Organizing Leadership:
A Way of Seeing What Is Happening in Organizations and a Pathway to Coherence

Richard N. Knowles Ph.D.
Center for Self-Organizing Leadership, USA

Introduction

This article introduces a new form of leadership called self-organizing leadership, which complements and supports strategic and operational leadership. It looks at organizations from the complex adaptive systems (CAS) perspective and offers a novel way of seeing and understanding some of the self-organizing patterns and processes operating in organizations. I developed a number of new, transformative process models over a 12-13 year period that help us to see what is happening in our organizations, and to make the changes we need to improve our effectiveness and competitive performance.

Many writers who use CAS theory to try to understand what organizations are doing work from the perspective of standing outside the organization. They are acting on the organization as if it is a thing to study, manipulate, and control. The work described in this article was undertaken from the perspective of standing within the organization with people who are deeply engaged in conversations, together, clarifying who we are, and developing agreements about how we will behave and work together, as well as addressing our important issues and doing something about them. Stacey (2000, 2001) has written about the importance of transformative processes and complex responsive processes. My work appears to be highly aligned with Stacey's work, and presents a way to bring these ideas into reality in organizations.

In some organizations the people are working coherently, energetically, and effectively. Information flows freely and the parts are well connected. These organizations are self-organizing and have many of the characteristics of living systems.

In other organizations, the people struggle with mixed messages and incoherence. The information flow is blocked and the parts are disconnected. Confusion, cynicism, and burnout hamper the organization's effectiveness. These organizations are treated by their leaders as if they are machines to control and manipulate; these are command-and-control organizations.

In my experience, most people in organizations are hard working, intelligent, and well-intentioned. Very few people get up in the morning with the express intent of messing up their organization. So why is it that in some organizations people blossom, while in others they wither?

One key lies in the unobserved, unexamined patterns and processes that are operating below the surface, beneath the operational and strategic leadership processes. This article presents some transformative process models that are helpful in revealing and deepening our understanding of what is happening below chaotic surface events.

GROUNDING IN OUR SHARED EXPERIENCES

The approach used in this article is grounded in shared experiences of trying to understand and help make organizations as successful as they can be. To begin this process, the organization is depicted on a sheet of paper as a circle, which implies the whole. The circle also brings in the idea of feedback loops characteristic of organizations. Nine different but interconnected perspectives are used to look at the organization and to see what is happening (see Figure 1). The attributes of these perspectives are easily seen and felt when we step into an organization like a store, a family, a business, a church, or a school. We all use some of these perspectives almost automatically as we step in the front door.

Figure 1 The whole from nine perspectives

These perspectives, in the form of questions, are:

This may seem like a complicated way of looking at organizations, but it is actually a process that we all use, at least in part, quite often. To illustrate, let's pretend to go into two large stores near my home in Niagara Falls, NY. We'll look at each store from each of the nine perspectives, making brief statements about what we see. As we do this, it is hoped that you will begin to relate to the model.

One store is a large mass merchandiser. We will keep it anonymous, but from its name we know who it is and that its intention is to sell modestly priced clothing and home furnishings. One issue that we immediately see is that the store has narrow aisles full of boxes and clutter. The housekeeping is poor. The behavior that the employees show toward each other, and us, is neither courteous nor considerate, so we conclude that their relationships are tense and that they don't have, as a principle, much value for each other or the customers. Their physical work is to sell lots of stuff. Whenever we ask for help about where to find something, we get vague answers. We conclude that the clerks don't know much about what is going on and that information isn't being effectively shared. We also conclude that there isn't much learning going on about the business and ways to improve it, since this experience is the same as we have had in this store before. Finally, we also conclude that they must be organized in a rigid, command-and-control hierarchy, because that is the only way that the store can keep things organized. The clerks seem to be waiting around for directions from the boss.

When we go into the other store, Home Depot, the picture is quite different. We know from its name who it is and that it is trying to sell stuff to help us to keep up our homes. The store has broader aisles and things are well ordered; the housekeeping is quite good. Staff's behavior toward each other, and us, is helpful and courteous. They are interested in us, the customers. Their physical work is to sell stuff, like in the first store. Information seems to be freely flowing because the clerks know the answers to our questions. When we take something back, they talk with us, trying to learn how to serve us better and have a more successful business. We conclude that to do the work they are doing, and to behave in the way we see them behaving, they must be organized in teams and their structure must be fairly flat.

The patterns we have seen in these two stores show up over and over again as we move among the stores in each of these companies. For example, when I told the story of the first store in Australia, the people in the group immediately recognized the behavior and correctly identified the company. I was just talking about the store in Niagara Falls, NY and yet here were similar patterns showing up half way around the world. The patterns are quite fractal.

If we had owned stock in these two companies over the last five years, we would have lost money on the first company and made money on Home Depot. I expect that there is a relationship between their respective behaviors and their performance in the stock market.

These are simple, quick analyses that we all can, and do, run whenever we see and experience an organization. The nine perspectives are arranged around the circle in Figure 1 and the internal lines show the interconnectedness of the parts. A change in any one part results in changes in all the other parts. The web of connections is like a nervous system with each perspective informing all the others. As in a nervous system, some connections are stronger than others; the connections need to function in a specific order.

COMMAND AND CONTROL

The first store that we visited is a typical command-and-control organization. In organizations like this, management desires reliability, stability, predictability, and control. In a recent survey by the author of 29 CEOs in various TEC groups, 28 desired these features. While these are the attributes that managers want, they are often disappointed. All the CEOs felt that their employees were not contributing their best; 28 report high stress-related problems in their organizations; 14 are confronted with high turnover; 22 face serious resistance to change; and 28 of them are unsatisfied with overall business performance. Yet all of them report that they are working very hard to improve results. In their personal lives, 16 comment that they are not happy with the balance between work and family. Reliability, stability, predictability, and control are great for machines, but people and organizations are not machines.

Often in command-and-control organizations, those at the top are vague and unclear about their organization's identity. Even when they do have some sense of the organization's identity, they rarely engage the rest of the organization in developing a deeper understanding of who they are. When those at the top decide that a change is needed, it is unusual for them to engage the rest of organization in examining the deeper reasons for, and the thinking behind, the necessary change. When they announce a new initiative, no matter how good and well thought out, it often comes as a surprise and isn't well understood, so lots of issues, problems, and objections get raised; those at the top view all of this as resistance to change.

This first step can be illustrated in Step 1 in Figure 2. Since management feels the need to move on it often overcomes resistance by reorganizing, which is illustrated as Step 2. Management dictates a solution and structure (e.g., a new organizational structure as in reengineering). The answer is imposed as illustrated in Step 3.

Figure 2 The command-and-control pattern and process

Those doing the actual, physical work, and trying to make sense of what is going on, get stuck in the unresolved issues. They raise all sorts of questions about unresolved issues while top management pushes to get the work out the new way. As the top people push harder, the people doing the work push back harder. The organization often gets stuck between the need to get the work out and the need to resolve issues. This is where the human resources people, and the unions, make their living. If the conflict becomes severe enough, management will reorganize again, and maybe again. Around and around this triangular pattern of issues-structure-work we go. Communication breaks down, people become isolated, creativity is driven into negative paths. Enormous energy is wasted; our organization is incoherent and floundering. From inside, the organization is unable to learn and to improve, so its competitive position erodes. The organization is unable to tell the truth.

This is the management trap, as Argyris (2000) calls it. This organization is a Model I organization filled with the problems and incoherence that Argyris describes. Some characteristics of Model I organizations are:

BEHAVING LIKE A LIVING SYSTEM

When we see the organization in a new way and engage each other in the self-organizing leadership patterns and processes, we break out of the management trap. Home Depot, the second store we visited, seems to be an organization that is breaking out of the trap. Here is a story about some work we did in 1988 that illustrates the use of self-organizing leadership; we also broke out of the trap.

SAVING THE AMINES BUSINESS

In the hills and valleys of West Virginia, a plant was built in 1927 to make ammonia. The chemical industry came to West Virginia because of the abundant supply of good water, coal, and salt. Many chemical process operations were first developed in the Kanawha Valley near Charleston in the first half of the twentieth century. But over the years the industry lost the support of much of the community because of economic and environmental problems. After the Second World War, many of the new plants for products like those made in the Valley were built in the Gulf Coast in Texas and Louisiana, where gas was cheap and the climate better. Employment in the Valley's chemical industry has dropped by over 50 percent since the war.

One of the product groups still made at the DuPont plant in Belle, near Charleston, was called methylamines (mono-, di-, and tri-methylamines). All of these methylamines are stored under pressure; two are gases at atmospheric pressure. All of them smell like dead fish at very low concentrations in the air. The plant was over 30 years old and used nitrogen and air pressure to measure temperatures and process pressure changes. At the time these pneumatic control processes were the best around, but by 1987 they were obsolete. There were problems in keeping the facilities running when they were needed, with product quality, and with emissions to the environment. With problems like these and being the second largest producer in this business, it was difficult to earn enough money to justify staying in the business. Even though it was a sizable business with annual sales of around $50 million, the business team decided that we had either to restore the business or get out of it; staying where we were was unacceptable. Getting out would result in the loss of over 100 jobs.

As plant manager, I worked with the business team and the corporate process control research team and we decided to upgrade the product and plant to become more competitive. One major change we decided on was to change from the pneumatic control system to an electronic system called a DCS (Distributive Control System). Normally when an old facility faces a process control retrofit change like this, a parallel control system is built and brought into operation before the old control system is dismantled. This is a conservative way to do it, but takes longer and is more expensive. The engineering department estimated that it would take two years and cost around $6 million; we decided that this was both too long and expensive. After much soul searching, we decided to do the job ourselves at the plant, partnering with the R&D people and our instrument vendor. We also decided not to build the new process control system in parallel with the old pneumatic process. While this would save both time and money, failure would mean the loss of the business. The project planning started in January with the goal of installing the new process control system during our maintenance shutdown scheduled for the following November.

We knew that everyone had to be involved in this if we were to have a chance of being successful. We held project status meetings every week in the control room where the operators, mechanics, and engineers all talked about what was happening; everyone had a chance to raise questions and to make their input. At first some felt that the control room was a poor place for the meeting because it was crowded, with no room to sit down, and was actually operating during the meetings. It turned out to be the very best place. Each month everyone was able to get up to speed as the shift rotations brought operations and maintenance people through the day shift. I stood up along with most of the others during the 90minute weekly meetings.

The first meeting was in January 1988 and by June we had two capital projects, totaling around $3 million rather than the originally forecast $6 million, ready for authorization by the vice-president in Wilmington, Delaware. We signed off on the plant level of authorizations for these projects during the course of one of the meetings in about 10 minutes; the normal routing procedure would have taken six to eight weeks. The business team representative was there and she took the projects to the vice-president the next day.

We continued to meet every week on through the summer and fall. The intensity and excitement built as we got closer to the November shutdown. We sent eight operators and mechanics to the Honeywell Instrument School in Phoenix for three weeks of training. When they got home, they helped to train all the others. Operators, mechanics, and engineers from several organizations within DuPont as well as engineers from Honeywell were working closely in teams around all of the key work. Since people had all the information and knew what had to be done, they spontaneously formed teams around the work and did it.

The operators helped to design the graphics for the process control monitor screens. They worked with the engineers deciding on the information they needed so that they would be able to run the process safely and easily. The mechanics worked with the engineers and operators on the choice and location of the instrument sensors in the process. The mechanics installed the instruments, pulled the wire, and did the loop checks to be sure that everything was connected up correctly. The planning and scheduling people worked closely with the teams to build the shutdown schedule and make sure that we hadn't missed anything.

When our two-week November shutdown came, people worked day and night getting everything done. The close coordination and cooperation were a beauty to watch. In addition to the DCS project, we also did the critical maintenance of all the major pieces of safety and process equipment, including, for example, taking the tops off the 100-foot tall distillation columns to do the internal inspections. About 100 or so pneumatic process control loops, which had to be changed when the whole process was shut down, were taken out and the new DCS system controls were installed. After two weeks of beehive-like activity, we started up the process. Within five days we were running at capacity, achieving the quality and environmental improvement goals.

In my past experience with changes like this where a parallel system was installed by the engineers and then handed over to the operators, it often took a year or more to get the process running well. We did not encounter any of the resistance to change problems that normally plague projects like this one.

The deep involvement of everyone, the sharing of all the information, the trusting relationships we had built in all the project meetings, and the clarity that the business would die unless we were successful, helped everyone to bring all this to a successful conclusion. Everyone was pleased and proud at having set a new DuPont standard for retrofitting DCS systems into old plants; we had cut the time and cost in half. Since then, the plant site has done 15 more process control conversions on other product units, continuing to improve its performance in converting to the new process each time. Many of the operators became active in corporation-wide teams to help other plants to do this.

My leadership role really shifted. I participated in all the project status meetings each week in the control room. I was very conscious that I needed to insist that everyone really listened to one another, sharing ideas, insights, and questions. Together we made many decisions, right there in the meetings. We made adjustments and corrections to the process and plan as we went, on the fly. In addition to being sure that we lived up to the high standards of keeping the information flow open, listening, building relationships, and keeping people connected, I was a cheerleader. At night I prayed a lot too!

This is an example of self-organizing leadership. We all shared information week after week in the control room meetings. We talked a lot about what we needed to do with the business and our important role in its successful transformation. We sent people to school to learn about the computer processes. We learned to trust each other and became more interdependent. Self-organizing occurred extensively as one person after another stepped up to take one of the multitude of tasks facing us. The organization became “leaderful” (a phrase learned from Karen Anne Zien of Boston).

The three domains of self-organization are identity, relationship, and information. These are illustrated by the bold triangle in Figure 3. In the weekly amines project status meetings we saw these ideas play out. Our shared identity emerged over the 10 months of the project; we had clarity and focus on our task. We listened, talked, debated, thought out loud, and struggled together to find the best decisions. Information was openly shared. Our relationships became more authentic and interdependent. We knew that we were all in this together and the business depended on our being successful. People learned to go directly to those who had critical information for their work, so the information pathways became shorter, the quality of the information improved, and it took less time to get what they needed.

Figure 3 The domains of self-organization

It is clear from this story that identity, relationship, and information change and interact all the time among people in the organization. This is quite dynamic and enables the organization to function far from equilibrium without coming apart. Self-organizing leadership helps to create the conditions where all this comes together successfully.

Based on my observation of many organizations, self-organizing patterns and processes are going on all the time. In command-and-control organizations the self-organizing processes tend to run underground, often conflicting with the goals of the organization in some way; these processes tend to show up as resistance to change.

In organizations behaving more like living systems, the self-organizing patterns and processes are visible and encouraged. Out of these self-organizing patterns and processes meaning, creativity, and energy flow. These organizations more quickly and effectively meet, and often exceed, their goals and objectives. Resistance to change is minimal.

THE WORK

The processes of self-organization must be connected to the process of the organization's physical work. The pattern for the way work gets done is embedded in the web of Figure 1. It is brought out and illustrated by the bold lines in Figure 4.

Everyone in the system is engaged in conversations about identity, intentions, and the business context, so we have a clear sense of ourselves and what we are in relationship with (this is called self-reference). Managers have a lot of input, but not the only input; they are part of the system and have a voice in it. Together we develop and agree on the principles and standards of behavior that we need to embrace if we are to fulfill our intention. Everyone is accountable to each other to live up to these. As we engage on the issues, our principles and standards of behavior enable us to stay in the heat to find the reconciliation we need to move ahead. The issues tend to be either/or questions, such as if we only had enough money we could do this, and so on. The principles and standards introduce a third force that enables us to break out of the either/or trap.

Figure 4 The living pattern and process of how the work gets done

When we have all the information, we spontaneously self-organize; teams form (creating structure) to do the physical work of the organization. As we do the work, we immediately find better ways to do it. We learn as we go. In this process energy and creativity are released as people find meaning in their work.

All work seems to follow this pattern. Consider this simple example. If our intention is to connect two boards using a nail, we have to use principles and standards related to the proper and safe ways to use a hammer to drive the nail. Deciding where to put the nail and the size and type of nail are issues we must take into account. Preparing the workplace so that we can drive the nail safely creates the structure. Driving the nail is the work. On completion of the work, we often see that there was a better way to have done it, so there is the opportunity for learning that we build on the next time we want to drive a nail.

The nine attributes have to be considered in the context of an important, relevant, and urgent question for the organization. The question must be compelling enough for the people in the organization to want to rise above their differences, and be willing to create and live by the new principles and standards. For the amines business example, the question was how to make an effective, low-cost conversion from pneumatic to distributive control systems so that we could have a strong and thriving business.

The questions we choose can vary enormously in scope. They can be very narrow—like “How do I type this paper?”—or broader—like “How do we improve the customer service in our business?”—or very broad—like “How do we change the entire nature of our business?” The scope and breadth of the opening question set the scope for the questions that we address for each attribute. Thus the transformative patterns and processes are fractal in nature and can be used at any level of scale.

I have used these patterns and processes of self-organizing leadership in my personal development; for mentoring others; for developing the Belle plant leadership team; for operating the whole plant; and for working in the community.

These patterns and processes are now used with the leadership team for the City of Niagara Falls, NY. They apply to all organizations, regardless of the nature of their physical work. I and my associates have used them from South Africa to Australia to North America to the UK in businesses ranging from mining to steel and sugar production; in brick, wall board, concrete beam, and pipe plants; in places for the homeless and for disturbed teenagers; in tough union negotiations; in a prison; with school boards; and with a city government. The critical issue relates to the way that people come together to do their work, not the physical work itself.

Each time a group uses these self-organizing leadership patterns and processes, its members create a map of their work that is unique and relevant to them; it is a new creation. The map is on one sheet of paper and is built around the process enneagram that will be discussed later. The map shows both the whole and the parts with which the group is engaged. As the group uses these patterns and processes, the map is posted on the wall so that everything stays open and visible. Group members are in constant conversation about what they are doing, and as they grow the maps evolve. The self-organizing leadership patterns and processes stay alive.

As stated earlier, these living systems patterns and processes are running all the time in all organizations, even beneath the command-andcontrol patterns that we may try to impose (Figure 2). However, they are often invisible to those at the top. This is what some people call the shadow organization. It is where the grapevines are running. It is in the little groups that form deep within the organization where people are trying to do a good day's work and make sense of what is going on.

The more coercive the command-and-control processes are, the more abnormal the living systems processes running deeply in the organization become. In extreme cases the whole system becomes quite sick and dysfunctional. The more those at the top push, the more dysfunctional the organization becomes.

PEELING OFF THE COMMAND-AND-CONTROL PROCESS

While we can consciously work to develop self-organizing leadership patterns and processes in organizations, many of us have also experienced situations where we move into these processes spontaneously and abruptly.

For example, in an emergency there is so much that has to be done so quickly that there is no way management can exercise normal control over the situation; the command-and-control process is peeled right off. This is illustrated in Figure 5. Visualize the processes running as if they are like a layer cake with the command-and-control process of Figure 2 sitting on top of the cake. In an emergency, the organization almost immediately begins to behave like a living organization. Everything changes!

Figure 5 Simultaneous processes

In an emergency everyone knows who we are (our identity is that we are flat on our backs) and what we need to do (our intention is to get up and running again). The need to get up and running transcends all our differences. With little if any discussion, people start to help each other, work together, do the things that are needed, pulling together in ways never imagined possible. Most people don't need someone to tell them what to do; the need is clearly there. The people plow into all the issues facing them. Since all of the information about their condition is visible, they spontaneously self-organize, do the work, learn how to do it better, and keep going until they overcome the emergency; energy and creativity flow abundantly. People are often quite proud and feel good about the way they worked together in the crisis. Then, as the crisis comes under control, management steps in to reassert the old command-and-control patterns and processes.

The experience of an emergency teaches us that these processes of self-organization are natural; many of us already know how to do them.

CONCLUSIONS

With self-organizing leadership patterns and processes superior results are achieved and people feel better at the same time; we can have it all. The system becomes coherent. Self-organizing leadership lies under and supports operational and strategic leadership. Effective leaders need to move consciously and smoothly among these various roles as situations and demands change; I call this “the leadership dance.”

The combined patterns for self-organizing and work processes for the living system are shown in Figure 6. The pattern of identity, relationships, and information (the pattern for self-organization) is combined with the pattern for doing the work. Using this map, called the process enneagram, we keep these self-organizing leadership patterns and processes before us as we co-create our future together. The people in the organization create the map as they discuss their insights about each of the nine attributes. The leader's role is to keep the organization consciously aware of the map, posting it in meeting rooms, using it in day-to-day operations, and constantly testing progress toward this shared future.

Figure 6 The process enneagram

It is in this process of engagement with the people in the organization using these patterns and processes that we break out of Argyris's (2000) Model 1 and move into Model II. Characteristics that Argyris sees in Model II organizations are:

The experience that I have had in using the self-organizing leadership patterns and processes with a wide variety of organizations shows that these are a highly effective way to lead to the sort of characteristics that Argyris would call Model II. These are a way out of the management trap.

There are many other important patterns and processes of self-organizing leadership, many are within the web (Figure 1). This article merely begins to open up the possibilities before us.

References

Argyris, C. (2000) Flawed Advice and the Management Trap, New York: Oxford University Press.

Stacey, R. (2000) Complexity and Management, London: Routledge.

Stacey, R. (2001) Complex Responsive Processes in Organizations, London, Routledge.