From evidence-based management to management of non-knowledge

Introduction

Today, evidence is considered a critical management concept. Inspired by evidence-based medicine,1^,2 the concept of evidence-based management (or synonyms like evidence-based practice or evidence-based policy) has spread to a number of public areas, from social programs, municipal preventative programs, pedagogy and education to law enforcement. Also in direct relation to management and management development the concept of evidence-based management has found resonance3^,4^,5. Evidence-based management ideals have become institutionalized in worldwide evidence producing organizations, like for instance the Cochrane Collaboration within health care and Campbell Collaboration within education. To this a long line of national evidence-producing organizations can be added6^,7. In different ways, both central and decentralized government agencies have sought to institutionalize evidence-based initiatives by implementing project days, knowledge memorandums and other similar initiatives. Furthermore, educational institutions have begun to offer training in evidence-based work.

Evidence-based initiatives have developed into a central aspect in the struggle for professional legitimacy and resources. However, it has also become a critical aspect of the management of public organizations, because it creates access to forms of knowledge, which previously were a privilege belonging to the professions8. At the same time, however, the specific nature of evidence-based efforts is not self-evident. Evidence is not an unambiguous concept, and we can observe a number of ongoing struggles to define its content and boundaries9^,10.

People often associate evidence-based practices with its ability to increase efficiency in the public sector, because evidence helps establish “best practice” in a given field. Thus, the evidence-based trend is also fueled by the desire to know what is being done and the effects those actions have. The notions of evidence and knowledge generate a sense of security and stability. We might not know exactly what we are doing right now, but the concept of evidence is linked to the hope that we will at some point be able to master the effects of our actions.

In response to evidence-focused management, I want to promote the development of a particular form of non-knowledge management. Only the most ignorant or inexperienced among us cannot cite examples of organizational initiatives marked by ignorance or perhaps even stupidity. As an example, a colleague of mine recently found herself in a battle with the Office of the Registrar. The office had made a scheduling mistake, which caused it to decide to postpone an exam by three weeks - three days before the exam was scheduled to take place. The stupidity was not so much the fact that a mistake had been made, but the fact that the office did not understand that it would be a problem for the students that the exam they were preparing for had been moved. Only following several phone calls from weeping students and with the threat of the case finding its way into the local newspaper did the Registrar's Office find the additional exam supervisor required for the exam to take place as planned.

We often perceive this kind of stupidity as signs of moral failure: as if the reason for people making stupid choices has to do with flawed moral judgment. They should have thought it through. However, we might also see such choices as a sign of ignorance: the Registrar's Office does not know what it is like to be a student and to study for an exam (while working part time and attending classes at other institutions). We see a lot of this kind of ignorance—and I doubt that any amount of evidence-based examination planning will help the issue.

Clearly, it is better to know what one is doing rather than acting blindly. However, knowledge, often makes us realize how much we do not know. The relationship between knowledge and ignorance is not a zero sum game whereby our ignorance decreases in accordance with the amount of knowledge we acquire. In fact, the opposite might be true: the more we know, the more we realize that which we do not know. And the more others know, the more there is for me to not know. Thus, it is important to directly address this ignorance rather than pretend and hope that acquiring still more knowledge will eliminate ignorance.

The article is structured as follows: In the first two sections I present a set of fundamental reasons why I am convinced that producing more knowledge does not eliminate ignorance. Then I shall mention some prominent examples of analyses of ignorance. These examples raises the question what an intelligent approach to ignorance looks like. In order to answer that question I first present a systems-theoretical concept of non-knowledge. Secondly and inspired by Foucault's analyses of the emergence of liberalism I argue that a core ingredient in management of non-knowledge is self-limitation.

Weber, science and demons

Although the evidence-based trend accentuates knowledge and new forms of knowledge-based practices, the ambition to base practice on knowledge is not new. The evidence-based trend can be seen as the latest development of a long process, which in sociology has been defined as rationalization or disenchantment. Max Weber's lecture titles “Science as a Vocation” from 1917 contains one of the most often quoted passages describing the disenchantment of the world:

This quote expresses the spirit of today's focus on evidence: the belief that if one wants to, one can use knowledge to calculate and thus master the effects of one's efforts. Weber's lecture discusses the limits of rationalization. His central point is that science can support the realization of factual relations, but it cannot decide how something ought to be or whether or not something has value. In the same way, professors are not football masters in the vital problems of life, as Weber puts it. Weber does not directly reject the potential of rationalization and the disenchantment of the world. And yet his text is tricky, because it describes the practices of rationalization and scientific knowledge in a way that suggests that they themselves are not rational and, therefore, calculable. He writes:

According to Weber, the processes of intellectualization and rationalization through science rely on the knowledge or belief that “principally there are no mysterious incalculable forces” (139), but at the same time Weber's own text contains a number of mysterious forces in his description of science. Weber's text incorporates a series of concepts, which point to the non-scientific foundation of science. Moreover, these concepts generally refer to something unknown, incalculable—something whose origins are uncertain and beyond our control. “Fate,” “gift,” “inspiration,” “imagination,” “intoxication” are words that refer to something we cannot control or calculate. Rather than pointing to specific meaning, they all reference a form of conceptual embarrassment by hinting at something unknown. They suggest that what cannot be observed is constitutive for what can be observed. Knowledge depends on non-knowledge. Secrets and incalculable elements undergird science, Or in other words: science is not disenchanted and cannot be mastered by means of calculation. And if disenchantment does not even apply to scientific knowledge, it more than likely does not describe the rest of the world.

Why knowledge is never complete

Theoretical arguments as well as empirical studies support Weber's critique of the possibility of achieving complete mastery of the world through knowledge. Good evidence, as it turns out, can be defined in many different ways. The relationship between meaning and knowledge is not unambiguous, not even within the natural sciences. As an example, one medical study has discovered 121 different ways to evaluate the quality of individual studies. This clearly indicates the uncertainty pertaining to what is considered knowledge and what is not12:82. In other words, the answer to the question of what counts as knowledge and what does not is not self-evident. The concept of evidence-based hierarchies renders this fundamental uncertainty invisible by replacing the notion of uncertainty as a basic condition with a discussion of levels of uncertainty. Similarly, empirical studies point to disagreement about how to translate scientific results into guidelines designed to provide information about—or master—practice. One such example is a study of the relationship between guidelines and the evidence behind them, which shows that doctors—even within the same specialty field—perceive that relationship differently. Some doctors believed that the scientific literature supported the guidelines while others disagreed13:579.

Similarly, when it comes to the practical implementation of knowledge, Weber's demons lurk in the shadows. On one hand, the patient (the student, the client, the situation) or the case can be characterized on the basis of rather general categories (e.g. ADHD), but retain, on the other hand, unique and singular characteristics as well. We can always trace a level of tension between the unique case and the law, rule or evidence (as described by Derrida in his analysis of the relationship between the law and the just decision14:23. The two never fully coincide. Different students take something different away from the same lesson. This is true even in the field of medicine, which helps to explain the increased focus on individualized medicine, where medical studies seek to incorporate the fact that different patients respond differently to the same medical treatment.

While it is true that any unique case always contains elements that fail to match generalized knowledge or established standards, we can also note the opposite tendency for broader connections to break down when we seek to establish evidence based on specific effects. As Lukács notes, the relationship is between the rationality of the detail and the irrationality of the whole15:102. Rationalization requires the dissolution of any complexity into its composite parts. The separation of analysis/design and execution means that the analysis breaks down the subject into increasingly smaller sub-functions, which can then be optimized. However, this creates an inherent problem: the optimization of sub-functions makes it exceedingly difficult to reassemble that which has been separated. Let us once again use the Registrar's Office as an example. As an organizational unit, it represents the result of a desire to centralize certain administrative functions, which were previously relegated to individual departments. Such centralization has made it possible to standardize and rationalize the examination schedule down to the smallest detail. However, it has simultaneously led to what Lukács refers to as “the disregard of the concrete aspects of the subject matter”,15:101 that is, a form of ignorance with respect to what it means to prepare for an exam and, in a broader sense, what it means to be a student, including a multiplicity of complex expectations, activities, considerations, etc. associated with student life.

We might expand upon Weber's and Lukács' critique of the possibility of complete knowledge and rationality by incorporating Niklas Luhmann and his theory about functional differentiation16. According to Luhmann, modern societies are differentiated in accordance with a number of systems (politics, economy, science, education, religions, law, etc.), which observe the world differently and contain different measures of relevance. What counts as important knowledge to one system may be seen as pure noise to another. Similarly, the ‘same' scientific knowledge can mean fundamentally different things in different functional systems. If, for instance, we look at a hospital as a whole, it includes today care and treatment but also economy, production, service, quality, etc. Hospital administration systems contain a number of sub-systems, which are sought optimized individually but which do not make up a totality. The hospital looks different depending on whether one observes it from the perspective of medicine, organizational concerns, legal aspects or economic growth. Economic, quality-based, professional and other aspects do not merge into a united whole, but rather pull the organization in different directions. This is a reflection of the reality that it is not possible today to consider all aspects of a case or a field. This reality is what the term complexity refers to.

In today's society, social problems are so complex that one cannot approach them from a single vantage point. Or, to put it differently: ignorance is a basic condition. The acquisition of knowledge about certain general relations leads not only to ignorance of those concrete aspects of a singular situation that escape generalized knowledge but also broader connections across systems. For instance, psychiatry knows about the connection between a given medication and its specific effects. However, it does not know how a specific patient will respond to the medication in the same way that is also does not know how this knowledge will be observed from the perspective of the political system: perhaps it will be used as an argument to legitimize cutbacks of funding for psychiatric approaches based on talk therapy?

In organizational studies, the attention to the limits of reason is a similarly traditional concern. A common topic in decision theory is the fact that blindness (i.e. the limited inclusion of information) is the norm rather than the exception in decision-making processes—since the opposite would lead to endless meandering and exploration of possible outcomes, which makes reaching a decision almost impossible17^,18^,19^,20. Thus, Simon has introduced the concept of ‘bounded rationality' as suggestive of an actor's limited ability to make rational decisions because of insufficient time, information and information processing capacity21.

So far, I have outlined some of the reasons why ignorance is the obvious result of the limits of rational knowledge. I will now go on to comment upon some critical approaches to ignorance. Ultimately, ignorance is not simply the absence of knowledge; it relies on certain conditions, functions and effects—and these can be analyzed.

Agnotology

The critique of rationality has become a classic genre in philosophical and sociological terms, the most prominent example probably being The Dialectics of Enlightenment22 by Horkheimer and Adorno. The basic structure of their argument relies on the analysis of the fundamental irrationality of reason, which manifests itself in counter-intentional effects. Likewise, the comprehensive literature pertaining to risk and risk management is related to ignorance in the sense of uncertainty about future conditions23^,24. In sociology, ignorance has been articulated through discussions about secrecy, where some people know about something that others do not25^,26. The discussion of secrecy is related to what Proctor terms ‘agnotology' (a neologism formed from the Greek word ’gnosis,' which means knowledge)27. The term covers the study of culturally produced ignorance. Proctor has for instance conducted a study of the American tobacco industry, which shows how it has worked deliberately to obscure existing awareness of the links between smoking and cancer. Thus, the industry is engaging in the strategic production of ignorance. Or in McGoey's words: “strategic unknowns”28.

Other studies take on a more functionalist approach and study self-created ignorance29^,30. In these cases, ignorance is not studied as consciously created by others but as the logical result and unavoidable shadow side of knowledge itself and as the potential solution to certain problems. A classic article among these is “Some social functions of ignorance” by Moore & Tumin31. Their article demonstrates the potential social functions of ignorance, such as for example to protect certain privileged positions and thereby reduce conflict31:790. Similarly, a more recent study shows how deliberate ignorance plays a central role in the professional management of conflicting interests in the context of organ donation32.

Knudsen has studied the way that knowledge, which might potentially undermine or question the direction of a decision process, becomes segregated from the sphere of attention33. The study of the construction of a comprehensive national model designed to ensure quality of service in the Danish health care system showed that some of the most obvious topics were not included in the conversations (i.e. which problems is the model intended to solve? Are there potential alternative models? Do we know if such models are effective? What are the costs of implementation of the model to the health care system?) In the context of, on one side, decisions based on evidence, knowledge and information and, on the other, a decision process characterized by the absence of central and highly relevant issues and therefore the absence of knowledge/information, the question arises about how these issues have managed to become excluded from the process. Knudsen described a line of ‘forms of inattention' as an answer to this question.

Alvesson and Spicer argue that the concept of ignorance focuses exclusively on content, and thus suggests that the addition of further knowledge would be sufficient to solve the problem. However, the concept of ignorance tells us very little about the limits of the active use of cognitive and intellectual capabilities. To rectify this, Alvesson and Spicer introduce the concept of ‘functional stupidity'34^,35. They define functional stupidity as the “inability and/or unwillingness to use cognitive and reflective capacities in anything other than narrow and circumspect ways. It involves a lack of reflexivity, a disinclination to require or provide justification, and avoidance of substantive reasoning”34:1201. Based on this definition one might wonder if the institutionalization of evidence-based efforts in the form of guidelines and standardized programs run the risk of increasing organizational stupidity as a result of the lack of reflexivity. I agree with Alvesson and Spicer's critique of the insufficiency of the notion that ignorance can be cured with the introduction of more knowledge. However, rather than retracing the path of stupidity, I want to ask: what does an intelligent approach to ignorance look like? In response to the heavy focus on evidence and knowledge, it seems prudent to discuss not only how an organization might manage knowledge but also their ignorance or lack of knowledge. In preparation for a more thorough discussion of the question of an ‘intelligent approach to ignorance,' let me start by providing a more specific definition of knowledge and ignorance—or rather non-knowledge.

Non-knowledge

The term knowledge is given many different definitions. Contrary to a concept of knowledge defined by content and facts, where knowledge is perceived as something one can possess to a lesser or greater degree (knowledge as trivia), I want to focus on a more sociologically based concept of knowledge. With inspiration from Luhmann, I propose an understanding of knowledge as that which structures observation30^,36^,37. Knowledge functions as a structuring of selection processes that determines what we respond to and what we do not respond to. Our observation of phenomena in the world is characterized by the fact that we are unable to observe everything all at once. Instead we make selections and decide what to focus on. Systems theory proposes to understand knowledge as the structures that guide these selection processes. Knowledge selects particular things—and not others—as information. When a doctor observes a patient, she uses her highly specialized knowledge to pay attention to certain types of symptoms in the patient. Thus, medical knowledge serves as a structuring of the doctor's selective observation of the patient. The observation is structured by the doctor's knowledge while also serving as an observation that produces knowledge about the patient. Subsequently, this knowledge about the patient will typically structure future observations of the patient (by other doctors or nurses). Thus, the connection between knowledge as the structuring of observation and the observations themselves is circular. Knowledge structures observations but can also change as a result of observations.

The view of knowledge as the underlying structures that select our observations implies that knowledge always includes the production of non-knowledge. Social workers, policemen or psychologists make selections that are different from medical specialists. Their specific structures of knowledge will lead them to observe the patient differently and will therefore produce different knowledge about the patient. Knowledge implies the selection of specific types of observations and therefore also the de-selection of observations. It excludes potential observations by virtue of its method for structuring observations. We may refer to such excluded observations as non-knowledge30^,36:13. One characteristic of non-knowledge is that it is hidden. We are not aware that our observations are selective and that they produce non-knowledge. We observe, but we do not observe the selection processes that structure the observations, in the same way that we obviously are unable to observe what has been excluded.

Systems-theoretical thinkers have proposed a conceptualization of organizational intelligence in relation to knowledge and particularly in relation to the ongoing production of non-knowledge37:62^,36. The idea is to see intelligence as the ability to relate to the other side of knowledge, to non-knowledge. By this logic, an organization is intelligent to the extent that it is aware of its non-knowledge and pays attention to it in its internal operations. This is a paradoxical figure, which might nevertheless make sense. To possess knowledge of one's non-knowledge means forming representations of things one does not know. Intelligence is defined by one's ability to represent what one does not know.

Representations of non-knowledge can take many different forms. Seidl provides three examples of organizational intelligence: inter-organizational networks, heterarchies, and organizational interaction36. We can illustrate the representation of know-knowledge in relation to inter-organizational networks: Social workers working with youth and families in a municipal setting can achieve representation of their non-knowledge about psychological phenomena via their network with educational psychologists. The social workers do not know what the psychologists know, but by communication about their relationship with the psychologists, they obtain representation of their non-knowledge. Or if we return to our previous example: the Registrar's Office would more than likely increase its organizational intelligence by entering into a network with student-driven organizations, which would provide it with an occasion for communicating about its relationship with students.

Management of non-knowledge

The systems-theoretical concept of representing non-knowledge on the side of knowledge creates a problem with what has been called unknown unknowns—that is, the things we do not know that we do not know38. I know that there are certain things I do not know. But I do not know what it is I do not know, which means that I cannot represent my non-knowledge. The question is how to handle such unspecific non-knowledge, that is, the non-knowledge that we are unable to represent?

One possible answer can be found in Foucault's descriptions of the emergence of liberalism39. A central aspect of liberalism as an economic and political ideology is the limitation of state intervention into society. The political economist and philosopher F.A Hayek, for example, is a prominent proponent of society's ability to handle greater amounts of knowledge through decentralized rather than centralized planning40. This means that the state needs to limit its control and planning ambitions. Foucault's analyses focus on the emergence of this idea about the benefits of the state's self-limitation.

Foucault shows how the idea of a state's self-imposed limitations emerged as a response to political trends in the 1500-1700s aimed at creating detailed control in specific social areas. Cameralism and mercantilism were names for state sponsored efforts to bolster the state's wealth by means of regulation of foreign trade in particular. Prior to and along with financial regulation, the 1500-1700s witnessed comprehensive societal regulation throughout Europe. There were regional differences and varying synchronicities, but in German-speaking areas, the efforts were related to an ideal about “the well-ordered police state”41^,42. This meaning of the word police is different from contemporary usage and indicated a much broader effort to create social order. Oestreich uses the term “regulation mania”42:157 to sum up the wide-ranging desire to regulate just about any area of society in both the public and private spheres—from local issues in cities and states to private concerns such as health, hygiene, street cleaning, fire prevention, buildings and traffic. To this list was added the regulation of infidelity, the amount of money spent on christenings, funerals, wedding, clothing, etc. The scope of the public's interest widened considerably during the 1500-1700s.

Up against such regulative ambitions, Foucault analyzes the birth of the notion that the government must relinquish regulation in order to achieve its goals. It must suspend or limit itself in order to allow for different forms of rationality and different ways of organizing productions. A line is drawn between things that need to be done and things that should not be done: “The whole question of critical governmental reason will turn on how not to govern too much”39:13. Instead of regulating and managing foreign trade, for example, the thinking that emerges in the 1800s suggests that states may increase total trade and thus production by relinquishing regulation—through free trade.

The notion of governmental self-regulation is linked to the spread of the idea that the subjects of governmental control possess their own natural laws and ways of functioning. Foucault describes the logic in this way:

Thus, at the beginning of the 1800s the problem, according to Foucault, was perceived as governments' lack of knowledge about the issues they were dealing with. And one answer was for governments to self-limit in order not to act in a way that impeded those issues. Precisely because government cannot know everything about the social field that it is interested in, it has to retract and allow the social field to construct its own structure in accordance with ‘the invisible hand.' In the 1800s, dominance and ignorance are increasingly perceived to be a bad combination.

This line of thinking has since gained currency in the context of liberalist economic models. As mentioned above, F.A. Hayek has provided the models with an information-theoretical foundation: as a decentralized entity, the market can handle much greater amounts of knowledge than the state as a centralized planning organ. Or in other words: the state needs to limit itself because, in comparison to the market, it is stupid and ignorant.

The root of this kind of self-limitation was, according to Foucault, a new form of governmental self-description in the 1800s, which was referred to as ‘political economy.' Political economy was both a) a mode of government intended to promote national progress and wealth and b) a general reflection of the organization, distribution and limits of society's power structures39:13. It represented, in other words, the production of a new form of knowledge, which made it possible to address the non-knowledge of the state.

My aim is not to promote a new round of neoliberalism. But I do want to suggest that we consider the possibility of generalizing the liberalist idea of governmental self-limitation, whose emergence Foucault diagnoses. Thus, if not only the state, but also individual organizations relate to fields (education, medical treatment, care work, social work…) about which their knowledge is very limited, how do they avoid acting in ways which are in conflict to the inner logics of these fields?

I believe that the question about the relationship between regulation and non-knowledge, which according to Foucault was central in the political economy in the 1800s, contains a more general dimension. Today, regulation is not simply a question about the relationship between state and market. It is also, for instance, about the relationship between organizations and different professions. As an organization, a hospital relies entirely on doctors, nurses and therapists. But at the same time, the knowledge that guides the decisions made by health professionals is not created by the organization (as an organization, a hospital has very limited knowledge of things like cancer and cancer treatment), but by the health professionals themselves (and to some degree by research).

There is a clear parallel between the law enforcement ambitions of the 1700s with respect to achieving detailed regulation of social issues and the tendency within the evidence movement to strive for increasingly more detailed guidelines and generic programs. This development is possibly paradoxical in that it, on one hand, relies on increased knowledge about individual efforts and their effects (in the form of evidence-based programs), but, on the other hand, encounters the growing problem of non-knowledge. Such non-knowledge is similar to the non-knowledge on which liberalists have based their critique of the state. It breathes new life into the question of the power of government and the potential self-limitation of regulatory powers.

Since organizations rely on others (professionals) to produce something, how can they limit themselves in a way so they do not impede the efforts of professionals? Professionals and organizations base themselves on different knowledge structures and hence also different non-knowledge. Here, the liberalist stance would be to require organizations to limit themselves in order that their knowledge is not also transformed into non-knowledge for the professions. The attempt to use organizational tools to fix the problems of public schools, for example, might in effect create a more ignorant school. The organization has to recognize that its knowledge about teaching and education is rather limited. One can create lesson plans and teaching goals on the level of the organization, but simply telling students about pedagogical goals is not the same as them learning something. The organization can ensure that teachers receive pay, that students are placed in the right grades, etc. But these decisions do not mean that students will learn. The more the organization seeks to organize educational programs, the greater the risk that its program will not be successful—because it obstructs the professional knowledge of teachers. In other words, the organization has to concern itself with its non-knowledge by limiting itself, that is, refraining from initiating potential operations.

This article wants to invite discussion of the possibility that self-limitation might also today provide an answer to the question of non-knowledge. There are no final and general answers to the question. As proven by countless critics of (neo)liberalism, it is possible to regulate and govern too little. Governing tends to oscillate between either too much or too little. Rather than aiming for the golden level of governing, the answer might be to address the question of non-knowledge as directly as possible. If organizations learn to ask themselves the question of how to limit themselves in order to achieve their goals—then they will be on their way to a more intelligent approach to managing their own non-knowledge.