Olivier Pilmis

Escaping the Reality Test. How Macroeconomic Forecasters Deal With 'Errors' 1

The success of this saying makes it difficult to trace its precise origins.

4 Although tackling issues of legitimacy and critique, the expression 'reality test' does not, here, strictly match 'pragmatic' conceptions (Boltanski and Thévenot 1991, 40-42; Boltanski 2009, 103-107). 5 In order to underline that a statement will be here considered 'erroneous' with respect to how actors define it, the nouns 'error' and 'mistake', as well as the adjectives 'right', 'wrong' and so on will be kept in quotes throughout the chapter.

Introduction

'Why did no one see it coming?' H.M. Queen Elizabeth II reportedly asked during a visit to the London School of Economics in November 2008. 2 The question echoes the criticisms economic forecasters often face for not having 'anticipated' the emergence of a particular economic crisis. The 2008 subprime mortgage crisis and its aftermath were no exception and gave birth to an important literature that takes the inaccuracy of forecasts as its starting point. Springing from sociology [START_REF] Fligstein | Why the Federal Reserve Failed to See the Financial Crisis of 2008: The Role of 'Macroeconomics' as a Sense-Making and Cultural Frame[END_REF], economics [START_REF] Galbraith | The End of Normal. The Great Crisis and the Future of Growth[END_REF], popular science [START_REF] Orrell | Economyths. Ten Ways Economics Get It Wrong[END_REF] or the media [START_REF] Turin | Le bal des aveugles [The Ball of the Blind[END_REF], critical assessments of macroeconomic forecasters' records rejuvenate the well-known saying: 'economists are experts who know tomorrow why the things they predicted yesterday didn't happen today'. 3 N. N. Taleb's best-selling book The Black Swan goes further and calls 'predictions' a 'scandal' (Taleb 2007, 137-164).

Whatever their sources, criticisms of forecasting usually present a mirror image of praise for whistleblowing: economists and forecasters are blamed for being overly optimistic and consequently failing to warn of future crises. These criticisms endorse a systematic comparison between 'what actually happened' and 'what had been predicted' as the only 'reality test' 4 possible to warrant forecasts' accuracy and eventually pronounce them 'right' or 'wrong'. 5 Repeated experiences seem to have brought clear-cut evidence that predicting future economic reality is impossible. The mere existence of an activity such as forecasting therefore turns out to be puzzling: how do forecasters manage to fend off criticisms and persuade themselves and others of their own credibility and that of their activities?

This attention to the relationship between beliefs and practices on one hand, and 'reality' on the other is reminiscent of early anthropologists' discussion of magic, religion, and science, all of which have been defined as belief-based practices. Building bridges between anthropology, economics, and sociology, Durkheim's enduring conception of belief offers an understanding of the persistence of forecasting despite recurring 'errors'. The Durkheimian ground of economic conventions and fictions has long been emphasized in both heterodox economics and sociology: while Orléan (2011, Chapter 5) has shown that following Keynes's track eventually leads back to Durkheim, Beckert (2016, 192-204) demonstrates how Durkheim's notion of totem and mana [START_REF] Durkheim | The Elementary Forms of the Religious Life[END_REF]) sheds light on the kind of fictionality that consumption encapsulates. However, the issue of whether beliefs are still held in adverse situations encourages us to turn to Hubert's and Mauss's seminal theory of magic. As disciples of Durkheim, they claim that the authority of magic and magicians originates in collective forces, so much so that beliefs and confidence in magic are a priori rather than evidence-based. Prior confidence in magic arises from the collective need to find explanations to phenomena that would otherwise remain inexplicable. Thus, magical beliefs stand firm when challenged. Moreover, the 'will to believe' is the origin of magicians' belief in magic: at the very least, they keep faith in their own ability to perform magic because they believe in the magic of others [START_REF] Hubert | A General Theory of Magic[END_REF].

In the case of macroeconomic forecasting, a similar somewhat functionalist argument posits that forecasting has more to do with coordination than with prediction. When uncertainty prevails, actors' decisions are necessarily anchored in 'fictions', requiring actors a priori to 'suspend disbelief' and adopt an 'as if' convention. When the future has yet to be created and cannot be known at present (Shackle 1972(Shackle [1992]]; and the introduction to this volume), economic actors can base their action only on 'fictional expectations'-that is, 'pretended representations of a future state of affairs' (Beckert 2013, 226) drawing upon actors' imagination. In this perspective, fictional expectations may be supported by 'instruments of imagination', such as forecasts or economic theories, but a 'right' forecast is one that is shared within a large community, thus enabling economic action. The ex post judgment of the accuracy of ex ante forecasts does not matter, as forecasts are judged according to their practical credibility at the time made [START_REF] Beckert | Imagined Futures. Fictional Expectations and Capitalist Dynamics[END_REF]. The notoriously poor track record of macroeconomic forecasting does not put its legitimacy at risk, because actors need forecasts anyway to design strategies. A rather provocative Hughes-inspired insight might even suggest that forecasters are to a certain extent hired by economic actors to 'make their mistakes for them' (Hughes 1951, 321).

But a series of 'failures' may lead to the persistence, or return, of disbelief. 'Fiction in economic contexts is vulnerable to contradictory experiences in the real world' (Beckert 2013, 225). Such tension within the forms of beliefs in which future-oriented activities engage is shared by both macroeconomic forecasting and magic. The belief may vanish if the expected benefits from magical acts do not come 'true'. Magic, like religion, is viewed as a totality; either you believe in it all, or you do not. … Conclusions are immediately generalized, and a belief in a single case of magic implies the belief in all possible cases. Conversely one negative instance topples the whole edifice; magic itself then comes under suspicion. We have examples of obstinate credulity and deeply rooted faith crumbling before a single experience. (Hubert and Mauss 1902, 113) Even though Hubert and Mauss pay little attention to it, they implicitly suggest a distinction between faith in 'general' and 'singular' beliefs-that is, between magic as a whole and local magical acts, or between forecasting per se and some forecasts or forecasters.

Their work also stresses that, in some circumstances, a 'will to believe' does not suffice and that 'make-believe' practices are needed. Magical acts thus encompass 'simulation' a notion, the authors remark, which should not be confused with that of 'fraud', especially as it applies not only to the public or clients of magicians, but also to magicians themselves. Magicians' practices and discourses are consequently aimed at convincing both others and themselves. In this chapter, a similar inside approach is proposed for the case of forecasting 'errors'. Indeed, while forecasters freely acknowledge that 'errors' and 'mistakes' are their bread-and-butter, they also devote significant time to replying to critiques, sometimes even highlighting them in public6 or making fun of them.

After we met, the interviewee walks me back through the building. We come across another economist who, I am told, joined the Institute in the 1980s. I am encouraged to benefit from his long experience: 'Didn't you [the other economist] tell me [the interviewee] that, in the 80s, forecasts were as right as they are today?' The other instantly replies: 'You mean I said "as wrong"?' Both laugh.

(Institute, Observation, 1 September 2015) Forecasters' discourses in practice question the relevance of a 'reality test' in its most commonsense form. Because fully endorsing such a test would jeopardize the whole project of forecasting, forecasters' justifications often design alternative definitions of forecasting 'rightness' that rely on the procedures it follows rather than its substance. Scrutinizing how 'errors' are dealt with internally provides insights into forecasters' activity and the organization of the forecasting world. This shift implies a move from front stage, where forecast clients stand and public discussions take place, to back stage, wherein lie interpretative and discursive processes, as well as the actual production of forecasts [START_REF] Reichmann | Epistemic Participation. How to Produce Knowledge about the Economic Future[END_REF]. It underlines the role of key features found in both magic and macroeconomic forecasting, such as rituals, professional identity, and representations of causal mechanisms. Ultimately, forecasters' de facto replies to criticisms delineate a singular 'regime of veridiction'; that is, 'the set of rules enabling one to establish which statements in a given discourse can be described as true or false' (Foucault 2004, 35).

Paying attention to insiders' perspective on this issue also makes it possible to understand how forecasters remain motivated to perform an activity that is bound to be 'mistaken', if its purpose is to depict correctly future economic reality. Competing definitions of 'rightness' are connected to differing representations of 'errors': one is defined externally, and the other internally. In line with Hubert's and Mauss's argument that magicians truly believe in their magic, including their own, even if it involves simulation to some degree, forecasters' discourse should not be considered to be merely an expression of dishonesty-even though excluding bad faith a priori would obviously pose mirroring methodological problems. In a market for symbolic goods, actors' justifications are for the most part rooted in sincere beliefs as to the worth of what is done, rather than solely expressing denial or concealing 'real' economic motivations (Bourdieu 1977, 5, 21-22). 7 Going 'back stage' allows us to shed light on forecasters' reflections on their own craft and their judgments on their own performance. These reflections and judgments revolve around three major arguments: the ontological indeterminacy of economies, the constructivist epistemology of forecasting, and the claim of professionalism.

Data and Method

The material drawn upon in this chapter consists of interviews and observations from ongoing research on macroeconomic forecasting that focuses mainly, but not exclusively, on France (that is, macroeconomic forecasts produced about France). Thirty-six interviews have been conducted since June 2014 (average duration: eighty minutes) with economists and forecasters from public (either national or international) and private (banks, insurance companies, and so on) institutions. Among them, ten interviews were conducted with members of the same prominent French forecasting institution, which will be referred to here as the Institute, and where data from observation have been collected upon four successive forecast-production cycles between 2015 and 2017. During the interviews interviewees were encouraged to describe the actual practice of forecasting, rather than just expressing their own views about it in its most abstract aspects. The observations used in this chapter include those obtained at twenty-two work meetings (around three hours each), two press conferences (one hour each), and two debates during which the newly-produced forecasts were discussed by a pair of economists from other French institutions (two hours each).

The purpose of work meetings varies along with the process of forecasting, within which three stages may be roughly identified: post-mortem (the analysis of 'errors' in previous forecasts and of recent economic trends) and scenario and point forecasting. A 'forecast' indeed comprises two distinct elements. One is the 'point forecast', a precise figure depicting the value an economic variable may take in the future (for example, 'next year's GDP growth rate will be x %'), and whose calculation relies on econometric modelling and economic expertise. The 7 For example, Tetlock's analysis of political forecasts regards forecasters' justifications as mere belief defence strategies (Tetlock 2005, 129-137). While stressing similar mechanisms to those that are tackled in this chapter, Tetlock incorporates them as elements for testing forecasters' reliability in a psychology-inspired positivist perspective. However, the distinction he makes between 'Getting it Right' and 'Thinking the Right Way' undoubtedly applies to macroeconomic forecasters as well, as will be shown in this chapter. other is the 'scenario' -a narrative that exhibits a set of causal mechanisms leading from the present to the forecast horizon (the next quarter, semester, year, and so on), established through the combination of national accounts and judgmental assessments.8 Often considered a summary of the 'scenario', the 'point forecast' is determined at the end of the process, once a scenario is agreed upon. It attracts public attention, if only because it fits formats in use in the media, such as newspaper headlines.

Is It Even Possible to Know? The ontological Indeterminacy of Economies

A first line of argument proffered by forecasters seeking to absolve themselves from forecasting 'failure' is related to the nature of economic reality and the tools forecasters use to apprehend it. The unknowability of the future is a constraint on economic decision-making and gave rise to the profession of forecasting. But uncertainty is also a resource for forecasters when facing contradiction. Their poor results in the context of a reality test may be qualified as consequences of the shortcomings of econometric models rather than as 'errors'. By combining equations, models reflect the relationships between major economic variables (prices, exports, imports, public spending, employment, and so on) and describe the behavior of 'representative' economic agents (Evans 1999, 50-76) in a way that is taken to be symbolic of the whole economy. In this respect, models hold a position in the world of forecasting similar to that of instruments and tools in magical rituals: specifically, they belong to the 'class of objects which appear to be used for their own sakes by virtue of their real or imaginary properties, or … because they coincide with the nature of the rite' (Hubert and Mauss 1902, 59).

By definition, models focus only on chosen variables: completeness is not an option. It is thus no surprise that a significant part of the economic literature devoted to macroeconomic forecasting conveys a statistical representation of forecast 'errors' through the notions of 'residuals', 'error terms', or 'loss functions' (see, for example, [START_REF] Elliott | Macroeconomic Forecasting[END_REF]. Residuals capture all elements left outside the equations, such as the unexpected events (such as surprising weather conditions or accidents) that dramatically change the course of events and render the former assumptions, reasoning, and hence forecasts obsolete. These shocks are treated as external or 'exogenous' to the system-and therefore as legitimate excuses for forecasting 'errors'. In practice, though, at least some of these shocks-the 2008 financial crisis for example-are 'endogenous', as they originate from innovations and other features within the economy [START_REF] Bronk | Uncertainty, Modelling Monocultures and the Financial Crisis[END_REF]). More importantly still, although economists discuss plausible stock market movements and possible changes in central bank policies, most, if not all, econometric models do not include the financial sector and its impact on the real economy: this is a consequence of the dominance of a macroeconomic framework within which the 'real' and the 'financial' spheres of the economy are unconnected [START_REF] Goodhart | Macro-Modelling, Default and Money[END_REF][START_REF] Fligstein | Why the Federal Reserve Failed to See the Financial Crisis of 2008: The Role of 'Macroeconomics' as a Sense-Making and Cultural Frame[END_REF].

All shocks or decisive factors not captured by models are treated in the same way, whether endogenous or exogenous. Obviously, the further out the forecast horizon, the greater the odds that shocks will occur. Whereas macroeconomic forecasting, along with other activities such as credit rating [START_REF] Carruthers | From Uncertainty toward Risk: the Case of Credit Ratings[END_REF], purports to turn uncertain futures into calculated probabilities (or 'risks'), to put it in classical terms [START_REF] Keynes | A Treatise on Probability[END_REF][START_REF] Knight | Risk, Uncertainty and Profit[END_REF], shocks ensure the persistence of the very uncertainty that forecasting aims at reducing: the economy remains ontologically undetermined and the future remains open.

The dream of a perfect elimination of uncertainty can never be realized and forecasts can thus never be 'right' in a realistic sense. Forecasters' emphasis on radical uncertainty implicitly acknowledges that economic reality is non-ergodic, that is, non-stationary and not determined by the past. Because 'future outcomes are [not] merely the statistical shadow of past and current market signals' (Davidson 1996, 480; see also the introduction to this volume), forecastingoperation. As described in this paragraph, the first refers to the 'baseline scenario', namely the 'story' (to quote numerous interviewees) that forecasters tell once causal mechanisms are identified and a set of hypotheses is settled upon. Another refers to 'variants', that is, changes in the set of hypotheses and assumptions in order to build 'alternative scenarios' and fathom the risks associated with the baseline scenario (Carnot et al. 2005, 148-154). To avoid confusion between these two meanings, the word 'scenario' will be used in this chapter only in the sense of a baseline scenario or narrative.

which implies assessing economic regularities-is threatened at a microeconomic level by innovations, and at a macroeconomic level by shocks. A review of the ECB Survey of Professional Forecasters (SPF),9 for example, relates the 'errors' panel members made regarding the inflation rate to the various unexpected shocks that affected the newly created eurozone (Garcia 2003, 16).

Economic crashes have a special place among such shocks. The case of the Great Recession, regularly evoked by forecasters in the interviews, illustrates this, all the more as the crisis exploded at the end of a forecasting process. Since forecasts are produced on a quarterly (end of March, June, September and December) or biannual (end of March and September) basis, the collapse of Lehman Brothers on 15 September 2008 took place when point forecasts had been settled. 'Error' then brings into play both the shock itself and the organization and timing of the forecasting process, eventually making forecasts and forecasters 'run off the road', to quote a French economist.

We had our scenario frozen on September 15 th , 2008. The very same day Lehman Brothers collapsed! The forecast was not published yet, but the numbers were set: actually, to be released around October 15th, numbering has to be frozen earlier, next comes a phase of writing and harmonizing. […] We actually had ended up, by mid-September 2008, with a growth scenario that was pretty much settled on and stated 'well, there is a crisis, a liquidity crisis and a rather severe financial crisis' but, back then, it seemed to us that ... Big mistake we made was thinking the economy would be more or less resilient and there wouldn't be another shock like this. So, what happened that day is that we didn't change our forecasts. We are committed, though unofficially: people expect us to release our forecasts around mid-October and, at that moment, it was impossible to start all over again and make a brand new scenario. ... We decided to go public anyway but said we'd make new forecasts by the end of the year. We warned our readers that the forecasts had been made before [the Lehman Brothers collapse] and would be updated. ... Indeed, one bank going bankrupt is not enough for us to say that our scenario for growth in the Eurozone is 'minus 4 %', '4 % recession'. It needs the bankruptcy and a set of later developments. But it was embarrassing because we knew we were going to release something that ran off the road.

(Economist, Institute, 28 April 2015)

The ontological indeterminacy of economies is a key issue for forecasters, who regard it as a major cause of 'errors' and 'mistakes'. External critics argue that this brings discredit on forecasting per se. 'Black Swans'-that is, highly improbable events with devastating consequences-underline experts' inability to accurately predict what departs from the ordinary (Taleb 2007, 206-214). But for most economists, either orthodox or heterodox, the 'Black-Swan theory' is merely a wake-up call that emphasizes the limits of forecasters' reasoning and modelling in the face of indeterminacy (Galbraith 2014, 4). Significantly, even ninety years after the Great Crash, the economic literature continues to address the possibility of being able to predict it, sometimes leading to posthumous reassessments of the importance and significance of economists with respect to their 'prescience' [START_REF] Irwin | Who Anticipated the Great Depression? Gustav Cassel versus Keynes and Hayek on the Interwar Gold Standard[END_REF]. For instance, Dominguez and her colleagues (1988) argue that the Great Depression could not have been foreseen either by contemporary forecasters (the then competing Harvard Economic Service led by Warren Persons and the team Irving Fisher directed at Yale10 ) or, more interestingly, by forecasters making use of late 1980s methods and data. In the same breath, the authors thus both refute the assertion that Harvard forecasters actually had anticipated the Crash (Bullock and Crum 1932, 137;[START_REF] Schumpeter | History of Economic Analysis[END_REF]Schumpeter , 1131) ) and exculpate all forecasters from any period, on the grounds that 'error' is the consequence of unexpected events outside all models.

On this line of reasoning, the number of 'mistaken' forecasters serves as a proxy measure of the historical uniqueness of an event, and hence of its unexpectability. As McGoey (2012, 563-571) argues, collective ignorance offers an 'alibi' that forecasters use to deflect individual accountability: the wider the extent of ignorance, the more easily 'mistakes' are denied. The very fact that no expert knew about or understood what was coming proves it was impossible to know. Collective ignorance does not make the reality test ineffective as such but it makes forecasters immune to its verdict as they are not accountable for not having predicted what was unpredictable: forecasts were 'wrong' but no error was made.

Contrary to mainstream economics, whose perfect-competition hypothesis assumes predictability as a methodological feature of reasoning, forecasting as a form of applied economics deals only with actual economic conditions and thus must cope with a reality that is partly indeterminate and uncertain. The activity of forecasting is, consequently, doomed to be at least a partial failure: by definition, it cannot capture ex ante what a dynamic and unstable 'reality' will be. This provides forecasters with a rationale to discard a supposedly stable 'reality' as a relevant benchmark to assess the accuracy of forecasts.

At the same time, acknowledging the difficulties of forecasting involves shedding new light on the process of forecast production and suggests methodological refinements. In this respect, 'errors' mark the shifting frontier of knowledge about the economic environment, the relationship between variables in an econometric model, and the values they are expected to take. Hence the statement that forecasts are usually 'erroneous' leads to the apparently paradoxical conclusion that forecasting methods and models should be improved rather than withdrawn once and for all [START_REF] Evans | Soothsaying or Science? Falsification, Uncertainty and Social Change in Macroeconomic Modelling[END_REF]. As part of a never-ending trial-and-error process, 'error' may then even be of some value by pointing towards the limits and flaws of forecasting models and reasoning.

What Is It All About? The Epistemology of Forecasting

Emphasizing the way in which forecasts are produced offers forecasters another argument for proclaiming the irrelevance of reality testing. Criticisms addressing macroeconomic forecasting usually adopt a 'realistic perspective': they focus on numerical outcomes to compare them ex post to the 'reality' they were supposed to anticipate 'correctly'. 'Mistakes' then come from mismeasurements [START_REF] Desrosières | The Politics of Large Numbers. A History of Statistical Reasoning[END_REF]. A stress on the nature and epistemological basis of forecasting helps forecasters to challenge this standpoint on two different, yet related, grounds: first, by focussing on the nature and consequences of forecasting as a practical activity based around scenarios; and secondly, because the adoption of a 'realist' perspective on forecasting is considered misguided, because the use of statistical data requires the adoption of a 'constructivist' [START_REF] Desrosières | Entre réalisme métrologique et conventions d'équivalence: les ambiguïtés de la sociologie quantitative[END_REF] or 'constructionist' [START_REF] Hacking | The Social Construction of What?[END_REF]) perspective.

To a large extent, narrative forecast scenarios matter more to forecasters than calculated point forecasts. Forecasters often describe the establishing of scenarios as the 'true' purpose of their activity. Macroeconomic evolutions and causal mechanisms are then described in relation to a statistical and qualitative assessment of the current macroeconomic situation. Qualitative information is distributed along a wide range of phenomena, from public policy announcements to scandals involving major firms, through elections, weather conditions, and geopolitical environment. Built from a set of assumptions and guiding the subsequent econometric modelling, narratives illustrate the blend of statistical data and judgment that characterizes macroeconomic forecasting. Scenarios rely on experience, expertise, and intuition, all of which are assumed to endow forecasters with an ability to spot economic processes as they unfold: while economic reality is too erratic for accurate point forecasts, it remains predictable enough for regularities to be discerned. Such scenarios comprise representations of causal mechanisms in the economy. As emphasized in the Durkheimian tradition, religion, science, and magic all rely on representations of causality: whether in Newtonian science, primitive religion, or magic, unexplained 'forces' ultimately operate as explanatory principles [START_REF] Durkheim | The Elementary Forms of the Religious Life[END_REF]. A more or less comprehensive analysis shows how scientific, religious, or magical actions can set these forces in motion in order to produce effects (Hubert and Mauss 1902, 75, 78). In macroeconomic forecasting, representations of causality deal only with the analysis of economic causes and consequences.

Differences between forecasts from different institutions, or from the same institution at different times are thus often related to differences between scenarios that, in turn, may be understood with reference to the various possible depictions of economies. Baseline scenarios remain interpretations of the present economic environment, and one of the key challenges facing forecasters is selection of the relevant information. Besides, scenarios are thoroughly discussed within forecasting institutions and among forecasters from different institutions. As Reichmann (2013, 857; see also [START_REF] Evans | Social Networks and Private Spaces in Economic Forecasting[END_REF] notes, 'economic forecasters are permanently negotiating their views with each other and with others to come to a certain view'. Since interpretations based on judgments and shaped through collective deliberation are central to scenarios, they are often revised as new information and elements are taken into account. In this way, they combine facts (what is known) and bets regarding developing situations in a way reminiscent of the way in which uncertainty is managed in the case of journalism [START_REF] Pilmis | Producing in Urgent Situations. The Management of Unpredictability in the World of Journalism[END_REF].

This can be illustrated by the case of 'Brexit'. Until the vote (23 June 2016), forecasters from the Institute bet that the United Kingdom would remain in the EU, in accordance with a common 'nothing changes' assumption, and hence Brexit was not regarded as a key issue for forecasting. By contrast, once the referendum results were known, Brexit gained major importance in the process of forecasting. This can be measured by the amount of time devoted to it: Brexit was not covered in any of the five observed meetings of the spring 2016 forecasting session, whereas the fall 2016 work meetings usually begin with Brexit, with up to two hours devoted to it. It may also be measured in terms of coverage in the published forecasts: some two and a half pages out of 180 in spring 2016 (with the word 'Brexit' appearing on ten occasions) as against twenty-one pages out of 206 in fall 2016 (with the word 'Brexit' appearing on sixty occasions).

Forecasters often present narratives as the 'real' forecasts, those that should be paid attention to rather than 'point' forecasts. In this perspective, forecasting has more to do with storytelling than with calculation. Whereas it seems easier to assess the accuracy of forecasts, and thus their 'errors', with reference to calculated numbers, the quality of storytelling is a more appropriate, though less convenient, test of forecasts. 11 To fellow forecasters, a 'good', or 'right', forecast 'correctly' describes macroeconomic progression in such a way that the 'story' matches the coming string of events, even if the anticipated figures are 'wrong'. The often-quoted expression 'being right for the wrong reasons, and being wrong for the right reasons' suggests that forecasting is not subject to simple falsification because it leads to assertions unsuited to a process of refutation. In short, forecasters argue for a more nuanced epistemological position in relation to their craft.

The partial construction by contingent theoretical frames of the data used in forecasts also introduces indeterminacy. This suggests that it is appropriate to adopt a constructivist rather than a realist epistemology, which deals with the work routines, deliberations, and conventions involved in the process of forecasting. Conventions characterize not only forecasting institutions but also those surrounding them. A detailed investigation of the principles according to which data are built and knowledge is produced is often described as a prerequisite for judging forecasting: the classical statement that data do not preexist their collection or gathering is a common caveat among forecasters. This is obviously the case for abstract notions, such as potential GDP or the output gap, with which actual economic performances are compared and whose estimations vary widely across forecasting institutions. It is also true of the more or less formal information forecasters collect from the economic actors they meet, as well as of the raw material statistical bureaus supply them with. Forecasting 'errors' may originate in misleading information or erroneous statistical data, as they may lead to 'inaccurate' depictions of the economy. The case of economic indices exhibits the constructivist point of view forecasters adopt: whereas they are regularly considered to be reliable measures of economic activity (as shown by their use as the unquestioned 'reality' against which forecasts are compared), their regular revisions have a strong impact on macroeconomic forecasting.

You need people with a bit of knowledge about public stat[istics], people who know how key figures, those everyone pays attention to, are constructed. When you've been in the public sector, you know in what condition figures are brought to you, because you have practiced their production, so you're a bit better than others since you know what may play. I, for instance, since I used to forecast inflation, I know that in May and December, revisions are large because basic welfare benefits rise. So, these two figures are a bit more important. I know that rises in the price of natural gas (by 2.3% as announced today) or transportation will have an impact on inflation on the following month… (Chief economist, insurance company, 13 February 2017) Statcan [Statistics Canada] is for me a model. And why does it work great? They estimate GDP monthly, and it is a very good proxy for the upcoming quarterly data, with few revisions. That's why I told you [earlier] that 'the [Japanese] data is no good'. Japanese data is tremendously 11 Forecasters regularly complain about what seems to them an excessive focus on numbers, leading to situations that are described as 'absurd': calling a 1.5% GDP growth rate forecast 'wrong' because the observed rate was 1.3% would just be a matter of the 'thickness of the line' (Economist, asset manager, 22 October 2015).

revised. And when you use a model to forecast GDP in the short run, you not only check the last statistical point, but also the preceding quarters. If the quarters are revised a lot, you'll be mistaken a lot. That's why I called them 'disastrous', because of the revisions.

(Economist, OECD, 9 November 2015)

Discussing the nature and purpose of forecasting gives forecasters the opportunity to dispute the reality test as a relevant assessment of quality. First, they argue that a focus on calculation misses the 'true' locus of forecasting, which consists in establishing scenarios. Though less apparent, scenarios are presented as a decisive outcome of forecasting. Secondly, forecasters' epistemological arguments make it possible to qualify the results of a reality test as consequences of the data used in the process of forecasting. As data come at the conclusion of a series of conventions and operations, they may not accurately measure today's economic situation; and, consequently, neither can forecasts envision tomorrow's situation. Like magical rituals, forecasting implies that 'the confection or preparation of [the] materials, the ritual ingredients, is a long and finicking business' (Hubert and Mauss 1902, 58). 'Error' springs not only from the centre of the universe of forecasting, but also from its margins, where raw data are produced at the earliest stage of the forecasting process. In any event, forecasters should not take the blame. The emphasis on using second-hand data and information to produce forecasts ultimately depicts the universe of forecasting as a 'social world', or as a segment of a broader 'social world of economic analysis'. Like the art worlds H. S. [START_REF] Becker | Art Worlds[END_REF] studied, the 'forecasting world' cannot be reduced to those who are located at its heart: beyond the most obvious practitioners (forecasters themselves), support personnel-such as statistical agencies or economic actors who 'epistemically participate' [START_REF] Reichmann | Epistemic Participation. How to Produce Knowledge about the Economic Future[END_REF] in the activity-have to be taken into account.

What Makes a 'Good' Forecaster? Forecasters' Claims of Professionalism

The notion that forecasting is a 'social world' implies that it is ruled by a set of conventions according to which there is an established boundary between what ought to be done and what not, enabling cooperation between actors (Becker 1982, 28-34). Conventions delineate a set of rules and methods that 'professional' forecasters are expected to comply with. Although central bank staff are unlikely to be familiar with the sociology of professions and occupations, their actual definition of what the forecasting 'profession' might be when surveying 'professional forecasters' is noteworthy. They identify the implemented methods as a key feature of professionalism: in the case of the ECB-SPF, the selection and inclusion of panelists are delegated to each member country's central banks. The criteria applied include a 'formalized approach' (by which is meant the use of macroeconomic modelling), which is taken as crucial evidence that a given forecasting institution 'fits the … description [of] professional forecasters' (Economist, European Central Bank, 26 June 2014). Generally speaking, according to forecasters, a major aspect of 'professional' forecasting is consistency; that is, the mutual compatibility of all macroeconomic aggregates that are included in the analysis, whether it relies on narratives or statistical models. Specifically, macroeconomic forecasts have to fit national accounting principles, and for example make sure the total exports from all countries equal total imports. Some widely used terms, such as 'closure' (French: 'bouclage'), express this concern.

Both econometricians dealing with large-sized statistical models and economists who only make use of spreadsheets to implement small-scale models consisting of supply-use tables insist on the importance of consistency for forecasting. A national accounts basis for forecasting professionalism delineates the legitimate forms of forecasting and operates as a key criterion to judge the quality and validity of forecasts. Mavericks' claims to have foreseen what other forecasters missed is often met with skepticism, if not hostility, due to their perceived lack of professionalism: it is no surprise that they are regarded by 'professional' forecasters as mere outsiders whose reasoning lacks basic economic principles and who should be kept beyond the boundaries of the world of macroeconomics. The skills of economic 'visionaries' are regularly disputed. By stressing their lack of training in macroeconomics and challenging the outlined causal sequence, forecasters question the expertise of people who were probably just 'lucky'.

Sometimes you read something and think 'this is rubbish', because it doesn't fit, there is no closure with respect to macroeconomics. I do not pretend mine are always perfect but still… ... Actually a first simple closure is accounting identity: make sure national accounts are a closed system, which means one cannot see an increase in households saving rate and an increase in consumption if income is lower. Unfortunately, it happens sometimes. ... Well, it's a bit more subtle. If you take a look, you see the income doesn't rise enough to allow what has been reported. It's not that blatant, but still. Accounting identities have to be checked. Then another thing is … for example, if you report a decrease in growth in China, in the US, and in France, you cannot see a rise in the foreign demand for French goods. There is an identity at the international level which is actually pretty weighty because world demand has to be taken into account. (Economist, research institute, 23 October 2015) Predicting the worst is a good strategy because as time goes by … See, nowadays, X [another French observer who, the interviewee assures us, is a professor of finance and 'knows nothing of macroeconomic matters'] made a business out of it. He keeps repeating 'the crisis lies ahead'. And he is right: the crisis is ahead! Inevitably there will a crisis at some point. If you want to win the media, you have to say the crisis lies ahead because afterwards people think you should have been listened to. ... Last summer, X said (we know each other well, he is a nice guy by the way) 'there is a crisis ahead'. 'Sure, you're right, date it'. 'This summer.' And, once the Chinese thing occurred [the 2015 Chinese slowdown], he came back: 'See? I told you so.' 'No, you didn't say that. You said the stock markets would collapse in the United States and Europe for intrinsic reasons that had nothing to do with emerging countries'. (Economist, Institute, 7 September 2015) Conversely, forecasters who go by the book are exonerated of forecasting 'errors': they are not to be held responsible for 'mistakes' since none have been made. Forecasters advocate an approach to 'error' that relates it exclusively to the degree of compliance with internally-defined standard practices and formal rules. In line with the Durkheimian view that magicians represent an example of early professionals, forecasters' emphasis on procedures seem connected to their claim that their professional identity is first and foremost that of economists who shall be judged according to the narratives they set up rather than the calculations they end up with. Like other occupations, protection from blame is obtained by proving that the ritual of forecasting has been followed: insofar as the process has been correctly handled, there is no point in criticizing its outcomes and only 'professionals' are, consequently, authorized to assess that 'errors' have been committed (Hughes 1951, 324-325).

As noted in the case of magic, the general belief in forecasting remains to a certain extent immune to contradictory experiences because it may be argued that the dubious ritual was not properly executed, rendering it null and void. The conditions required for magic or forecasting to be successful are numerous 'to such an extent that it seems to be providing itself with loopholes, and often successfully', so that 'it is natural for a magician to take refuge behind questions of procedures and technicalities, to protect himself in case of failure of magical prowess' (Hubert and Mauss 1902, 62). Forecasters, then, like magicians, argue that all that really matters is that they follow due process.

The outcome can be that stage-front (public discussion) is dismissed as a place for disputing the accuracy of forecasts and that discussions remain confined to the back-stage area (the profession itself): while economic actors may provide forecasters with relevant information, debates and disputes are kept between 'professionals'. As mentioned earlier, that does not mean that forecasters do not acknowledge that forecasts are frequently 'erroneous' from a realist point of view, but a firm distinction is made between 'erroneous forecasts' and 'faulty forecasters'. As a social world based on shared conventions, the forecasting world also displays some of the properties of 'professional' ecologies. The combination leads to a representation of the world of forecasting as autonomous. Social boundaries protect forecasters both from competitors whose abilities are contested and from criticisms originating from outside the profession.

However, forecasting cannot achieve full autonomy, because it necessarily remains connected to the economy, which takes part in the process of forecasting and within which forecasters' clients are to be found. At some point, forecasters are bound to turn back to the external sphere and stage-front. Ironically, then, the distance that forecasters can put between themselves and the reality test should not be overestimated. Provided the general credibility of forecasting is maintained, the credibility of specific forecasts or forecasters can be questioned. Indeed, regarding its connection to the outside, the forecasting world shares features with a market within which actors compete with each other to access economically defined opportunities with clients and customers (Weber 1922, 635-640). Such competition necessarily takes place in the foreground and uses the 'reality test'-that is, the accuracy of ex ante forecasts-to assess forecasters' reliability: 'errors' may be put forward as 'evidence' that some forecasters are more trustworthy than others. The reality test is then commonly used to increase the symbolic capital or prestige of individual forecasters or forecasting institutions.

[At the end of the interview, the interviewee shows me a couple of files on his computer] I can't help the sweet treat: you'll get why. [He opens a spreadsheet: two tables display data from a dozen forecasting institutions -French government, OECD, IMF, research institutes and banks -for two different periods : 1999-2016 and 2013-2016]. So I look at what has been forecasted for Year y in September of Year y-1, then I look at the first GDP growth estimate for Year y, which is released in January of Year y+1. Obviously I consider the difference in absolute value. Here it is: our average difference for 2013-2016 is 0.15 point [he waves the computer mouse to stress this figure is lower than that of other institutions]. For example, we forecasted an average 1.15 GDP growth and it came out at 1. ... I fill the form myself. So, here is what my little fellows have marked for 2017 so, when it'll be January 2018, I will check it out. ... Everyone is convinced [they] have the best forecast except [he says it as if whispering a secret] that I have the evidence.

(Chief economist, research institute, 22 March 2017) Like many other social spaces, the forecasting world is not characterized by an opposition between symbolic and economic rewards: on the contrary, both coalesce to a large extent. Indeed, retaining clients and gaining new ones is a key concern for privately-funded institutions. In this perspective, the accuracy of forecasts exhibits 'evidence' of excellence in economic analysis, which 'clients remember' (Economist, private bank, 30 September 2015), even though they often value the narrative element of forecasting as well. Dismissed as being irrelevant to assess the quality of forecasts in general terms, the reality test remains the most commonly used instrument when specific macroeconomic forecasts come to be publicly debated.

Conclusion

The contradiction between what had been anticipated and what actually happened is a common challenge for all belief-based practices, and serves as the basis for comparing present-day macroeconomic forecasters to the magicians that Hubert and Mauss analysed in the early twentieth century. Rather than presenting the condescending claim that sophisticated macroeconometric models barely conceal primitive forms of reasoning, however, the aim of this chapter is to suggest that, though different, magic and forecasting share important traits. Both are activities whose results become apparent somewhere down the line, and in each case their performance relies on precise forms of representation, purposively designed materials, defined sets of procedures, and trained professionals.

The analysis presented in this chapter of the discourse forecasters use to legitimate 'errors' and preserve forecasting from a reality test that could endanger its very existence involves three arguments. Taken together, they serve to reduce the relevance of a comparison between ex ante and ex post figures to assess the 'quality' of forecasts and to mitigate the impact of forecasters' overall poor records in this respect.

In the first place, forecasters argue that forecasting focusses on narratives rather than on calculations: figures and numbers are thus discarded as the appropriate criterion by which forecasts should be judged. Instead, narratives identify the causal mechanisms through which economic processes unfold, and make representations about causality a key feature of forecasting. In this perspective, a 'reality test' based on estimating the accuracy of ex ante point forecasts is revoked.

The causal representations may also account, however, for the discrepancies between forecasts and 'reality' in order to dispute their qualification as 'errors'. Indeed, forecasters contend that many representations of 'reality' are misleading, as they assume that economies are more stable and predetermined than they actually are. The causal sequences economic theory posits, for example, are often pulled off track by contingent-hence unanticipated-phenomena. A proper appreciation of the indeterminacy of economic futures, consequently, absolves forecasters of blame for 'errors'.

The materials used to produce forecasts pose a second set of challenges. Whether they are made up of qualitative information or statistical data, they are considered a possible source of 'error'; namely, as the result of the conventions involved in their construction. While, on an ontological level, economic 'reality' is regarded as shaped by events over which forecasters have no control, this epistemological argument suggests that attention should be paid to the series of operations leading to the depiction of such 'reality'. The implication, again, is that forecasters should not necessarily be held responsible for 'errors'. A reality test is not, in this case, dismissed in principle, but its results are framed in such a way that they may not be qualified as 'errors'.

A third argument puts forward the notions of ritual and profession. Forecasters claim that they are mostly, if not exclusively, accountable for their compliance with sets of standards and practices. In this perspective, the process of forecasting matters more than its results: provided forecasts are produced according to the proper methods, they may not be considered faulty. A 'professional' representation of the forecasting world disputes the relevance of an external adjudication of the reality test: instead of the test being part of a public debate, the emphasis on professionalism tends to confine discussions about forecasting 'quality' within the limits of the forecasting world. Another shared feature between magic and forecasting is thus their separation from the ordinary, or 'profane', world. Ontological, epistemological, and professional claims resonate to challenge the appropriateness of the reality test in general.

Nevertheless, forecasting cannot stand entirely apart from the rest of the economic world. Stage-front does not lie so far away from back-stage. Since its actual purpose remains to guide economic actions, forecasting takes part in a division of labour that makes it necessary for forecasters to engage in relations with 'outsiders'. Moreover, while forecasts are regularly used to depict the future evolution of the economy, especially in the private sector they are also used competitively to assess the quality of economic analysis and gain clients. Forecasts and forecasters' credibility thus remain at stake. Despite forecasters' efforts to frame 'errors' as an issue solely for insiders, the reality test cannot be dismissed once and for all. But the resurgence of a reality test presupposes the narrowing of its scope: maintaining a general belief in forecasting is a prior condition for disputing particular forecasts and forecasters.

In early 2017, the chief economist of the Bank of England provided an example of the necessity for forecasters to address public criticisms (Phillip Inman, 'Chief economist of Bank of England admits errors in Brexit forecasting', The Guardian, 5 January 2017. https://www.theguardian.com/business/2017/jan/05/chief-economist-of-bank-of-england-admits-errors, retrieved 11 January 2017).

In the world of forecasting, two kinds of 'scenario' may be distinguished, along with two different kinds of

The ECB-SPF is a quarterly survey of expectations concerning inflation rates, real GDP growth, and unemployment in the euro area for several horizons. Explicitly inspired by a similar survey carried out in the United States in 1968 by the American Statistical Association and the NBER, and taken over by the Federal Reserve in Philadelphia in 1990, the ECB-SPF was launched in 1999 to coincide with the introduction of the euro.

This early period of forecasting in the United States has been vividly recounted by W. A.[START_REF] Friedman | Fortune Tellers. The Story of America's First Economic Forecasters[END_REF].