Question:
You are given the full text of a scientific research paper.

    Summarize the paper's key **research findings** in the style of a policy brief. Your response should:
    - Focus on the major empirical or model-based results
    - Present findings that are policy-relevant and supported by the study
    - Use concise, non-technical language with essential technical terms as needed
    - Avoid quoting directly or summarizing minor details

    Write a short, clear paragraph appropriate for the “Scientific Research Findings” section of a policy brief.

    Scientific Research Paper:
    Articles
https://doi.org/10.1038/s41560-020-0549-1
Department of Earth and Environment, Boston University, Boston, MA, USA. *e-mail: kaufmann@bu.edu
B
etween the second quarters of 2007 and 2008, the nominal
price for a barrel of crude oil rises from US$65 to US$124.
Nine months and a financial crisis later, the same barrel sells
for about US$43. Such rapid changes suggest a speculative bubble,
which the US Commodity Futures Trading Commission defines as
a rapid run-up in prices caused by excessive buying that is unrelated
to any of the basic, underlying factors affecting the supply or demand
for a commodity (https://www.cftc.gov/ConsumerProtection/
EducationCenter/CFTCGlossary/glossary_s.html). But empirical
evidence for a speculative bubble is mixed1,2.
Nor is the 2007–2008 spike and collapse the first time that prices
seem to move away from a trading range implied by market funda-
mentals. Between the second quarters of 1973 and 1974, nominal
oil prices nearly triple, from about US$3.50 to US$10 per barrel.
This increase coincides with production reductions and embargoes
by the Organization of the Petroleum Exporting Countries (OPEC)
that reduce net production by about 3.1 million barrels per day
(mbd), which represents about 5.5% of total output. But histori-
cal perspectives suggest that these reductions are not the primary
driver of the price increase, “it was not loss of supply, but fear of
possible loss that drove up the price” (ref. 3).
Beyond market fundamentals, oil prices may respond to geo-
political events4,5 and structural changes6. The structure of the oil
market changes when the ‘Seven Sisters’, a group of seven interna-
tional oil companies that dominated the world oil market between
the 1940s and the early 1970s, sign 50-50 profit-sharing agreements
with producing countries and changes again when producing coun-
tries nationalize properties owned by the Seven Sisters. But analy-
ses that focus on such changes largely ignore market fundamentals,
which they represent by assuming that prices revert to a constant
mean or follow a deterministic trend.
Here, we identify the price effects of speculative bubbles and
changes in market governance and property rights by defining
regimes as two or more consecutive quarters when oil prices move
away from the level implied by market fundamentals. To identify
regimes, we start with a simple assumption: in a world without
regimes, a stable long-run cointegrating relation translates market
fundamentals into an equilibrium price for oil. Observed prices
move towards these ever-changing equilibria as represented by an
error correction model (ECM). Statistically significant deviations
from market fundamentals (that is, regimes) are identified by an
indicator saturation technique. This methodology identifies nine
regimes that we tie to a speculative bubble in 2007–2008 (and 2010),
changes in property rights, such as OPEC nations gaining control
of oil resources from the Seven Sisters, and changes in market gov-
ernance, such as the extensive US energy legislation in the 1970s.
Conversely, not all sharp price changes constitute regimes: market
fundamentals account for the 13% decline in real prices during the
Asian financial crisis, which indicates that market fundamentals can
account for some of the price changes that are attributed to geopo-
litical events by previous analyses.
A cointegrating relation for price
Because a cointegrating relation for price cannot be chosen a priori,
we estimate 58 possible specifications (Supplementary Table 1).
Of these, we focus on nine (Models 1–9), which specify a single
measure of capacity utilization by the Texas Railroad Commission
(UtilTRC) and OPEC (UtilOPEC), and include inventories (Days)
and refinery utilization rates (UtilRef), that cointegrate with and
have a statistically measurable relation with oil prices. ECMs find
that oil prices adjust towards the equilibrium implied by the cointe-
grating relation (Supplementary Table 2). After we allow for regime
changes, UtilOPEC no longer has a statistically measurable long-
run relation with real oil prices (Table 1). Similarly, the statistical
significance of UtilTRC disappears when specified as a cubic term.
Finally, the statistical significance of Days disappears in many mod-
els that use a non-linear specification of UtilTRC or UtilOPEC.
Of the nine models estimated, the elements of the cointegrat-
ing relation in the simplest Model 1 indicate that a 1-day increase
in crude oil inventories reduces the equilibrium price by about
US$0.14, a one percentage point increase in refinery utilization
rates reduces the equilibrium price by about US$0.92 and a one per-
centage point increase in capacity utilization by the Texas Railroad
Commission (TRC) raises the equilibrium price by about US$0.21
(Supplementary Note 3). We focus on Model 1 because it is the
simplest. But this focus does not affect the discussion that follows
because results generally are similar across all models (Table 1 and
Supplementary Table 3).
Oil price regimes and their role in price diversions
from market fundamentals
Robert. K. Kaufmann    * and Caitlin Connelly
Speculative bubbles, market governance and property rights are thought to affect oil prices, but their timing and magnitude
are uncertain. Here, we quantify these effects using econometric techniques that identify periods between 1938:1 and 2018:3
(denoting year:quarter) when prices strayed from the levels implied by market fundamentals. We identify nine price regimes
that are associated with the Organization of the Petroleum Exporting Countries gaining control over the marginal supply of crude
oil, US energy legislation, a precautionary demand shock, the Arab Spring and speculative bubbles. These bubbles raised real oil
prices by US$14.31 and US$4.65 per barrel in 2007:4–2008:3 and 2010:1–2011:1, respectively, which transferred US$42.8 bil-
lion from US consumers to US oil producers and US$87.4 billion from the US economy to oil exporting nations. Conversely, some
sharp changes, such as the price decline associated with the Asian financial crisis, can be explained by market fundamentals.
Nature Energy | VOL 5 | February 2020 | 141–149 | www.nature.com/natureenergy
141

Articles
NATUrE EnErgy
ECMs indicate that prices adjust towards the equilibrium that is
implied by the cointegrating relation (Table 1 and Supplementary
Table 2) in about four quarters (ln(2)/0.17). The first differences
of UtilRef, UtilTRC, Days and Price have a short-run relation with
Price in all ECMs, whereas there is no short-run relation with
UtilOPEC in any ECM.
Identifying regimes
The saturation indicator technique identifies periods when prices
move away from the level implied by market fundamentals for two
or more consecutive quarters. Regimes start in the quarter when a
step (SIS) changes the equilibrium price and end when the next step
either moves the equilibrium price back towards or further away
from the equilibrium that is implied by market fundamentals. For
models that specify a linear (or squared) relation between price and
capacity utilization by producer organizations, the gets procedure
within the R package of that name identifies 12 steps (Table 1),
which define nine regimes (Fig. 1). For each regime, the devia-
tion from the equilibrium price implied by market fundamentals
(rectangles in Fig. 1) is given by the accumulated value of individual
steps. When the sum of steps is not statistically different from zero
(that is, the sum of step 1 and step 2), no regime is present.
The start and end dates of the regimes identified by the nine
models are similar, except for Regime 5, which starts either in
the first or third quarter of 1981, and the end of Regime 8, which
starts in the third or fourth quarter of 2009 or the first quarter of
2010 (Table 1). Eight regimes are identified in models that specify
UtilTRC3. Finally, the regimes identified by Model 1 are robust
to the sample period, the variables used to measure price and/
or crude oil inventories, and the frequency of the observations
(Supplementary Note 4).
Interpreting regimes
Results for the cointegration/ECMs are consistent with Occam’s
razor; market fundamentals account for much of the variation in oil
prices in 139 (or 138) of the 323 quarters in the 1938:1–2018:3 sam-
ple period. During the other 184 quarters, the price implied by the
cointegrating relation is altered by regimes (Table 2). Most regimes
Table 1 | Regression results for different models of equilibrium oil price
Model 1
Model 2
Model 3
Model 4
Model 5
Model 6
Model 7
Model 8
Model 9
Dynamic ordinary least squares (DOLS) estimate of the cointegrating relation (see Equation (4) in the Methods section)
Constant
0.258
0.210
0.209
0.242
0.204
0.211
0.379*
−0.086
−0.089
Days
−0.190*
−0.177*
−0.178*
−0.134
−0.130
−0.137
−0.318***
−0.078
−0.081
UtilRef
−0.528***
−0.471***
−0.473***
−0.487***
−0.437***
−0.443***
−0.440***
−0.393***
−0.400***
UtilTRC
0.504***
0.450***
0.452***
UtilTRC2
0.497***
0.457***
0.469***
UtilTRC3
−0.013
0.066
0.072
UtilOPEC
0.010
0.071
0.022
UtilOPEC2
0.076
0.054
0.013
UtilOPEC3
0.084
0.066
0.025
SIS1946 Q1
−0.874***
−0.838***
−0.854***
−0.862***
−0.834***
−0.839***
−0.660***
−0.576***
−0.574***
SIS1949 Q4
0.700***
0.632***
0.636***
0.624***
0.636***
0.646***
0.475***
0.483***
SIS1953 Q1
−0.468***
−0.473***
SIS1957 Q3
−0.635***
SIS1970 Q3
−1.041***
−0.947***
−0.950***
−0.948***
−0.871***
−0.892***
SIS1974 Q1
0.929***
0.952***
0.969***
0.833***
0.857***
0.872***
1.110***
0.933***
0.941***
SIS1979 Q3
1.304***
1.233***
1.233***
1.336***
1.271***
1.267***
1.312***
1.300***
SIS1981 Q1
−1.754***
−1.789***
SIS1981 Q3
−1.717***
−1.725**
−1.756***
−1.761***
−1.764***
−1.778***
−1.780***
SIS2004 Q3
1.205***
1.242***
1.230***
1.244***
1.279***
1.259***
1.278***
1.335***
1.317***
SIS2007 Q4
1.564***
1.608***
1.599***
1.599***
1.623***
1.613***
1.637***
1.653***
1.642***
SIS2008 Q4
−2.269***
−2.398***
−2.377***
−2.418***
−2.537***
−2.358***
−2.404***
−2.556***
−2.380***
SIS2009 Q3
1.134***
1.141***
SIS2009 Q4
1.090***
1.080***
1.106***
1.058***
1.100***
1.062***
SIS2010 Q1
1.029***
SIS2011 Q2
0.613***
0.648***
0.636***
0.670***
0.731***
0.633***
0.673***
0.728***
0.635***
SIS2014 Q4
−1.564***
−1.596***
−1.601***
−1.603***
−1.628***
−1.627***
−1.548***
−1.664***
−1.661***
Ordinary least squares (OLS) estimate of ECM (see equation (6) in the Methods section)
μt−1
−0.17**
−0.19***
−0.19***
−0.16**
−0.21***
−0.18***
−0.19***
−0.17***
−0.16***
Independent variables include US refinery utilization rates (UtilRef), days of forward consumption in US oil inventories (Days), capacity utilization by Texas Railroad Commission (UtilTRC) and capacity
utilization by OPEC (UtilOPEC). DOLS regression results for the sample period 1938:1–2018:3 (denoting year:quarter) for the cointegrating relation given by Models 1–9 (Equation (4)) and the error
correction term (ρ) that is associated with the lagged equilibrium error (μt−1) from the ECM (equation (6)). Note that all models represent cointegrating relations for price. Periods when price changes
from the equilibrium price in a statistically significant fashion (steps) are given by step indicator saturation (SIS) followed by the year and quarter when the deviation occurs. Test statistics reject the null
hypothesis at *P < 0.05, ** P < 0.01 and ***P < 0.001.
Nature Energy | VOL 5 | February 2020 | 141–149 | www.nature.com/natureenergy
142

Articles
NATUrE EnErgy
occur after OPEC gains control over the marginal supply of crude
oil in the early 1970s (Regime 2).
Once the cointegrating relation allows for regimes, UtilOPEC
does not have a statistically measurable long- or short-run relation
with prices. The lack of a significant relation does not indicate that
OPEC is unimportant. Rather, the lack of a relation may be caused
by changes in OPEC behaviour, such as a change from acting as the
marginal supplier to defending a fair share of the market (and vice
versa). Such changes alter the relation between UtilOPEC and Price;
the relation would be positive when OPEC acts as the marginal sup-
plier and negative when OPEC acts to defend a fair share of the mar-
ket. Such changes in sign prevent the statistical methodology from
quantifying a coefficient for UtilOPEC that applies over the entire
sample period.
To understand price regimes in Model 1, we relate each to real-
world events and explain why some changes noted by previous
analyses do not appear in Table 2. For regimes prior to 2000, expla-
nations borrow heavily from a chronology of the world oil market7;
after 2000, explanations rely heavily on articles in the industry
newsletter Petroleum Intelligence Weekly. We recognize that associ-
ating real-world events to regimes represents post hoc explanations.
This cannot be avoided because there is no a priori definition for a
price regime and it is very difficult to quantify time series that could
represent such regimes.
Regime 1, 1946:1–1949:3, witnesses a price reduction by an aver-
age of US$9.26 relative to the equilibrium price implied by market
fundamentals (Supplementary Table 3). This reduction is opposite
the price increase that is associated with the elimination of price
controls on motor gasoline after World War II. During the war, con-
trols create a lack of capacity, as indicated by chronic shortages in
drilled wells and transportation facilities8,9. When controls are lifted,
price increases are suppressed by non-market interventions. For
example, Standard Oil of Indiana and Phillips Petroleum Company
ration gasoline to dealers in June 1947 (ref. 10). To stimulate new
capacity, the US federal government relaxes anti-trust legislation to
–30
–20
–10
0
10
20
30
40
50
60
1938:1
1948:1
1958:1
1968:1
1978:1
Year:quarter
1988:1
1998:1
2008:1
2018:1
Price (US$1982 per barrel)
Regime 1 US$9.26
Regime 2 US$11.03
Regime 3
US$1.19
Regime 4
US$12.63
Regime 5 US$5.95
Quarterly change
in price
Regime 6
US$6.82
Regime 9
US$17.85
Regime 7 US$23.39
Regime 8 US$10.90
Simulated
price
Observed price
Equilibrium
price
Fig. 1 | Modelled and observed prices under different regimes. The nine regimes identified from Model 1 by the indicator saturation technique. The real
(US$1982 per barrel) price for crude oil simulated by the cointegrating relation (blue line) and ECM (red line) and the observed price (black circles). Note that
the size of quarterly changes in real oil prices (black line) increase after OPEC takes control over real oil prices during Regime 2. Note that the model simulates
observed prices fairly well during Regime 5 without separate regimes for the 1986 price collapse, the Asian financial crisis and the first Persian Gulf War.
Table 2 | Nine regimes identified by Model 1
Regime
Start
End
Cause
Regime 1
1946:1
1949:3
Rationing and price suppression
Regime 2
1970:3
1974:1
OPEC gains control over
domestic resources
Regime 3
1974:1
1979:2
US price controls
Regime 4
1979:3
1980:4
Panic buying after the
Iranian Revolution
Regime 5
1981:1
2004:2
OPEC fair share of the market
Regime 6
2004:3
2007:3
Reallocation of quotas for
OPEC members
Regime 7
2007:4
2008:3
Speculative bubble
Regime 8
2010:1
2011:1
Speculative bubble
Regime 9
2011:2
2014:3
Arab Spring
Regimes start in the quarter when a step changes the equilibrium price for two or more
quarters. Regimes end when the next step either moves the equilibrium price back towards or
further away from the equilibrium that is implied by market fundamentals. The cause for
the regime is based on real-world events. Explanations for regimes prior to 2000 borrow
heavily from a chronology of the world oil market7; after 2000, explanations rely heavily on
articles in the industry newsletter Petroleum Intelligence Weekly. We recognize that associating
real-world events to regimes represents post hoc explanations. This cannot be avoided because
there is no a priori definition for a price regime and it is very difficult to quantify time series
that could represent such regimes.
Nature Energy | VOL 5 | February 2020 | 141–149 | www.nature.com/natureenergy
143

Articles
NATUrE EnErgy
enhance cooperation among US oil firms. But one form of coopera-
tion is forbidden: “In February 1948, the attorney general explicitly
excluded from antitrust prosecution cooperation among oil compa-
nies so long as this cooperation attempted to alleviate shortages by
any mechanism other than raising price.” (ref. 8).
Regime 1 ends in 1949:3 when the second step increases prices
by US$7.42, which moves the equilibrium price back to the level
implied by market fundamentals. We attribute the end of Regime 1
to higher production and lower consumption, which relax the need
for market interventions. Informal rationing ends when the 1949
recession reduces consumption. Its effect on the supply/demand
balance is reinforced by greater US output and the 1950 opening of
the Trans-Arabian Pipeline.
Regime 2, 1970:3–1974:1, often is associated with a series of
OPEC-induced price rises4. Here, Regime 2 represents an US$11.03
price reduction. We reconcile this seeming contradiction by arguing
that Regime 2 is the period when the marginal supply of crude oil
shifts geographically from the TRC (US) to OPEC, but this change
in market fundamentals does not affect price because production
agreements allow international oil companies to set production
from fields in OPEC nations. Regime 2 ends when OPEC nations
gain control over marginal supply and demonstrate this control by
cutting production and imposing an embargo on the US in 1973:4.
Before the start of Regime 2, UtilTRC rises rapidly and reaches
100% in the middle of Regime 2, 1972:2 (Fig. 2). The loss of spare
capacity means that Texas no longer produces the marginal barrel of
oil, it comes from fields in OPEC nations. But this increase in UtilTRC
does not increase price as indicated by the cointegrating relation
because concession agreements allow international oil companies
to expand production in low-cost OPEC nations. Concession agree-
ments specify the terms for international oil companies to pay host
countries for the oil produced, but allow international oil companies
to control production. Because the Seven Sisters control production,
there is no ‘market mechanism’ to raise the price when UtilTRC rises
to capacity and the marginal supply shifts from the TRC to OPEC.
But the ability of international oil companies to control produc-
tion from fields in OPEC nations ends during Regime 2. OPEC
nations nationalize oil holdings and/or replace concessions with
participation agreements7. These changes are not fully appreciated
by the market; their import is recognized by a select few who wield
little influence. In November 1970, James Placke, a petroleum offi-
cer in the US embassy in Tripoli, warns of potential consequences,
but there is little reaction7. Similarly, few heed a warning that the US
is losing control over the marginal supply of oil11. Instead, Regime 2
ends in 1974:1 when OPEC demonstrates control over its produc-
tion and hence marginal supply by cutting production and Saudi
Arabia stops selling oil to the US and a few other nations.
Regime 3, 1974:1–1979:2, starts when OPEC demonstrates con-
trol over marginal supply, which raises the price by US$9.84. But
this increase leaves prices slightly (US$1.190.51) below the price
implied by market fundamentals. Lower prices may be associated
with non-market responses to OPEC’s control over marginal supply.
US energy legislation in the 1970s is the largest peacetime inter-
ference with the economy, which has many (contradictory) effects,
including lower prices12,13. For example, divisions between ‘new’ and
‘old’ oil keep the wellhead price for much of US production (includ-
ing West Texas intermediate, WTI) below world prices.
Regime 4, 1979:3–1980:4, is associated with a precautionary
demand shock that raises the price by US$12.63 relative to that indi-
cated by market fundamentals. Prior to the start of Regime 4, revo-
lutionary chaos in Iran subsides and Iranian production of crude oil
rises from 0.73 mbd in February 1979 to 4.2 mbd in April 1979 (ref. 14).
Commensurate with this increase, Saudi Arabia cuts production by
about 2 mbd back to its official ceiling of 8.5 mbd. This rebalancing
“was the moment at which some kind of order, well short of disaster
might have been reestablished” (ref. 7).
But order is not re-established. Instead, panic buying pushes pur-
chases well beyond current consumption such that inventories rise
significantly7. Consistent with previous analyses15,16, we argue that
Regime 4 is a precautionary demand shock. During Regime 4, US oil
inventories rise from 15 to 23 days of forward consumption (Fig. 3).
The positive price effect of this inventory build is not consistent
with the negative relation that is implied by market fundamentals17.
Regime 5, 1981:1 (or 1981:3)–2004:2, starts with a US$18.58
price drop due to declines in both inventories and consumption18.
This drop pushes prices US$5.95 below the levels indicated by the
cointegrating relation (Fig. 1). Regime 5 includes the 1986 change in
OPEC strategy, when OPEC abandons its official price and instead
seeks a fair share of the market. But we argue that this change starts
before 1986, when OPEC’s official price simply formalizes changes
in refined product prices19,20. Note that Regime 5 includes several
large price changes, such as the 1986 price collapse, the Asian finan-
cial crisis and the first Persian Gulf War. But these events do not
change prices in a statistically significant fashion relative to that pre-
dicted by Regime 5 because they are largely captured by the proxies
for market fundamentals in the cointegrating relation (Fig. 1). For
example, the 13% price reduction during the Asian financial crisis is
associated with increases in Days and UtilRef.
Regime 6, 2004:3–2007:3, is defined by a US$12.77 price increase.
OPEC reallocates national quotas in a way that enhances its ability
to control output, which pushes the price US$6.82 above that indi-
cated by market fundamentals.
At its December 2004 meeting, OPEC suspends its US$22–29
price range and reallocates production quotas based on existing
rates of production as opposed to the existing quotas. This change
is described as a “pragmatic way of gradually leaving the old sys-
tem behind without an open squabble over quota redistributions”
(ref. 21). Although OPEC cannot enforce quotas, quotas influence
observed levels of production22,23. As such, harmonizing quotas
with operable capacity reduces the incentive to cheat and increases
OPEC’s ability to control marginal supply.
20
30
40
50
60
70
80
90
100
110
1938:1
1948:1
1958:1
1968:1
Year:quarter
1978:1
1988:1
1998:1
2008:1
Oil supply capacity utilized (%)
UtilOPEC
UtilTRC
Regime 1
Fig. 2 | OPEC gains control over marginal supply from the TRC.
The percentage of capacity allowed to operate by the Texas Railroad
Commission (UtilTRC, black line) and the percentage of capacity allowed
to operate by OPEC (UtilOPEC, red line). The two lines cross in 1973:1. As
UtilTRC approaches 100%, the TRC loses control over marginal supply.
After Regime 2, control over the marginal supply shifts to OPEC because
UtilOPEC is well below 100%. The timing of Regime 2 (a US$11.03
reduction in price) is indicated by the area shaded blue.
Nature Energy | VOL 5 | February 2020 | 141–149 | www.nature.com/natureenergy
144

Articles
NATUrE EnErgy
Regime 7, 2007:4–2008:3, raises the price by US$16.57 relative to
Regime 6. We postulate that this increase is a speculative bubble, in
which noise investors raise crude oil prices relative to the equilib-
rium that is implied by the proxies for market fundamentals in the
cointegrating relation. A speculative bubble is consistent with the
explanations offered by industry and academia. One quarter into
Regime 7, Rex Tillerson, then CEO of Exxon Mobil states “I cannot
explain why we have US$70 oil. The fundamentals behind supply
and demand do not support US$70 oil. The fundamentals support
something much less.” (ref. 24). Academic researchers recognize the
potential importance of a speculative bubble: the peer-review lit-
erature contains seven articles that mention ‘oil’, ‘prices’ and ‘specu-
lation’ before the end of Regime 7. As of February 26, 2019, 173
articles appear.
The role of a speculative bubble can be tested indirectly by exam-
ining the relation between Regime 7 (and Regime 8, see below) and
the discussions of a speculative bubble by industry experts. To eval-
uate this relation, we compile information on the number of articles
in the industry newsletter Petroleum Intelligence Weekly that include
the words ‘oil’, ‘price’ and ‘speculation’, with speculation referring to
oil prices as opposed to guessing about future events. If our hypoth-
esis about a speculative bubble is correct, Regime 7 (and Regime 8)
should be marked by an increase in the number of articles about
speculation. Consistent with the visual impression given by Fig. 4,
a logit model (Regime 7t ¼ α þ β*Articlet þ εt
I
) indicates that a
binary variable for Regime 7 is related to the number of articles that
mention speculation (^β ¼ 1:00; t ¼ 2:89; p<0:01
I
).
We recognize that a speculative bubble may be a catch-all for
the inability to describe the effect of market fundamentals (or some
other non-market force) on prices during Regime 7. But it is diffi-
cult to relate a speculative bubble to a concrete measure. Theoretical
models indicate that a speculative bubble moves prices away from
fundamentals by increasing the risk premium for arbitrageurs25.
Consistent with this mechanism, Regime 7 is associated with an
increase in the risk premia that is caused by and causes a complex
series of changes in long and short positions by both non-commer-
cial and commercial traders26. These complex sequences may be one
reason that simple measures of trader positions fail to validate the
importance of speculation1,2.
Regime 7 ends with the start of the financial crisis in 2008:4. The
financial crisis reduces interest rates and convenience yields, which
reduces returns to holding oil as a commodity relative to a financial
asset27. This changes the correlation between returns to crude oil
and the S&P 500 from negative to positive, which means that lower
equity prices at the start of the financial crisis correlate with lower
oil prices. Note that the end of Regime 7 returns the equilibrium
price to the level implied by market fundamentals.
Regime 8, 2010:1–2011:1, raises the equilibrium price by
US$10.90 relative to that implied by market fundamentals. Although
this period coincides with cuts in production by OPEC, their effects
on market fundamentals are offset by increased production and
slowed demand such that inventories rise28. As such, reduced pro-
duction by OPEC probably cannot account for Regime 8. Instead,
we argue that Regime 8 also is associated with a speculative bubble.
Before Regime 8 starts, the divergence between market funda-
mentals and prices is highlighted by price forecasts that consistently
under-predict observed increases. The Center for Global Energy
Studies warns that nominal prices are unlikely to rise above US$40
per barrel29 and market analysts forecast nominal prices to average
US$50 during 2009 (US$42 to US$57)30. By the first quarter of 2010,
the price of WTI rises beyond US$75 and remains there for the rest
of the year.
When asked about these price increases, the Secretary General
of OPEC, Abdullah al-Badri, states, “If there is a problem with fun-
damentals then we would be concerned, but if the fundamentals are
okay and there is enough oil in the market and the prices shoots to
US$147 as what happened in 2008, this is not our problem. This is
a speculation problem.” (ref. 31). Consistent with this interpretation,
Fig. 4 shows a second rise in articles that mention ‘speculation’, but
the relation between a binary variable for Regime 8 and the number
Days of forward consumption
0
5
10
15
20
25
30
1975:1
1976:1
1977:1
1978:1
1979:1
1980:1
1981:1
1982:1
1983:1
1984:1
1985:1
Regime 4
Year:quarter
Fig. 3 | Precautionary demand and Regime 4. Uncertainty about the
ability to purchase oil induces firms to increase their inventories, which is
indicated by a steady increase in days of forward consumption (Days) in US
inventories (black line) during Regime 4 (area shaded red). This increase
represents a precautionary demand shock that raises the equilibrium price
by US$12.63 relative to that simulated by market fundamentals. Inventories
decline to a more ‘normal level’ after Regime 4 because of fears about
supply diminish.
0
1
2
3
4
5
6
7
8
9
1998:1
1999:1
2000:1
2001:1
2002:1
2003:1
2004:1
2005:1
2006:1
2007:1
2008:1
2009:1
2010:1
2011:1
2012:1
2013:1
2014:1
2015:1
2016:1
2017:1
2018:1
Number of articles
Year:quarter
Fig. 4 | Discussions of a speculative bubble. The degree to which the
industry thinks that a speculative bubble inflates the price of oil, as
indicated by the number of articles in Petroleum Intelligence Weekly that
contain the words ‘oil’, ‘price’ and ‘speculation’, with speculation referring
to price speculation. The first set of red bars occur during Regime 7 and
the second set of red bars occur during Regime 8. A logit model indicates
that Regime 7 is related to the number of articles (P < 0.01), whereas the
relation with Regime 8 is significant at P = 0.06.
Nature Energy | VOL 5 | February 2020 | 141–149 | www.nature.com/natureenergy
145

Articles
NATUrE EnErgy
of articles that mention ‘speculation’ is significant only at P = 0.10
(^β ¼ 0:38; t ¼ 1:93; P<0:06
I
).
Regime 9, 2011:2–2014:3, raises the equilibrium price by
US$6.95 relative to Regime 8. We argue that Regime 9 is associated
with the Arab Spring, which refers to political changes in Tunisia,
Egypt and Libya, and worries about political stability throughout
the region. These political changes alter market fundamentals,
but they are not captured by proxies in the cointegrating rela-
tion. There is little change in UtilOPEC because political changes
in Libya reduce both production and capacity (as reported by the
Energy Information Administration, EIA). Nor are these political
changes captured by crude oil inventories (Days). In June 2011, the
International Energy Agency releases 60 million barrels from stra-
tegic reserves, including 30 million barrels from the US Strategic
Petroleum Reserve (SPR)32. But the SPR is not part of Days, and so
this release is not represented in the cointegrating relation.
Regime 9 ends (2014:4) with the 2014 price collapse, which is
triggered by increasing production from tight formations in the
US, which rises 3.6 mbd between April 2011 and November 2014.
This forces OPEC to choose between defending price by cutting
production to accommodate this additional supply or defending its
share of the market by letting the increased production of tight oil
reduce prices33–35. By choosing not to cut production, OPEC hopes
that lower prices will reduce the profitability of producing oil from
tight formations, which would lower production. But this strategy is
not successful. After a brief decline, tight oil production resumes its
increase because technical innovations increase the productivity of
rigs drilled into tight formations and the break-even price (~US$50
nominal) is lower than previously thought and is consistent with the
prices implied by the market fundamentals that prevail after Regime
9 (ref. 36).
The presence of a low-cost alternative prompts discussion that
the United States now controls marginal supply37. But tight oil can-
not be considered marginal supply because there is no spare capac-
ity38. Instead, tight oil may act in the medium term to impose some
discipline on prices. As such, it may discourage OPEC actions to
create the next ‘oil price regime.’ And this may be one reason that
the end of Regime 9 returns oil prices back to the level implied by
market fundamentals, where they remain through to the end of the
sample period 2018:3.
Conclusion
We associate two regimes (Regimes 7 and 8) with a speculative
bubble, in which prices move away from the level implied by the
basic, underlying factors that affect the supply or demand for a com-
modity. A speculative bubble is different from speculative trading,
which is an integral part of price discovery in spot and future mar-
kets39–41. Speculative trading is critical because it allows commer-
cial participants to hedge, which transfers risk to non-commercial
participants42. This transfer increases the efficiency of commercial
operations.
But if spot and futures markets do not operate properly, a specu-
lative bubble can impose significant costs. During Regimes 7 and 8,
speculative bubbles raise prices by US$14.31 and US$4.65 per bar-
rel, respectively (Table 3 and Supplementary Note 3). These price
increases transfer US$42.9 billion from US consumers to US oil
producers and transfer US$87.4 billion from the US economy to oil
exporting nations. These totals suggest transfers that reduce total
social welfare. Such welfare losses suggest that regulations, which
prevent speculative bubbles, can avoid significant costs.
Methods
Data. In this study we compile quarterly observations for oil prices and proxies
for the balance between oil supply and demand from 1938:1 through 2018:3,
which is the entire period for which observations are available. This long sample
period poses two challenges. First, many time series are not available over the
entire sample period. Instead, we use observations for the data available, even
when they are not the best choice. For example, we proxy market fundamentals
using observations for US oil inventories and utilization rates for US refineries. We
prefer observations for global (or the Organisation for Economic Co-operation
and Development, OECD) inventories and global rates of refinery utilization, but
observations are limited; observations for OECD inventories start in the 1980s.
For some time series, the frequency of observations changes during the
sample period. For these series, we use a simple algorithm to interpolate annual
Table 3 | The effect of speculative bubbles on the price of crude oil and the monies transferred between consumers and producers
Model 1
Model 2
Model 3
Model 4
Model 5
Model 6
Model 7
Model 8
Model 9
∆Price (US$1982 per barrel)
 Regime 7
14.31
13.21
13.24
10.39
13.05
13.35
13.04
13.34
13.40
 Long-run
16.57
17.04
16.94
16.94
17.20
17.09
17.34
17.51
17.40
 Regime 8
4.65
5.08
5.05
5.16
6.33
5.04
5.01
6.62
6.08
 Long-run
10.90
11.55
11.44
11.72
12.01
11.21
11.65
12.09
11.25
Domestic transfers (Billion US$1982)
 Regime 7
29.05
26.83
26.89
21.12
26.52
27.11
26.49
27.09
27.21
 Regime 8
13.80
15.05
14.96
15.28
19.59
14.91
14.82
19.29
23.82
 Total
42.85
41.88
41.85
36.40
46.11
42.01
41.31
46.37
51.03
Foreign transfers (Billion US$1982)
 Regime 7
64.77
59.78
59.90
47.03
59.07
60.39
59.00
60.35
60.63
 Regime 8
22.66
24.79
24.65
25.20
33.10
24.57
24.41
32.63
23.82
 Total
87.43
84.57
84.55
72.22
92.18
84.96
83.41
92.98
84.45
Total transfer (Billion US$1982)
 Regime 7
93.83
86.61
86.79
68.15
85.59
87.50
85.49
87.44
87.85
 Regime 8
36.45
39.84
39.61
40.48
52.69
39.47
39.23
51.92
47.63
 Total
130.28
126.45
126.39
108.63
138.28
126.97
124.72
139.35
135.48
The first panel reports the increases in the equilibrium and predicted price during Regimes 7 and 8. The second panel reports the total funds transferred from US consumers to US producers due to the price
increases listed in the first panel. The third panel reports the total funds transferred from the US economy to foreign producers due to the price increases listed in the first panel. The fourth panel reports
the sum of funds transferred from consumers to producers as reported in the second and third panels. These totals suggest transfers that reduce total social welfare. Such welfare losses suggest that
regulations, which prevent speculative bubbles, can avoid significant costs.
Nature Energy | VOL 5 | February 2020 | 141–149 | www.nature.com/natureenergy
146

Articles
NATUrE EnErgy
observations into quarterly observations. To interpolate annual observations for
US oil inventories before 1972 into quarterly observations, we create an adjustment
factor (QAdj) for each quarter q (q = 1, 2, 3, 4) as follows:
QAdjq ¼
PEnd
Y¼Start
XY;q
XY
End  Start
ð1Þ
in which X is the observed value for one of the indepdent variables X, in year Y for
quarter q over the sample period for which quarterly observations are available.
These adjustment factors are used to convert annual values XY
I
, which represent the
average for year Y, into quarterly values (~XY;q
I
) as follows:
~XY;q ¼ QAdjq*XY
ð2Þ
Nominal oil prices are measured using quarterly observations for West Texas
intermediate (WTI) (http://www.globalfinancialdata.com)43. We recognize that
prices are specific to the quality of the crude oil44, that prices between 1973 and
1981 are affected by Federal efforts to control the price of old and new oil produced
in the US13, and that WTI becomes a weaker benchmark after 2011 such that the
EIA abandons WTI as its reference oil price in the 2013 Annual Energy Outlook45.
Nonetheless, this is the only measure for the price of crude oil that is available for
the entire sample period. Nominal prices are deflated (base year 1982) by the US
city average for all items (CUUR0000SA0), which is obtained from the Bureau of
Labor Statistics46 to calculate the real price (Price).
Without regimes, real prices for crude oil are determined by the balance
between supply and demand. We proxy this balance with capacity utilization
by producer organizations that control marginal supply, inventories of crude oil
and rates of refinery utilization. We specify capacity utilization by two producer
organizations: the TRC and OPEC. We expect prices to have a positive albeit non-
linear relation with capacity utilization47–49. Higher rates of capacity utilization by
the marginal supplier signal less spare capacity. This suggests that consumption
is high relative to supply, supply disruptions cannot be offset by increasing
production elsewhere and there is relatively little opportunity for individual
producers to cheat on production agreements. Under these conditions, prices
tend to rise. Conversely, low rates of capacity utilization by the marginal supplier
indicate higher amounts of spare capacity. This suggests that consumption is
low relative to supply, supply disruptions can be accommodated by increased
production elsewhere and there is greater opportunity for individual producers to
cheat on production agreements. Under these conditions, prices tend to decline.
The TRC’s control over the marginal supply of crude oil is conferred by
the Connelly Hot Oil Act of 1935, which gives the TRC and other US state
commissions the power to open and close existing capacity with the goal of
stabilizing price50. To stabilize price, the TRC determines the rate of output that
would balance supply and demand. This rate is enforced by controlling the number
of days per month that well owners are allowed to pump oil, which is known as
the market demand factor. We divide this market demand factor by the number of
days in a month to calculate the percentage of Texas capacity that the TRC allows
to operate (UtilTRC), which varies between 26 and 101% (Fig. 2). Monthly data
from January 1938 through December 1986 are obtained from the TRC. Monthly
observations of 100% from March 1977 through December 1986 are extended
through September 2018, which indicates that producers are free to choose the rate
at which they operate.
This loss of spare capacity gives OPEC control over the marginal supply of
crude oil. We calculate capacity utilization by OPEC (UtilOPEC) as the quotient
of OPEC production of crude oil18 and OPEC’s operable capacity51 (Fig. 2). For the
period before OPEC is founded in September 1960, we extrapolate the value from
1960:1, 97.1%.
We also proxy the balance between supply and demand with inventories
of crude oil. Inventories rise when demand falls relative to supply, which puts
downward pressure on price17. Conversely, prices rise when inventories fall, which
occurs when demand rises relative to supply. This balance between demand and
supply is measured by days of forward consumption (Days), which is calculated
by dividing barrels of crude oil in US inventories (excluding SPR)52,53 by the US
consumption of refined petroleum products.
Finally, we proxy the balance between supply and demand with the utilization
rates for US refiners (UtilRef), which are calculated by dividing US refinery runs
by US refinery capacity54,55 and multiplying by 100. UtilRef is expected to have
a negative relation with price49. As utilization rates near capacity, there is less
demand for new supply, which reduces price.
To eliminate the effects of inverting matrices with elements that differ greatly
in size due to different units of measure, each of the time series described above is
standardized as follows:
xt ¼
yt  y
ð
Þ
ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ
VarðyÞ
p
ð3Þ
in which yt is the value (in original units), y
I
 is the average value over the sample
period and Var(y) is the variance over the sample period. We recognize that non-
stationary time series do not have a constant mean or variance, rather the sample
mean and variances are used in a linear transformation, given by equation (3), to
harmonize values across time series.
We recognize that misreporting data could contribute to regimes. If OPEC
under-reports oil production, this could affect the relation between market
fundamentals (as reported by OPEC) and oil prices. But this effect could go beyond
data artefacts. If buyers believe that OPEC is producing less oil (as indicated by
its announcement) than it actually does, buyers may act accordingly and thereby
boost prices. Nonetheless, we do not believe that our regimes reflect poor data
because observations for many variables are compiled from primary sources and
not official sources, such as OPEC announcements.
Econometric methodology. The first step of the econometric methodology
identifies a cointegrating relation for the real price of oil from an unrestricted
model, which is given by:
Pricet ¼ f UtilTRCt; UtilTRC2
t ; UtilTRC3
t ; UtilOPECt; UtilOPEC2
t ;

UtilOPEC3
t ; Dayst; UtilRef t

ð4Þ
for which the variables have been defined previously.
We identify a final version of equation (4) by estimating versions that specify all
possible combinations of utilization rates by the TRC and OPEC (Supplementary
Note 1) and evaluating each model against three criteria: do the variables
cointegrate, do the variables have a statistically measurable relation with oil prices
and do oil prices error correct to disequilibrium in the cointegrating relation?
We evaluate cointegration by estimating each possible specification using OLS
and testing the residual for a unit root with four statistics (PT, DFGLS, QT and
DFGLSu)56. Statistics that reject the null hypothesis indicate that the residual does
not contain a unit root, which indicates that the variables cointegrate and therefore
represent a long-run cointegrating relation for price.
The long-run relation between Price and the proxies for the balance
between supply and demand are estimated using DOLS57. Lags and leads for
the first differences of the independent variables are chosen using the Bayesian
information criterion58. We test the null hypothesis that independent variable
i has no statistically measurable relation with Price (βi = 0) with a t-test that is
calculated with a standard error, which is robust to the presence of autocorrelation
and heteroscedasticity in the regression residual59. Rejecting the null hypothesis
indicates that independent variable i has a statistically measurable relation
with Price.
We test whether Price adjusts to disequilibrium in the cointegrating
relation by estimating an ECM. Disequilibrium (μ) is calculated by subtracting
the equilibrium price implied by the cointegrating relation from observed
price (^μt ¼ Pricet  ^Pequilt
I
). Disequilibrium from the cointegrating
relation estimated by the saturation indicator technique is calculated as
μt ¼ Price
d
^Pequilt þ P
t
i¼1
Stept
I
 in which Step is a step estimated by the indicator
saturation technique.
Disequilibrium is specified in an ECM as follows:
ΔPricet ¼ α þ ρ^μt1 þ
X
s
i¼1
γt ΔPricet1 þ
X
n
j¼1
X
s
i¼1
θj;t ΔXj;t1 þ εt
ð5Þ
in which X is a vector of n independent variables in the cointegrating relation, S is
chosen using the Akaike information criterion60, Δ is the first difference operator
(xt–xt−1), α, ρ, γ and θ are regression coefficients estimated using OLS, and ε is the
regression residual. Price adjusts towards the equilibrium that is implied by the
cointegrating relation if the error correction mechanism −1 ≤ ρ < 0.
For models in which the proxies for market fundamentals cointegrate with
price, have a statistically measurable relation with price and oil prices error correct
to disequilibrium in the cointegrating relation, we identify price regimes using an
indicator saturation technique that is implemented in the R package gets (refs. 61,62).
We ‘fix’ the independent variables used in the DOLS estimate (along with the lags
and leads of the first differences), use a P = 0.01 significance level and allow gets to
choose from a full set of impulses and/or steps. Impulses are a one-quarter change
in the equilibrium price for crude oil relative to that simulated by the proxies for
market fundamentals in the cointegrating relation, while steps are changes that
persist for two or more consecutive quarters. Impulses and steps are evaluated
iteratively for every possible quarter. The method used to calculate the statistical
significance of impulses and steps is summarized in Supplementary Note 2.
Short-run price dynamics are quantified by an ECM which is given by:
ΔPricet ¼ αi þ ρμi;t1 þ P
s
j¼1
ΨjΔPricetj þ P
s
j¼1
ξjΔDaystj
þ P
s
j¼1
γjΔUtilRef tj
P
s
j¼1
θjΔUtilTRCti þ P
s
j¼1
λjΔUtilOPECtj
þ P
k
R¼1
P
s
j¼0
ψR;jΔStepR;tj þ υi;t
ð6Þ
in which Step is each of the k steps in the cointegrating relation as defined above.
The number of lags (s) is chosen using the Akaike information criterion60 and the
standard errors are robust to the presence of autocorrelation and heteroscedasticity
in the regression residual59.
Nature Energy | VOL 5 | February 2020 | 141–149 | www.nature.com/natureenergy
147

Articles
NATUrE EnErgy
Data availability
Data on US petroleum stocks, US refinery utilization rates, prices for WTI (earlier
observations from the Federal Reserve Bank of St. Louis43), OPEC production
and OPEC capacity are available from US EIA (ref. 32), data for the US city average
for all items are available from the US Bureau of Labor Statistics46 and the TRC
demand factors are available from the Texas Railroad Commission. These data and
the computer code can be obtained from authors upon reasonable request. Source
data for Figs. 1–4 are provided with this paper.
Code availability
The code can be obtained from the authors upon reasonable request.
Received: 20 May 2019; Accepted: 3 January 2020;
Published online: 17 February 2020
References
	1.	 Boyd, N. E., Harris, J. H. & Li, B. An update on speculation and
financialization in commodity markets. J. Commod. Mark. 10, 91–104 (2018).
	2.	 Haase, M., Zimmerman, Y. S. & Zimmerman, H. The impact of speculation
on commodity futures markets – A review of the finding of 100 empirical
studies. J. Commod. Mark. 2, 3–15 (2016).
	3.	 Maugeri, L. The Age of Oil: The Mythology, History and Future of the World’s
Most Controversial Resource (Praeger, 2006).
	4.	 Noguera, J. Oil prices: breaks and trends. Energy Econ. 37, 60–67 (2013).
	5.	 Noguera-Santaella, J. Geopolitics and the oil price. Econ. Model. 52,
301–309 (2016).
	6.	 Noguera, J. The Seven Sisters versus OPEC: solving the mystery of the
petroleum market structure. Energy Econ. 37, 60–67 (2017).
	7.	 Yergin, D. The Prize: The Epic Quest for Oil, Money, & Power (Simon &
Schuster, 1991).
	8.	 Hamilton, J. D. Oil and the macroeconomy since World War II. J. Polit. Econ.
91, 228–248 (1983).
	9.	 Hamilton, J. D. Historical causes of postwar oil shocks and recessions. Energy
J. 6, 97–116 (1985).
	10.	Hamilton, J. D. in Routledge Handbook of Major Events in Economic History
(eds Parker, R. E. & Whaples, R.) 239–265 (Routledge Taylor and Francis
Group, 2013).
	11.	Akins, J. E. The oil crisis: this time the wolf is here. Foreign Aff. 51,
463–490 (1973).
	12.	Murphy, R. P. Removing the 1970s crude oil price controls: lessons for
free-market reform. J. Priv. Enterp. 33, 63–78 (2018).
	13.	Erickson, E. W. et al. The political economy of crude oil price controls.
Nat. Resour. J. 18, 787–801 (1978).
	14.	Total Energy: Iranian Crude Oil Production (US Energy Information
Administration); https://www.eia.gov/totalenergy/data/browser/index.php?tbl
=T11.01A#/?f=M&start=197812&end=201901&charted=13
	15.	Kilian, L. Not all oil price shocks are alike: disentangling demand and supply
shocks in the crude oil market. Am. Econ. Rev. 99, 1053–1069 (2009).
	16.	Alquist, R. & Kilian, L. What do we learn from the price of crude oil futures?
J. Appl. Econ. 25, 539–573 (2010).
	17.	Pindyck, R. The dynamics of commodity spot and futures markets: a primer.
Energy J. 22, 1–29 (2001).
	18.	Total Energy: Petroleum Products Supplied by Type (US Energy Information
Administration); https://www.eia.gov/totalenergy/data/browser/index.php?tbl
=T03.05#/?f=M&start=197301&end=201806&charted=9-13
	19.	Lowinger, T. & Ram, R. Product value as a determinant of OPEC’s official
crude oil prices: additional evidence. Rev. Econ. Stat. 66, 691–695 (1984).
	20.	Verleger, P. K. The determinants of official OPEC crude prices. Rev. Econ.
Stat. 64, 177–182 (1982).
	21.	OPEC Cairo Cuts May Solve Quota Quandry (Petroleum Intelligence Weekly,
2004).
	22.	Parnes, D. Heterogeneous noncompliance with OPEC’s oil production cuts.
Energy Econ. 78, 289–300 (2019).
	23.	Kaufmann, R. K., Bradford, A., Belanger, L. H., Mclaughlin, J. P. & Miki, Y.
Determinants of OPEC production: implications for OPEC behavior. Energy
Econ. 30, 333–351 (2008).
	24.	Fundamentals don’t justify $70 oil price – Exxon CEO. Reuters; https://
uk.reuters.com/article/exxon-oil-price/update-2-fundamentals-dont-justify-
70-oil-price-exxon-ceo-idUKN0720445120070907
	25.	De Long, J. B., Shleifer, A., Summers, L. H. & Waldmann, R. J. Noise trader
risk in financial markets. J. Polit. Econ. 98, 703–738 (1990).
	26.	Tokic, D. Rational destabilizing speculation, positive feedback trading, and
the oil bubble of 2008. Energy Policy 39, 2051–2061 (2011).
	27.	Kolodziej, M., Kaufmann, R. K., Kulatilaka, N., Bicchetti, D. & Maystre, N.
Crude oil: commodity or financial asset? Energy Econ. 46, 216–223 (2014).
	28.	UNITED STATES (Petroleum Intelligence Weekly, 2009).
	29.	OPEC Will Have Hard Time Eeviving Prices, CGES Warns (Petroleum
Intelligence Weekly, 2009).
	30.	Pundits See Oil Price Around $50 This Year (Petroleum Intelligence Weekly,
2009).
	31.	Factbox: OPEC ministerial comments at Quito meeting. Reuters https://www.
reuters.com/article/us-opec-comments/factbox-opec-ministerial-comments-
at-quito-meeting-idUSTRE6BA1UA20101211 (2010).
	32.	International Energy Agency members release strategic petroleum stocks.
Today in Energy (US Energy Information Administration, accessed 30
November 2019); https://www.eia.gov/todayinenergy/detail.php?id=1950
	33.	Manescu, C. B. & Nuño, G. Quantitative effects of the shale oil revoluton.
Energy Policy 86, 855–866 (2015).
	34.	Ansari, D. OPEC, Saudi Arabia, and the shale revolution: insights from
equilibrium modelling and oil politics. Energy Policy 111, 166–178 (2017).
	35.	Behar, A. & Ritz, R. A. OPEC vs US shale: analyzing the shift to a market-
share strategy. Energy Econ. 63, 185–198 (2017).
	36.	Ansari, E. & Kaufmann, R. K. The effect of oil and gas price and price
volatility on rig activity in tight formations and OPEC strategy. Nat. Energy 4,
321–328 (2019).
	37.	Sheikhs v shale - The new economics of oil. The Economist https://www.
economist.com/leaders/2014/12/04/sheikhs-v-shale (2014).
	38.	Kleinberg, R. L., Paltsev, S., Ebinger, C. K. E., Hobbs, D. A. & Boersma, T.
Tight oil market dynamics: Benchmarks, breakeven points, and inelasticities.
Energy Econ. 70, 70–83 (2018).
	39.	Shrestha, K. Price discovery in energy markets. Energy Econ. 45,
229–233 (2014).
	40.	Silvério, R. & Szklo, A. The effect of the financial sector on the evolution
of oil prices: analysis of the contribution of the futures market to
the price discovery process in the WTI spot market. Energy Econ. 34,
1799–1808 (2012).
	41.	Kaufmann, R. K. & Ullman, B. Oil prices, speculation, and fundamentals:
interpreting causal relations among spot and futures prices. Energy Econ. 31,
550–558 (2009).
	42.	Ederington, L. & Lee, J. H. Who trades futures and how: evidence from the
heating oil futures market. J. Bus. 75, 353–373 (2002).
	43.	WTI - Economic Data Series (FRED, St. Louis Fed, accessed January 2019);
https://fred.stlouisfed.org/tags/series?t=wti;http://www.imf.org/external/np/
res/commod/index.aspx
	44.	Kaufmann, R. K. Price differences among crude oils: the private costs of
supply disruptions. Energy Econ. 56, 1–8 (2016).
	45.	Scheitrum, D. P. & Carter, C. A. WTI and Brent future pricing structure.
Energy Econ. 72, 462–469 (2018).
	46.	Bureau of Labor Statistics Data (US Bureau of Labour Statistics, accessed
January 2019); https://data.bls.gov/cgi-bin/surveymost
	47.	Kaufmann, R. K. A model of the world oil market for Project LINK
integrating economics, geology and politics. Econ. Model. 12, 165–178 (1995).
	48.	Kaufmann, R. K., Dees, S., Karadeloglou, P., Sánchez, M. & Sanchez, M. Does
OPEC matter? An econometric analysis of oil prices. Energy J. 25,
67–90 (2004).
	49.	Kaufmann, R. K., Dees, S., Gasteuil, A. & Mann, M. The role of refinery
utilization futures markets, and non-linearities. Energy Econ. 30, 2609–2622
(2008).
	50.	Prindle, D. F. Petroleum Politics and the Texas Railroad Commission (Univ. of
Texas Press, 1981).
	51.	3c. OPEC Crude Oil (excluding condensates) Supply. STEO Data Browser (US
Energy Information Administration); https://www.eia.gov/outlooks/steo/data/
browser/#/?v=7&f=M&s=0&start=201401&end=201912&linechart=CO
PR_AG~COPC_OPEC&maptype=0&ctype=linechart&map
	52.	Petroleum Trade: Exports by Country Destination (US Energy Information
Administration); https://www.eia.gov/totalenergy/data/monthly/pdf/sec3_13.
pdf
	53.	Petroleum Facts and Figures (American Petroleum Institute, 1971).
	54.	American Petroleum Institute in Basic Petroleum Data Book Section VIII
(American Petroleum Institute, 1981).
	55.	U.S. Refinery Utilization and Capacity (US Energy Information
Administration); https://www.eia.gov/dnav/pet/pet_pnp_unc_dcu_nus_a.htm
	56.	Eliott, G., Rothenberg, T. J. & Stock, J. H. Efficient tests for an autoregressive
unit root. Econometrica 64, 813–836 (1996).
	57.	Stock, J. H. & Watson, M. W. A simple estimator of cointegrating vectors in
higher order integrated systems. Econometrica 61, 783–820 (1993).
	58.	Schwarz, G. Estimating the dimension of a model. Annu. Stat. 6,
461–464 (1978).
	59.	Newey, W. K. & West, K. D. A simple positive semi-definite heteroskedasticity
and autocorrelation consistent covariance matrix. Econometrica 55,
703–708 (1987).
	60.	Akaike, H. in 2nd International Symposium on Information Theory (eds
Petrov, P. N. & Csaki, F.) 267–281 (Akadémiai Kiado, 1973).
	61.	General-to-Specific (GETS) Modelling and Indicator Saturation Methods
v0.21 (CRAN, 2019); https://cran.r-project.org/web/packages/gets/index.html
	62.	Castle, J., Doornik, J., Hendry, D. & Pretis, F. Detecting location shifts during
model selection by step-indicator saturation. Econometrics 3, 240–264 (2015).
Nature Energy | VOL 5 | February 2020 | 141–149 | www.nature.com/natureenergy
148

Articles
NATUrE EnErgy
Acknowledgements
We thank F. Pretis for his help with the R package gets. We also thank C. A. S. Hall, J.
Noguera, the members of Project LINK and members of the Economics Department at
the University of Victoria for comments on preliminary versions of this work. Any errors
that remain are our responsibility.
Author contributions
This project was conceived by R.K. and modified by C.C. C.C. took the lead in compiling
the data used in the analysis reported in the main text; R.K. compiled the data used in
the analysis reported in the Supplementary Information. C.C. took the lead in estimating
the statistical models in R; R.K. adapted this code for the analyses in the Supplementary
Information. R.K. took the lead in writing the manuscript and designing the figures, with
significant input from C.C.
Competing interests
The authors have no financial or non-financial interests associated with the material in
this manuscript.
Additional information
Supplementary information is available for this paper at https://doi.org/10.1038/
s41560-020-0549-1.
Correspondence and requests for materials should be addressed to R.K.K.
Reprints and permissions information is available at www.nature.com/reprints.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.
© The Author(s), under exclusive licence to Springer Nature Limited 2020
Nature Energy | VOL 5 | February 2020 | 141–149 | www.nature.com/natureenergy
149

    Scientific Research Findings:

Answer: Proxies for supply and demand accurately account for oil prices in 138 of 313 quarters from January 1938 to March 2018. In the other 184 quarters, reduced accuracy creates nine regimes when oil prices deviate from the level implied by supply and demand. Of these nine regimes, two are associated with policy interventions to suppress price increases. The other seven are associated with non-market forces, such as strategic changes by OPEC, panic buying or speculative bubbles. This historical analysis of the oil market indicates that factors other than supply and demand have a large effect on prices for extended periods (with the shortest regime being a year and the longest being more than two decades). By summing the additional price paid for oil consumed in the US, we show that in recent regimes, speculative bubbles transferred US$42.9 billion from US consumers to US oil producers and transferred US$87.4 billion from the US economy to oil exporting nations. This suggests significant economic losses.