New U.K. Department of Energy and Climate Change Report: How NOT to Use Data to Make Policy

According to Ed Davey, Energy and Climate Change Secretary, “In the short term, biomass can help us decarbonize our electricity supplies, and we are committed to supporting cost-effective, sustainably produced biomass. This calculator shows that, done well, biomass can offer real carbon savings—which is why we are tightening our rules for sustainable biomass. Any producer who doesn’t meet those standards will lose financial support from next year.”

Part of the evidence that DECC used to make these determinations was extracted from Forest2Market’s blog. The report refers to three blog posts, in particular:

1. Export Wood Pellet Facilities’ Raw Material Delivered Cost Trends—US South (a blog based on Forest2Market’s Delivered Raw Material Cost Benchmark, cited as Forest2Market, 2013 in the report)
2. Southwide Stumpage Prices—September/October 2013 (a blog based on Forest2Market’s Timber Owner Market Guide, cited as Forest2Market, 2013a in the report)
3. Demand for Pulpwood in the US South: Historical and Future (a blog based on Forest2Market’s Five-Year US South Pulpwood Forecast, in the bibliography of the report as Forest2Market, 2014)

Since the UK is developing standards for determining what qualifies as “cost-effective, sustainably produced biomass,” based in part on evidence gleaned from these blog posts, Forest2Market would like to point out that the way in which DECC presents information from these posts is—frankly—misleading.

Let’s just look at one example from pages 11 and 38: “in the South USA, where many new pellet facilities that use pulpwood as a feedstock are being established, it has been reported that the demand for pine pulpwood from OSB and pellet manufacture increased between 2012 and 2013, contributing to an increase in the stumpage price of pine pulpwood (Forest2Market, 2013), which increased by 22% between the September/October periods of 2012 and 2013 (Forest2Market, 2013a).”Let me count the ways in which this statement, which conjoins two blog posts, #1 and #2 above, is misleading.

• The report fabricates a connection between the two blog posts that does not exist. The conjoining of these posts implies that the first topic (OSB and pellet demand) is the cause of the second (a 22% increase in pine pulpwood prices).
• If the report had mentioned the increase in stumpage price in the same blog post that connects OSB and pellet demand and price (#1, Forest2Market, 2013), it would instead have said that the increase in stumpage prices in the same period was 10%--not 22%. Instead, DECC chose the larger number—from a different blog post that covers a different time frame—instead of using the number that is reported in the same paragraph of the same blog post.
• The report implies that there is a large increase in demand for pine pulpwood from pellet manufacturers. Because the demand figure is not quantified, readers are encouraged to assume that the increase in demand from pellet and OSB manufacturers is contributing significantly to the price increase, since price is the only thing that is quantified (22%). Again, however, DECC appears to be selectively choosing what data to present. One of these posts (#3, Forest2Market, 2014) actually quantifies the annual increase in demand for pine pulpwood from 2012 to 2013. This blog post shows that demand from pellet manufacturers increased just 2%, a total of 80,000 short green tons during this period. Of the total increase in demand from 2012 to 2013 from OSB and pellets, less than 8% of that increase can be attributed to pellets. In fact, when you look at all pine pulpwood consumption in this period, demand increased by just 1.4 percent, or 1.7 million short green tons. Of this total, pellet demand accounted for less than 5% of the increase. OSB demand accounted for more than 55%, and pulp & paper accounted for 40%.
• If DECC had used the data available in all three blog posts, they could easily have accurately and fairly quantified the increase in demand just from pellet manufacture from 2012 to 2013, and if they were interested in the actual price increase that occurred over that same period (NOT the 22% was the change in the price of pine pulpwood from September/October 2012 to September/October 2013 of 22%, a comparison of one quarter’s price to another), a five-minute phone call would have sufficed. If we had been asked to pull the data so that DECC could use an apples-to-apples comparison, the resulting report would have revealed an 11 percent increase in pine pulpwood prices in the South from 2012 to 2013. If we assume that all of the increase in price was a result of increased demand, the increase in demand from pellets would represent just 5% of the price increase for the year. (It should be noted and taken seriously, however, that demand is just one of the causes of price change, so the actual effect is probably much lower.  This blog post outlines four more causes.)

This kind of selective use of the facts reported in the three Forest2Market posts cited in the report's bibliography gives the appearance that DECC is choosing only that data which is consistent with a particular perspective on the use of North American biomass. Based on the range of inaccuracies that this selective presentation of our blog content produces, DECC is likely to make a bad decision if it intends to use this particular interpretation of our data to make a decision about “cost-effective, sustainably produced biomass.”

And the ramifications of this type of carelessness are serious. From this data, the report draws the following conclusions:

• Page 38: After incorrectly implying that Forest2Market reported that pellet and OSB demand were responsible for a 22% increase in pine pulpwood price between September/October of 2012 and 2013,the report lists several studies that outline the potential effects on forests of high pulpwood prices, including an increased rate of harvests and displacement of wood used for non-bioenergy wood products and establishment of more intensively managed plantations (Abt and Abt, Walker et al) and the replacement of agricultural lands and natural timberland with plantations (Wear and Greis).
• Page 90: After repeating this same inaccurate view of the data from our blogs on page 89, the report concludes, “This indicates that counterfactuals relevant to high demand for wood may be most likely.”

Because the use of Forest2Market blog content is misleading at best and manipulated to a purpose at worst, the counterfactuals the report’s findings are based on may be equally misleading—they may be counter, but they are not factual. In fact, they are prime examples of how NOT to use data to make policy.