The 2009 economic recovery package included billions of dollars in Federal funding for state-run weatherization programs targeting low-income housing. Over the years, skeptics have questioned whether those programs delivered the energy savings they promised — and whether the cost of the weatherizing measures was justified by the reductions in home heating or cooling bills.

Those critics have new ammunition this month in the form of a newly released working paper by economists Meredith Fowlie, Michael Greenstone, and Catherine Wolfram from the University of California at Berkeley and the University of Chicago, examining the costs and benefits of measures applied in the federally-funded Weatherization Assistance Program (WAP) in Michigan. Online news organization Vox has a report (see: “A new study looks at federal energy-efficiency efforts — and the results are grim,” by Brad Plumer).

“The economists conducted a large randomized controlled trial of 30,000 homes in Michigan involving the federal Weatherization Assistance Program, which helps low-income families replace their furnaces, upgrade insulation, and seal up leaks along doors and windows,” Vox reports. “The researchers found that the upfront cost of efficiency upgrades in the Michigan program came to about $5,000 per house, on average. But their central estimate of the energy savings only amounted to about $2,400 per household, on average, over the lifetime of the upgrades. The federal program did help households save energy: after the upgrades, homes used 10 to 20 percent less energy for electricity and heating. But, importantly, that was only about 39 percent of the savings that modelers had predicted ahead of time. The program simply wasn't as effective at saving energy as everyone thought.”

The working paper itself is posted at the University of Chicago website (see: “Do Energy Efficiency Investments Deliver? Evidence from the Weatherization Assistance Program,” by Meredith Fowlie, Michael Greenstone, and Catherine Wolfram). The authors explain that they started out with a question: if energy efficiency upgrades are such a good investment, why don’t more people participate? The authors suggest a troubling possibility: “that the real world returns on energy efficiency investments are lower than the engineering models indicate.” Maybe, in other words, households who don’t bother with weatherization are making a smart choice.

To find out if the meager “take-up” of weatherizing offerings is just a communication issue, the authors tried a social experiment. Out of 30,000 eligible Michigan households who qualified for weatherization assistance, the team offered “encouragement” to one third, selected at random. The rest of the eligible households weren’t contacted by the team. “An aggressive encouragement intervention increased WAP participation from less than 1% in the control group to about 6% in the encouraged group,” the team reports. “Ultimately, these extensive efforts only managed to increase the participation rate by 5 percentage points at a cost of more than $1,000 per weatherized household, revealing low demand in the eligible population for a program with considerable potential benefits.”

So the team took a closer look at those “potential benefits” — and found that they came up short. “The findings suggest that the benefits of these investments are substantially less than  the upfront costs,” the economists report. “We estimate that the WAP energy efficiency investments reduce monthly energy consumption by 10-20% on average. Although this surely provides a substantial assist to participating low-income households in the form of reduced energy bills, the upfront investment costs are about twice the realized energy savings. Further, the model-projected savings are roughly 2.5 times the actual savings.” Weatherizing, in other words, delivered less than half the predicted value to building residents; and it would have been cheaper just to hand them the money.

But didn’t the weatherizing work also reduce carbon pollution? Sure, the economists say — but at an estimated cost of $329 per avoided ton of carbon — “about an order of magnitude larger than the U.S. government’s estimate of the social cost of carbon of roughly $38 a ton.”

Not surprisingly, the working paper has drawn fire from home energy efficiency policy advocates. National Resources Defense Council (NRDC) scientist Merrian Borgeson, for example, takes issue with the cost estimates used in the paper (see: “No Question: Energy Efficiency Investments Deliver Huge Benefits,” by Merrian Borgeson). “Unfortunately, in calculating the cost-effectiveness of this program, the researchers included not only the costs of the non-energy related improvements, but the full cost of the efficient equipment installed, concluding that this efficiency program doesn't deliver,” Borgeson writes. The project costs included basic non-energy improvements like removing dangerous wiring and toxic asbestos, which increased costs by about 25 percent (about $1,000 on top of the $4,000 in improvements).”

The full cost of new furnaces and windows were also included (as opposed to the incremental cost of the more efficient model which is the standard measure of energy efficiency program cost-effectiveness),” Borgeson goes on. “No one gets a furnace solely for the energy-saving features (they mostly want to be warm in the winter!). This accounting makes a huge difference. For example, about 1/3 of all participants in the program got a new furnace at a cost of over $3,100 each. If you assume that the ‘incremental’ efficiency premium on the furnace is about $400 (meaning the less efficient furnace option is $2,700), the cost of the efficiency portions these projects is far less - about $3,100 on average instead of $5,000.”

And in any case, Borgeson cautions against reading the paper’s conclusions too broadly: “If you read the study details, the authors are clear that these research finding are not widely applicable to energy efficiency, or even necessarily to other weatherization programs.”