In partnership with AESP: The increasing roles of DERs, connected technology and Big Data are driving rapid change in energy efficiency. As we shape the Utility of the future, this community will help you keep up with the latest developments. 


You need to be a member of Energy Central to access some features and content. Please or register to continue.


Energy program evaluation across the U.S.: varied practices, sound principles

Utility energy efficiency programs invested $8 billion nationwide in 2018, so determining their impacts and effectiveness is critical. How much energy was saved? Were program dollars well spent? Did programs meet their goals? How did customers benefit? And how much carbon was kept out of the atmosphereAsking and answering these questions is the realm of energy program evaluation—practices that are integral to today’s energy efficiency programs and that ensure efficiency’s multiple benefits accrue to customers, utilities, and society.

A new national survey of energy program evaluation policies and practices finds wide variation among states, as earlier studies have also shown. Despite the variation, the survey also found common underlying methodologies and evaluation techniques that have benefitted from industry efforts over the past several years to improve their consistency and quality.

Read the Report

Register for the Webinar

The art and science of energy program evaluation

A fundamental challenge for evaluating energy efficiency programs is estimating the energy savings that result from a given program—that is, quantifying the amount of energy that was not used. There is no such thing as an energy savings meter. Estimating savings typically involves applying statistical, engineering, and modeling techniques to customer energy data and other information. Such techniques have evolved, becoming more advanced through the ability to collect and analyze highly granular customer data. Program evaluators also rely on widely accepted methods, data, and algorithms to estimate savings, increasingly documented and required in technical reference manuals (TRMs), which serve as state guidelines for program evaluation. These practices and advancements have built and strengthened the science of evaluation, but there remains an element of art in program evaluation. The evaluator must know which techniques to apply in different situations given limits, conditions, and available resources.  

Common structures, varying practices

State policies and practices for evaluation are not uniform across the United States. This is not surprising, given the great diversity that exists among states in the structure of their utilities, the policies and regulations that govern them, and the nature of their energy resources. Prior ACEEE research (completed in 2012) found similar diversity in energy program evaluation policies and practices across the states.

Utilities and regulatory commissions rely on evaluation results to guide decisions on energy efficiency and other demand-side programs, such as which programs to offer and how much to spend on them. To make good decisions in the public interest, oversight and administration of program evaluation needs to be structured to ensure accurate and unbiased results.   

In ACEEE’s new update to our earlier research, we found that program evaluation continues to be performed largely by independent contractors. Assigning this role to evaluation professionals helps ensure that results are based on sound methodological practices and that they can be relied upon for good decision making by program administrators and regulators.

Another key issue is evaluation oversight. We found that nearly all regulatory commissions use evaluation results for general oversight of programs and provide opportunities for public involvement. For example, in Massachusetts and Arkansas, the stakeholder collaboratives that advise and review program development, implementation, and evaluation include members of the public.

Shared core principles and new tools to meet new challenges

Our survey confirmed that program evaluation is solidly grounded in a set of core principles. Evaluators today follow well-established professional practices and can draw on a wealth of resources, such as TRMs and the Database of Screening Practices, many of which have been developed in recent years. However, changes in energy markets, technologies, and program goals are affecting program designs and associated evaluation practices. Evaluators face new challenges, such as accounting for carbon reductions or valuing distributed energy resources. To meet these new challenges and improve traditional practices, evaluators can use various advanced technologies, such as smart meters and data analytics. We found that the uptake of such advanced applications is slow. Slightly more than half of the states we surveyed have not yet used advanced metering technologies or data analytics for program evaluation.

Because most utility energy efficiency programs are funded through customer bills, ensuring that the programs are cost effective is a primary function of program evaluation. Several cost-effectiveness tests are used, each representing a different perspective and sets of costs and benefits. As in our 2012 research, we found that the application of cost-effectiveness tests continues to vary widely among states. Despite the variation, there is a growing trend toward embracing core principles that yield balanced and comprehensive assessments of costs and benefits—something some past practices did not do, though it is critical for making prudent choices about investments in utility infrastructure and avoiding wasteful or unnecessary spending.

Energy efficiency is increasingly being called on to deliver multiple benefits to consumers, utilities, and society. In addition to its more traditional focus on reducing costs for customers and utilities, energy efficiency is a fundamental tool for addressing climate changepublic healthenvironmental justice, and energy equity. Evaluation is the way we measure and assess how well our energy programs are meeting these objectives. Our survey shows that evaluation policies and practices are generally well poised to meet current and emerging challenges for assessing the performance and impacts of energy efficiency programs. The critical role of evaluation, however, requires continued oversight and public involvement to ensure accurate and unbiased results that can be used to make wise choices about our energy resources.


Spell checking: Press the CTRL or COMMAND key then click on the underlined misspelled word.
Katherine Johnson's picture
Katherine Johnson on Nov 18, 2020

Thanks to ACEEE for conducting this study and reinforcing the value of energy efficiency programs. EE programs should be the starting point for utility activities, augmented by DR programs that help fill these gaps. Also, appreciate the shoutout to the effective stakeholder collaboratives that work hard to provide cost-effective and engaging EE programs across all sectors. A rising tide raises all boats, and effective stakeholder collaboration benefits everyone. 

Matt Chester's picture
Matt Chester on Nov 18, 2020

EE programs should be the starting point for utility activities, augmented by DR programs that help fill these gaps.

I'd love to hear more of your thoughts on this, Katherine. Is the implication that energy efficiency can have greater results so it's the better starting point, or just that encouraging EE is a lot easier and lower threshold on which to engage customers than DR so EE should be the basis and DR will then be encouraging to a subset of those who bought into EE? 

ACEEE Blog's picture

Thank ACEEE for the Post!

Energy Central contributors share their experience and insights for the benefit of other Members (like you). Please show them your appreciation by leaving a comment, 'liking' this post, or following this Member.

Get Published - Build a Following

The Energy Central Power Industry Network is based on one core idea - power industry professionals helping each other and advancing the industry by sharing and learning from each other.

If you have an experience or insight to share or have learned something from a conference or seminar, your peers and colleagues on Energy Central want to hear about it. It's also easy to share a link to an article you've liked or an industry resource that you think would be helpful.

                 Learn more about posting on Energy Central »