RESEARCH ACTIVITIES

PROCESS EVALUATION RESEARCH  |  IMPACT EVALUATION RESEARCH  |  DATA TRACKING RESEARCH  |  SURVEY RESEARCH
NEEDS ASSESSMENT  |  ECONOMIC & POLICY ANALYSIS  |  NON-ENERGY IMPACT ANALYSIS  |  TECHNICAL ASSISTANCE

Performance Measurement Research

Performance measurement involves the development of metrics to assess program success and the use of program data to measure those metrics. Some examples of metrics used include the number of households served by a program, the program penetration rate, a targeting index for vulnerable households, and the percentage reduction in energy usage or energy burden.

Performance Measurement Outputs

  • A Logic Model that describes the program assumptions, inputs, immediate outcomes, intermediate outcomes, and final program impacts.
  • A set of performance measures that allows program managers and policymakers to assess the success of a program in meeting defined goals.
  • A Data and Indicator Model that maps the specific outcomes to performance indicators and the data that are needed to measure those performance indicators.
  • Periodic calculations of performance measures to determine how a program performs over time and with changes in program design and implementation.

APPRISE Performance Measurement Research

Below we provide a few examples of some of our recent projects that included performance measurement.   Click the tabs below to learn more about these projects, or see here for a full list of projects that included performance measurement.

PGW’s Customer Responsibility Program provides low-income PGW customers with a reduced bill to increase energy affordability. Their other Universal Service Programs provide energy efficiency services, case management services, and emergency bill payment assistance.  The 2019 evaluation assessed the characteristics of program participants, payment compliance by program participants, and arrearage forgiveness received by program participants. The research estimated the impact of the program on affordability, payments, and collections actions and costs. Evaluation activities included interviews with program managers and staff, needs assessment, in-depth interviews with participating customers, and analysis of customer billing, payment, and collections data.

FirstEnergy offers a Low-Income Usage Reduction Program (LIURP) and Act 129 Low-Income Programs to assist low-income customers. The programs provide energy efficiency and energy education services, help low-income customers maintain electric service, and protect customers’ health and safety. APPRISE conducted a study to assess recent program changes, and to provide guidance to FirstEnergy on future tracking, reporting, and process improvements. The research included an assessment of what changes should be made to program procedures, whether additional measures should be added, and how projected energy savings for additional energy efficiency measures should be calculated.

Ohio’s Percentage of Income Payment Program (PIPP) was revised as the PIPP Plus in November 2010. The program helps low-income households maintain utility service by providing a reduced bill based on a percentage of the household’s annual income. APPRISE conducted an evaluation of this program to assess the effectiveness of the changes to PIPP that aimed to provide incentives for regular, on-time payments and consequences for failure to adhere to program rules. The research included administrative interviews, participant surveys, and analysis of program and utility data.

APPRISE reviewed DOEE’s LIHEAP design and made recommendations for program modifications in 2018. This study assessed DOEE’s LIHEAP procedures, analyzed the population of households eligible for energy assistance, and calculated the impacts of the assistance on participants’ energy burden. The study furnished DOEE with information needed to modify its program benefits design matrix in a way that meets the statutory guidance furnished to DOEE by the federal LIHEAP program.

As a result of the 2018 study, DOEE modified its LIHEAP program benefits design matrix.  In 2020, APPRISE worked with DOEE to calculate the impact of those changes on participants’ energy burden, update analyses of households eligible for energy assistance, and assess how best to integrate DOEE’s Solar for All program with the LIHEAP program.

APPRISE conducted national evaluations of the Department of Energy’s Weatherization Assistance Program (WAP) for program years 2008 and 2010.   In this study of New York’s WAP for the New York State Division of Housing and Community Renewal (DHCR), APPRISE used data collected from New York State to furnish detailed information on the performance of the New York WAP program.  This analysis provided statewide statistics, results for selected subgroups, and program savings and cost-effectiveness statistics.