APPRISE Performance Measurement Research

APPRISE conducted a comprehensive evaluation of Allegheny Power’s Universal Service Programs. The evaluation research included interviews with Allegheny and agency program managers and staff, surveys with participating and nonparticipating customers, analysis of program databases, and analysis of the impacts of the payment assistance program on energy affordability, payment compliance, arrearages, and collections costs.

Ameren’s Keeping Current Program provides electric bill payment assistance and arrearage forgiveness to low-income customers. APPRISE conducted four evaluations of the Keeping Current Program that included surveys, in-depth interviews, analyses of program benefits delivered, and analysis of the program’s impact on affordability, bill payment, and collections actions.  The reports and presentations are available for download here.

Reports: 2012, 2015, 2016, 2019

Presentations: 2015, 2016, 2019

The California Investor-Owned Utilities (IOUs) have introduced a Percentage of Income Payment Plan (PIPP) Pilot aimed to reduce the number of low-income households at risk of disconnection. The Pilot will limit energy bills to approximately four percent of a household’s income. APPRISE is conducting a comprehensive evaluation to assess the impact of the Pilot on payment behavior, disconnections, energy usage, and access to essential services. The research will include interviews with IOUs and Community-Based Organizations; surveys with participating customers; and analysis of customer bills, payments, usage, and collections actions using a matched comparison group. The evaluation will recommend whether the Pilot should be implemented with modifications based on the findings from the comprehensive study.

The Federal Community Services Block Grant (CSBG) Program provides funding to alleviate the causes and conditions of poverty in communities through services that address employment, income management, education, housing, nutrition, and health.  APPRISE is working with the U.S. Department of Health & Human Services, Division of Community Assistance to develop a Performance Management Website and Data Warehouse that facilitates the analysis of performance measurement data reported by grantees of the CSBG Program.  APPRISE is also providing training and technical assistance to the Division of Community Assistance, grantees, and other partners of the CSBG Program in the use of these performance management resources.

APPRISE reviewed DOEE’s LIHEAP design and made recommendations for program modifications in 2018. This study assessed DOEE’s LIHEAP procedures, analyzed the population of households eligible for energy assistance, and calculated the impacts of the assistance on participants’ energy burden. The study furnished DOEE with information needed to modify its program benefits design matrix in a way that meets the statutory guidance furnished to DOEE by the federal LIHEAP program.

As a result of the 2018 study, DOEE modified its LIHEAP program benefits design matrix.  In 2020, APPRISE worked with DOEE to calculate the impact of those changes on participants’ energy burden, update analyses of households eligible for energy assistance, and assess how best to integrate DOEE’s Solar for All program with the LIHEAP program.

The purpose of this study was to assist DDOE’s efforts to mitigate the rising energy costs on those with the highest home energy needs, greatest energy burdens, and least amount of available resources. The study focused on households that pay utility gas and/or electric bills, and are income-eligible and program-eligible for rate discounts. The study characterized the population of households that are eligible for rate discounts, estimated the rate discount program penetration rates, and analyzed how the rate discounts complement the LIHEAP program to reduce energy burdens for participating households.

Duquesne Light implemented Universal Service Programs to improve affordability for low-income customers. The programs include the Customer Assistance Program (CAP), which provides reduced payments and arrearage forgiveness; Smart Comfort, which provides energy efficiency and energy education services; CARES, which provides outreach and referral services; and the Hardship Fund, which provides emergency assistance. APPRISE conducted an evaluation of the performance of these programs and the integration among the different programs. The research included interviews with program and agency staff, a participant survey, and analysis of the impacts of the program on affordability and payment compliance.

FirstEnergy offers a Low-Income Usage Reduction Program (LIURP) and Act 129 Low-Income Programs to assist low-income customers. The programs provide energy efficiency and energy education services, help low-income customers maintain electric service, and protect customers’ health and safety. APPRISE conducted a study to assess recent program changes, and to provide guidance to FirstEnergy on future tracking, reporting, and process improvements. The research included an assessment of what changes should be made to program procedures, whether additional measures should be added, and how projected energy savings for additional energy efficiency measures should be calculated.

The Illinois REACh project was an innovative pilot program aimed to help low-income households maintain energy service throughout the year and reduce the health and safety risks associated with energy service disruption. The comprehensive program worked with clients to increase resources available for paying energy bills, reduce the amount of energy used through effective energy-saving actions and participation in usage reduction programs, and develop the skills that are needed to maintain energy service through proactive management of their monthly budgets and energy bills. APPRISE conducted an evaluation of this program to assist program managers in assessing whether the program design is appropriate, whether there are any barriers to implementing the program as designed, and whether the program has achieved the intended impacts.

APPRISE conducted an evaluation of the Illinois Solar for All Program which brings photovoltaic power to low-income and environmental justice communities throughout Illinois and creates a long-term, low-income solar marketplace.

The ILSFA Program has four key components.

  • Low-Income Distributed Generation: Single- and multi-family residential properties occupied by low-income households can receive incentives for new solar generation systems.
  • Low-Income Community Solar: Low-income owners and renters can buy or lease a share of the solar system and receive credits on their utility bills for the energy produced by their share of the system.
  • Incentives for Non-Profits and Public Facilities: Non-profit organizations and public entities within environmental justice communities or low-income communities can receive incentives for new solar generation systems.
  • Low-Income Community Solar Pilot Projects: Projects with community partnerships and ownership will be awarded based on a competitive procurement approach.

The evaluation included analysis of community outreach, system cost and production, environmental and economic benefits, impacts on participants’ energy costs and burden, job training, and system reliability.

Louisville Gas and Electric Company and Kentucky Utilities Company offer a Home Energy Assistance (HEA) Program to assist the poorest households in their service territories least able to afford to pay their energy bills. Funded by ratepayers and shareholders, the programs (structured differently by company) provide fixed monthly credits to LIHEAP participants. APPRISE conducted a comprehensive HEA assessment to analyze the program management, availability, and customer participation; assess impacts on payments, arrearages, collections actions, and weatherization participation; and make recommendations for program improvement.

APPRISE previously worked with the federal LIHEAP office to develop a performance measurement plan for targeting heating assistance to households with vulnerable members and a logic model to support that performance measurement plan. APPRISE also developed a separate model to examine the targeting of elderly households. APPRISE then extended the logic models to address all LIHEAP program components including cooling assistance, crisis assistance, home energy-related equipment repair and replacement, weatherization, and Assurance 16 activities. APPRISE also developed a logic model for targeting LIHEAP income-eligible households with at least one young child. The final report recommended an integrated strategy for implementing all of the logic models.

The Minnesota Conservation Improvement Programs (CIP) portfolios include low-income components to assist income-eligible households with energy conservation. APPRISE conducted research through the Conservation Applied Research & Development (CARD) Grant Program to provide a comprehensive review of the low-income efficiency programs, evaluate the program processes, and assess program policies and guidelines. The goal of the research was to facilitate more effective program operations and policymaking.

The Community Service Block Grant Program provides funding to alleviate the causes and conditions of poverty in communities through services that address employment, income management, education, housing, nutrition, and health.  APPRISE supported the National Association of State Community Service Programs in the development of a Data Warehouse for the federal Community Services Block Grant (CSBG) program.  We developed tools for reporting, validation, and analysis of performance measure data at the local, state, territory, and national level.  Additionally, APPRISE supported state efforts to collect and report new performance metrics established by the Department of Health and Human Services, Office of Community Services.  This project has led to improved data quality, an enhanced understanding of CSBG outcomes, and will be used to inform program decision-making.

APPRISE was the lead contractor for the evaluation of the National Low-Income Weatherization Assistance Program (WAP). APPRISE had three key partners for this effort – Blasnik and Associates, Dahloff Associates, and the Energy Center of Wisconsin, and was also supported by technical experts from across the country. The evaluation consisted of data collection from state WAP offices, local agencies, and participating and nonparticipating clients. Utility data was collected and analyzed for a sample of clients from every state and submetering studies were conducted to assess impacts on clients who use bulk fuels and to conduct special technical studies. The evaluation assessed program operations and delivery, energy impacts, and non-energy impacts, and made recommendations for modifications to program design and delivery.

The New Hampshire REACh Furnace Cleaning Program aimed to decrease energy usage and reduce home energy insecurity for low-income households in northern New Hampshire by cleaning furnaces and identifying furnaces in need of repair and replacement. Extensive furnace repairs and furnace replacements were also provided in those homes where the need was identified. APPRISE’s evaluation included interviews with program managers, community agency staff, and furnace cleaners; and analysis of the impact of the program on energy usage, energy insecurity, and furnace safety and efficiency indicators.

The NJ Comfort Partners Program provides no-cost energy efficiency services to low-income utility customers throughout the state. APPRISE conducted a comprehensive evaluation of this program. The Data Tracking Research assessed the comprehensiveness, sufficiency, and accuracy of the program database. The Process Evaluation included review and assessment of materials and procedures; interviews with utility managers; interviews and on-site observations at service delivery contractors; on-site observation of audits, installations, and third party inspections; and post-completion inspections of program installations. The Affordability Evaluation included a telephone survey with program participants and analysis of the program’s impact on affordability of energy bills and participation in the NJ Universal Service Fund program. The Impact Evaluation included analysis of program data, engineering impact analysis, usage impact analysis, and savings realization analysis. The Energy Savings Protocols Research included review and assessment of protocols currently in use to estimate program savings.

The NJ Comfort Partners Program provides no-cost energy efficiency services to low-income utility customers throughout the state. The 2017 evaluation reviewed program changes that were made since the 2013 evaluation, estimated energy savings with a billing analysis, reviewed and made recommendations for changes to the Energy Saving Protocols, and conducted targeted participant telephone interviews and on-site inspections of completed jobs.  The evaluation found improved energy savings and recommended additional program refinements to further increase savings achieved through the program.

The NJ Comfort Partners Program provides no-cost energy efficiency services to low-income utility customers throughout the state. The 2020 evaluation reviewed program changes that were made since the 2017 evaluation, estimated energy savings with a billing analysis, reviewed and recommended changes to the Energy Saving Protocols, and conducted a cost-effectiveness analysis.

The New Jersey Statewide Heating Assistance and Referral for Energy Services (NJ SHARES) is a nonprofit corporation organized to provide assistance to individuals and families living in New Jersey who are in need of temporary help paying their energy bills. APPRISE conducted an annual impact evaluation of the New Jersey SHARES program from 2006 through 2019. The evaluations included analysis of administrative and utility data to assess the program’s impact on maintenance of electric and gas service and on energy bill payment.  Some of the evaluations also included surveys and in-depth participant interviews to assess whether the crises had been resolved, what additional assistance was needed, and the interest and potential for energy efficiency program participation.

APPRISE conducted national evaluations of the Department of Energy’s Weatherization Assistance Program (WAP) for program years 2008 and 2010.   In this study of New York’s WAP for the New York State Division of Housing and Community Renewal (DHCR), APPRISE used data collected from New York State to furnish detailed information on the performance of the New York WAP program.  This analysis provided statewide statistics, results for selected subgroups, and program savings and cost-effectiveness statistics.

The Ohio Department of Development, Office of Energy Efficiency, has implemented programs funded by the Universal Service Fund. Participating customers receive cost-effective energy efficiency services and education targeted at reducing electric usage. APPRISE conducted a Process Evaluation of this program. The work included review of all program documents; interviews with program managers, staff, and contractors; observation of service delivery; customer surveys; contractor surveys; and economic analysis of the impacts of the program on Ohio’s economy.

Ohio’s Percentage of Income Payment Program (PIPP) was revised as the PIPP Plus in November 2010. The program helps low-income households maintain utility service by providing a reduced bill based on a percentage of the household’s annual income. APPRISE conducted an evaluation of this program to assess the effectiveness of the changes to PIPP that aimed to provide incentives for regular, on-time payments and consequences for failure to adhere to program rules. The research included administrative interviews, participant surveys, and analysis of program and utility data.

Ohio’s Department of Development, Office of Community Services, was awarded a REACh grant to research how to provide additional resources to low-income weatherization services in a way that addresses the more comprehensive needs of low-income households. These funds were used to assess the in-home environmental health risks for vulnerable individuals, prioritize usage reduction measures to meet the energy and health needs of vulnerable individuals, and provide additional measures that enhance the health and safety of the home. The program also established a partnership with vulnerable households to take actions to mitigate the environmental health risks in the home and refer clients to other services that can improve the health of the home and the clients who live there. APPRISE conducted a process and impact evaluation of this project.

PECO implemented Universal Service Programs to help low-income customers maintain energy services and protect customers’ health and safety. The programs include CAP which provides reduced payments and arrearage forgiveness, LIURP which provides energy efficiency and energy education services, CARES which provides outreach and referral services, and MEAP which provides emergency financial assistance for overdue energy bills. The 2019 evaluation conducted by APPRISE included a review of program procedures and implementation; customer needs assessment; interviews with program managers and implementation partners; customer surveys; and an analysis of the impact of the program on energy affordability, bill payment, collections, and energy usage.

APPRISE conducted an evaluation of Peoples Natural Gas Universal Service Programs.  Peoples’ Universal Service Programs provide reduced natural gas bills, arrearage forgiveness, emergency bill payment assistance, energy efficiency services, and case management services to low-income households.  The evaluation assessed the efficiency and effectiveness of these programs through background research, needs assessment research, program administrator interviews, customer surveys, program database analysis, and analysis of the impacts of the program on affordability, bill payment, and collections actions and costs.

PGW’s Home Comfort Program provides no-cost comprehensive weatherization treatments to selected high-usage PGW customers. APPRISE’s annual evaluations estimate energy savings, assess savings achieved by individual contractors and energy efficiency measures, and analyze bill savings.  Evaluations have included analysis of client targeting; interviews with program managers and implementation contractors; and review of inspection reports to identify how PGW can improve the program’s efficiency and effectiveness.

PGW’s Customer Responsibility Program provides low-income PGW customers with a reduced bill to increase energy affordability. Their other Universal Service Programs provide energy efficiency services, case management services, and emergency bill payment assistance.  The 2019 evaluation assessed the characteristics of program participants, payment compliance by program participants, and arrearage forgiveness received by program participants. The research estimated the impact of the program on affordability, payments, and collections actions and costs. Evaluation activities included interviews with program managers and staff, needs assessment, in-depth interviews with participating customers, and analysis of customer billing, payment, and collections data.

APPRISE conducted Process and Impact Evaluations of PPL Electric Utilities’ (PPL) Universal Services Programs in 2008, 2014 and 2020. The Universal Service Programs provide bill payment assistance and energy efficiency services to PPL’s low-income customers. The 2020 evaluation included a needs assessment, interviews with program managers and staff, interviews with service delivery agencies, surveys with participating customers, and analysis of the impact of the programs on energy affordability and bill payment practices.

Sacramento Municipal Utility District’s (SMUD) Low Income Weatherization Program provides no-cost energy services to qualified low-income residential customers. The goals of the program are to reduce electric bills, improve safety and comfort, provide energy education to reduce usage and lower bills, and partner with contractors and community organizations in providing these services. APPRISE’s comprehensive evaluation included needs assessment, interviews with program administrators and contractors, a participant survey, on-site inspections of completed jobs, and billing analysis to measure the impacts on energy usage.

UGI’s Low-Income Usage Reduction Program (LIURP) provides no-cost energy efficiency services to low-income customers in its service territory. APPRISE conducted a Process and Impact Evaluation of this program. The research included review and assessment of program materials and procedures, on-site interviews with UGI managers and staff, in-depth telephone interviews with a sample of program participants, and analysis of the program’s impact on the energy usage of participating customers.

UGI has Universal Services programs to help low-income customers maintain affordable energy services. APPRISE conducted an evaluation of these programs in 2012 and is currently conducting another comprehensive evaluation. The focus of the evaluation is the Customer Assistance Program (CAP), that provides reduced energy bills to improve affordability for low-income customers. APPRISE’s research includes an assessment of the program design; review of the program costs and implementation; participant and non-participant feedback; and analysis of the impacts of the program on bill payment, arrearages, and collections costs.

The Vermont REACh program created a private, non-profit energy consumers’ cooperative called Consumerco. This cooperative aims to achieve the Vermont REACh goal by providing comprehensive energy and efficiency services to its members. Consumerco aims to assist its low-income consumers in attaining energy self-sufficiency by instructing them how to minimize energy usage, choose the most cost-effective fuel, obtain the least cost supply of fuel, receive fuel assistance, and finance weatherization. In this project, we conducted background research, provided a process evaluation of the Consumerco, provided a process evaluation of the case management, and conducted an impact evaluation. We furnished concrete performance data and program documentation to the federal LIHEAP office and REACh program managers.