RESEARCH ACTIVITIES – Process Evaluatation Research 2017-02-27T15:08:39+00:00

Process Evaluation Research

  • Administrative Research: Assessment of the administrative context for the program.
  • Implementation Research: Analysis of the effectiveness of program recruitment, intake, service delivery, follow-up, and tracking.
  • Customer Research: Analysis of satisfaction, motivation, retention of information, and behavioral changes.
  • Nonenergy Benefits Research: Measurement of health and safety, economic, environmental, corporate positioning, and customer benefits.

Process Evaluation Activities

Data and Document Review

Document Review: Review of program history, program goals, and program policy statements.
Program Budget Review: Analysis of resources devoted to the project, including contract funding and staff time allocation.

Program Statistics Review: Analysis of production and production rates by target groups, service delivery area, and other variables of interest.

Quality Control Check: Assessment of the quality and reliability of program operations by direct observation or review of quality control inspection documents.

Interviews and Surveys

Executive Interviews: Guided in-depth interviews with senior program managers regarding program goals, program resource requirements, program oversight, and metrics for success.

Staff Interviews: Detailed interviews with program staff to document details of outreach, recruitment, eligibility, intake, scheduling, service delivery, information tracking, and follow-up.

In-Depth Customer Interviews: Guided and open-ended interviews with program participants.

Customer Surveys: Telephone, mail, or in-person interviews with customers who have received program services.

Focus Groups: Group discussions with service delivery staff or customers who have received program services.

Observations and Inspections

On-Site Observation: Observation of all aspects of program implementation including outreach, recruitment, intake, scheduling, service delivery, and follow-up.

On-site Inspection: Post service inspection of work delivered by the program and interviews with program recipients.

 

Process Evaluation Outputs

Program Documentation

  • Documentation of goals, mandate, and funding for the program
  • Documentation of program administration roles and responsibilities
  • Documentation of program protocols

Procedures and Context Assessment

  • Assessment of the feasibility of program protocols
  • Assessment of the barriers to effective program operation
  • Assessment of the barriers to program impact
  • Assessment of the population served

Implementation Assessment

  • Verification of actual program procedures
  • Assessment of the effectiveness of each element of program operations in meeting the program objectives
  • Assessment of program operations efficiency and quality
  • Assessment of individual program experiences

Performance Measurement

  • Development of performance measurement indicators
  • Analysis and quantification of customer understanding, actions, and satisfaction
  • Analysis of program achievements as compared to program goals
  • Recommendations for program modification

APPRISE Process Evaluations – Payment Programs

APPRISE conducted a comprehensive evaluation of Allegheny Power’s Universal Service Programs. The evaluation research included interviews with Allegheny and agency program managers and staff, surveys with participating and nonparticipating customers, analysis of program databases, and analysis of the impacts of the payment assistance program on energy affordability, payment compliance, arrearages, and collections costs.
Ameren introduced a pilot of a low-income payment assistance program that provides electric bill payment assistance to low-income customers. Participants receive monthly bill credits toward their electric bill and additional credits toward their arrearages. APPRISE provided an evaluation of this program in 2012 that included interviews with local agencies that conduct intake for the program, a survey of participating customers, analysis of program data, and analysis of program impacts on affordability, payment coverage, and terminations. Ameren made program modifications in response to findings from the initial evaluation. APPRISE conducted a follow-up Process and Impact Evaluation of the program in 2014 that documented the impacts of the program changes. APPRISE is currently conducting a third evaluation of this program that includes interviews with defaulted customers, analysis of program data, and analysis of the impacts of the program on affordability and bill payment. Findings from this research will be used to further refine the program.
Baltimore Gas and Electric (BGE) designed and implemented pilot programs to determine the effect of energy bill discounts and payment counseling on the payment timeliness of limited-income customers. The Graduated Rate Discount (GRAD) Program provided a range of usage-based discounts. Some of the customers also received a home energy audit or payment counseling phone calls. The Customer Assistance Maintenance Program (CAMP) pilot tested changes to BGE’s CAMP including increased bill credits, payment counseling, or both. APPRISE’s process and impact evaluation assessed the effectiveness of these pilots in increasing customers’ on-time bill payment and energy conservation behavior.
APPRISE conducted research to assist the Colorado’s Governor’s Commission on Low-Income Energy Assistance to develop recommendations to improve the effectiveness and efficiency of the state’s low-income energy assistance services. The research included analysis of American Community Survey data, the Colorado LIHEAP database, and the Energy Outreach Colorado database; review and assessment of current LIHEAP procedures and implementation; surveys with participating and non-participating low-income households; and interviews with program managers at the state and agency level.
Duquesne Light implemented Universal Service Programs to improve affordability for low-income customers. The programs include the Customer Assistance Program (CAP), which provides reduced payments and arrearage forgiveness; Smart Comfort, which provides energy efficiency and energy education services; CARES, which provides outreach and referral services; and the Hardship Fund, which provides emergency assistance. APPRISE conducted an evaluation of the performance of these programs and the integration among the different programs. The research included interviews with program and agency staff, a participant survey, and analysis of the impacts of the program on affordability and payment compliance.
FirstEnergy implemented Universal Service Programs to help low-income customers maintain electric service and protect customers’ health and safety. The programs include the CAP Program, which provides reduced payments and arrearage forgiveness; LIURP/WARM, which provides energy efficiency and energy education services; CARES, which provides outreach and referral services; the Hardship Fund, which provides emergency assistance; and Gatekeeper, where field personnel recognize and report customers who may be in distress. APPRISE conducted an evaluation of the performance of these programs, their ability to help low-income customers maintain affordable service, and the integration among the different programs. This evaluation included analysis of program databases, interviews with FirstEnergy managers and staff, interviews with staff at organizations that implement the programs; and analysis of the impact of CAP on bills, customer payments, and affordability.
Louisville Gas and Electric Company and Kentucky Utilities Company offer a Home Energy Assistance (HEA) Program to assist the poorest households in their service territories least able to afford to pay their energy bills. Funded by ratepayers and shareholders, the programs (structured differently by company) provide fixed monthly credits to LIHEAP participants. APPRISE conducted a comprehensive HEA assessment to analyze the program management, availability, and customer participation; assess impacts on payments, arrearages, collections actions, and weatherization participation; and make recommendations for program improvement.
APPRISE assessed the fiscal integrity and operational efficiency of the NJ LIHEAP and USF Programs. This research included an assessment of the performance of the New Jersey Department of Community Affairs (DCA), Division of Housing and Community Resources (DHCR), Office of Home Energy Assistance (OHEA) and an assessment of the operations of the LIHEAP and USF programs. We reviewed program documents, and conducted interviews with HHS LIHEAP compliance staff, DCA management, USF managers at the NJ Board of Public Utilities, and managers at Community Based Organizations that administer the programs. We also reviewed program databases and statistics, conducted on-site interviews and observations with a sample of local agencies, and conducted a survey with LIHEAP recipients.
The New Jersey Statewide Heating Assistance and Referral for Energy Services (NJ SHARES) is a nonprofit corporation organized to provide assistance to individuals and families living in New Jersey who are in need of temporary help paying their energy bills. APPRISE conducts an annual impact evaluation of the New Jersey SHARES program. The evaluation includes analysis of administrative and utility data to assess the program’s impact on maintenance of electric and gas service and energy bill payment. In 2012, APPRISE also conducted a survey of 2011 grant recipients to assess whether the crises had been resolved, what additional assistance was needed, and the interest and potential for energy efficiency program participation. APPRISE also serves as an advisor to the NJ SHARES Board of Directors.
The New Jersey Universal Service Fund (USF) program provides fixed credits to eligible low-income families to limit electric and gas bills to six percent of household income. APPRISE conducted a process and impact evaluation of this program. This evaluation provided information on the benefits and costs of the USF program and alternatives for program design and operations. Program evaluation research consisted of administrative interviews, client surveys, program and utility data analysis, and financial data analysis.
Ohio’s Percentage of Income Payment Program (PIPP) was revised as the PIPP Plus in November 2010. The program helps low-income households maintain utility service by providing a reduced bill based on a percentage of the household’s annual income. APPRISE conducted an evaluation of this program to assess the effectiveness of the changes to PIPP that aimed to provide incentives for regular, on-time payments and consequences for failure to adhere to program rules. The research included administrative interviews, participant surveys, and analysis of program and utility data.
PECO implemented Universal Service Programs to help low-income customers maintain energy services and protect customers’ health and safety. The programs include CAP which provides reduced payments and arrearage forgiveness, LIURP which provides energy efficiency and energy education services, CARES which provides outreach and referral services, and MEAP which provides emergency financial assistance for overdue energy bills. APPRISE conducted the 2006 and the 2012 evaluations of PECO’s Universal Services Programs. The research included review of program procedures and implementation; customer needs assessment; interviews with program managers and implementation partners; customer surveys; and an analysis of the impact of the program on energy affordability, bill payment, and collections. The work also included preparation of the Universal Services Programs Evaluation Report, submitted to the Pennsylvania Public Utilities Commission.
APPRISE is conducting an evaluation of Peoples Natural Gas Universal Service Programs.  Peoples’ Universal Service Programs provide reduced natural gas bills, arrearage forgiveness, emergency bill payment assistance, energy efficiency services, and case management services to low-income households.  The evaluation will assess the efficiency and effectiveness of these programs through background research, needs assessment research, program administrator interviews, customer surveys, program database analysis, and analysis of the impacts of the program on affordability, bill payment, and collections actions and costs.
PG Energy implemented their Customer Assistance Program (CAP), called the PG Energy Partners Program (PGEPP), to assist eligible low-income residential customers to pay their gas bills and reduce arrearages. Participating households pay a monthly bill that equals a percentage of their income, and receive monthly arrearage forgiveness when bills are paid on time and in full. APPRISE evaluated this program to determine how to increase efficiency of administration and to measure the impacts of the program on affordability, bill payment, collections activity, and gas usage.
APPRISE conducted a Process and Impact Evaluation of PGW’s Customer Responsibility Program in 2006 and 2012. This program provides low-income PGW customers with a reduced bill to increase energy affordability. The evaluations assessed the characteristics of program participants, payment compliance by program participants, and arrearage forgiveness received by program participants. The research estimated the impact of the program on affordability, payments, and collections actions and costs. Evaluation activities included interviews with program managers and staff, a survey of participating and non-participating customers, and analysis of customer billing and payment data.
APPRISE conducted a Process and Impact Evaluation of PPL Electric Utilities’ (PPL) Universal Services Programs in 2008 and in 2014. The programs provide energy efficiency services and bill payment assistance to PPL’s low-income customers. The 2014 evaluation included a needs assessment, interviews with program managers and staff, interviews with service delivery agencies, surveys with participating and non-participating customers, and analysis of the impact of the programs on energy affordability and bill payment practices.
Sacramento Municipal Utility District (SMUD) has various programs to provide assistance to low-income and special needs households. These programs include rate discounts, home weatherization services, and bill assistance. SMUD contracted with APPRISE to conduct research on SMUD’s low-income assistance programs. The study included an assessment of the characteristics and needs of SMUD’s low-income population, a benchmark study of low-income assistance programs offered by other utilities and organizations, a review of SMUD’s current low-income programs, and recommendations for program assessment metrics and potential program modifications or additions.
T.W. Phillips established a customer assistance program called the Energy Help Fund (EHF) as part of its Universal Services Programs. The EHF provides low-income customers with rate discounts and arrearage forgiveness. APPRISE conducted a Process and Impact Evaluation of the program. The evaluation included administrative interviews, customer surveys, and analysis of payment and program data.
UGI has a Customer Assistance Program (CAP) to improve energy affordability for low-income customers. The program provides a reduced natural gas bill based on the customer’s poverty level and energy usage. APPRISE conducted a Process and Impact Evaluation of this program. The research included an assessment of the program design; review of the program costs and implementation; participant and non-participant feedback; and analysis of the impacts of the program on bill payment, arrearages, and collections costs.
Verizon’s Communications Lifeline Program provides discounted telephone service to low-income customers in New Jersey. NJ SHARES has been retained by Verizon to conduct outreach for the program by providing information about the program at workshops around the state. NJ SHARES is also assisting Verizon customers with near matches of Verizon account information data and eligible program participation data to enroll in the program. NJ SHARES is attempting to enroll these customers by sending out letters, fielding response phone calls, and calling customers who do not respond to the letters. APPRISE conducted an evaluation to assess the efforts that NJ SHARES has made toward increasing enrollment in this program. The evaluation included background research, database analysis, call center observations, customer interviews, successful enrollment analysis, program penetration research, and a detailed procedures review.

APPRISE Process Evaluations – Usage Reduction Programs

AmerenUE partnered with the Missouri Department of Natural Resources Energy Center and other utilities to provide weatherization services to low-income households through the Low Income Weatherization Program (LIWP). The goal of the program is to provide energy efficient improvements to low-income customers to reduce their utility bills and help them prepare for the heating and cooling seasons. APPRISE conducted a process and impact evaluation of the weatherization program. The goals of this research were to assess the effectiveness of the program and to identify opportunities for program improvement.
APPRISE was a subcontractor to Research Into Action on the California Low-Income Energy Efficiency (LIEE) Program Process Evaluation. The CA LIEE provides no-cost energy services to low-income households including attic insulation, energy efficient refrigerators, energy efficient furnaces, weather stripping, caulking, low-flow showerheads, water heater blankets, and door and building envelope repairs. The Process Evaluation focused on client recruitment and enrollment, program coordination, and efficiency of program delivery and implementation. APPRISE was responsible for in-depth interviews with implementation contractors.
APPRISE conducted an evaluation of California’s Single-Family Affordable Solar Homes and Multi-Family Affordable Solar Homes programs. The programs provide solar incentives to customers of three investor-owned utilities in California. The programs aim to stimulate the adoption of solar power in affordable housing, improve the quality of affordable housing, increase energy affordability for low-income households, and increase awareness and knowledge of the benefits of solar power among affordable housing occupants and developers. As a subcontractor in this research, APPRISE conducted interviews with the program administrators and conducted surveys with participating homeowners, participating building managers, and nonparticipating building managers.
Colorado’s Governor’s Office of Energy Management and Conservation implemented new energy efficiency initiatives in 2006 to improve the energy efficiency of low-income homes. The initiatives include a First Response Program to provide low-cost, cost-effective energy efficiency measures and education and an E$P Plus Initiative to provide heating system and other appliance replacement. APPRISE provided evaluation research on these new programs and additional research on Colorado’s ongoing E$P weatherization services.
The 2004 Colorado REACh grant developed a comprehensive cooling program to assist Colorado’s low-income households. Local agency partners implemented cooling program services, including equipment installation and client education, and the state LIHEAP office furnished community-wide education and notification services. APPRISE conducted a comprehensive Process and Impact Evaluation of this program.
The Cool Homes Program aimed to mitigate heat-related problems of the elderly. This program provided measures to reduce summer temperatures in the home, educated customers on how to keep their homes cool, and established a buddy system that initiated contact with seniors during high-risk periods. APPRISE conducted a Process Evaluation of this program.
FirstEnergy offers a Low-Income Usage Reduction Program (LIURP) and Act 129 Low-Income Programs to assist low-income customers. The programs provide energy efficiency and energy education services, help low-income customers maintain electric service, and protect customers’ health and safety. APPRISE conducted a study to assess recent program changes, and to provide guidance to FirstEnergy on future tracking, reporting, and process improvements. The research included an assessment of what changes should be made to program procedures, whether additional measures should be added, and how projected energy savings for additional energy efficiency measures should be calculated.
First State CAA is implementing a heating replacement and energy conservation program for the Delaware Department of Health and Social Services. The Repair Replace Heaters and Conserving Energy (RRHACE) program is designed to assess the needs of low-income households and to deliver a customized set of services, including heating equipment assessment and services, Healthy Homes assessment and services, and energy education. APPRISE is advising First State on the Data Tracking System and conducting Process and Impact Evaluations. The Process Evaluation will assess challenges and opportunities, and will develop performance measures to be assessed in the Impact Evaluation.
The Illinois REACh project was an innovative pilot program aimed to help low-income households maintain energy service throughout the year and reduce the health and safety risks associated with energy service disruption. The comprehensive program worked with clients to increase resources available for paying energy bills, reduce the amount of energy used through effective energy-saving actions and participation in usage reduction programs, and develop the skills that are needed to maintain energy service through proactive management of their monthly budgets and energy bills. APPRISE conducted an evaluation of this program to assist program managers in assessing whether the program design is appropriate, whether there are any barriers to implementing the program as designed, and whether the program has achieved the intended impacts.
The Minnesota Conservation Improvement Programs (CIP) portfolios include low-income components to assist income-eligible households with energy conservation. APPRISE is conducting research through the Conservation Applied Research & Development (CARD) Grant Program to provide a comprehensive review of the low-income efficiency programs, evaluate the program processes, and assess program policies and guidelines. The goal of the research is to facilitate more effective program operations and policymaking.
APPRISE was the lead contractor for the evaluation of the National Low-Income Weatherization Assistance Program (WAP). APPRISE had three key partners for this effort – Blasnik and Associates, Dahloff Associates, and the Energy Center of Wisconsin, and was also supported by technical experts from across the country. The evaluation consisted of data collection from state WAP offices, local agencies, and participating and nonparticipating clients. Utility data was collected and analyzed for a sample of clients from every state and submetering studies were conducted to assess impacts on clients who use bulk fuels and to conduct special technical studies. The evaluation assessed program operations and delivery, energy impacts, and non-energy impacts, and made recommendations for modifications to program design and delivery.
The New Hampshire REACh Furnace Cleaning Program aimed to decrease energy usage and reduce home energy insecurity for low-income households in northern New Hampshire by cleaning furnaces and identifying furnaces in need of repair and replacement. Extensive furnace repairs and furnace replacements were also provided in those homes where the need was identified. APPRISE’s evaluation included interviews with program managers, community agency staff, and furnace cleaners; and analysis of the impact of the program on energy usage, energy insecurity, and furnace safety and efficiency indicators.
The Comfort Partners Program is a statewide collaborative utility residential low-income program, providing joint delivery of services through electric and gas utilities. Participating customers receive direct installation of energy efficiency measures, energy education, and payment plans aimed to reduce arrearages. APPRISE conducted a comprehensive evaluation of this program, including a Process Evaluation and analysis of the program’s data tracking system, the comprehensiveness of service delivery, and the impacts on energy usage and bill payment.
The NJ Comfort Partners Program provides no-cost energy efficiency services to low-income utility customers throughout the state. APPRISE conducted a comprehensive evaluation of this program. The Data Tracking Research assessed the comprehensiveness, sufficiency, and accuracy of the program database. The Process Evaluation included review and assessment of materials and procedures; interviews with utility managers; interviews and on-site observations at service delivery contractors; on-site observation of audits, installations, and third party inspections; and post-completion inspections of program installations. The Affordability Evaluation included a telephone survey with program participants and analysis of the program’s impact on affordability of energy bills and participation in the NJ Universal Service Fund program. The Impact Evaluation included analysis of program data, engineering impact analysis, usage impact analysis, and savings realization analysis. The Energy Savings Protocols Research included review and assessment of protocols currently in use to estimate program savings.
The NJ Comfort Partners Program provides no-cost energy efficiency services to low-income utility customers throughout the state. APPRISE last conducted a comprehensive evaluation of this program in 2013.  The 2017 evaluation will review program changes that have been made since the last evaluation, estimate energy savings with a billing analysis, review and make recommendations for changes to the Energy Saving Protocols, and conduct targeted participant telephone interviews and on-site inspections of completed jobs.
Niagara Mohawk’s Low-Income Customer Assistance Program (LICAP) was designed to address the problems of low-income customers who are unable to pay their full energy bills. Payment-troubled customers may receive an affordable payment arrangement, energy use management education, and a variety of energy efficiency services including weatherization and energy efficient appliance replacement. APPRISE conducted Process and Impact Evaluations of this program.
The Ohio Department of Development, Office of Energy Efficiency, has implemented programs funded by the Universal Service Fund. Participating customers receive cost-effective energy efficiency services and education targeted at reducing electric usage. APPRISE conducted a Process Evaluation of this program. The work included review of all program documents; interviews with program managers, staff, and contractors; observation of service delivery; customer surveys; contractor surveys; and economic analysis of the impacts of the program on Ohio’s economy.
Ohio’s Department of Development, Office of Community Services, was awarded a REACh grant to research how to provide additional resources to low-income weatherization services in a way that addresses the more comprehensive needs of low-income households. These funds were used to assess the in-home environmental health risks for vulnerable individuals, prioritize usage reduction measures to meet the energy and health needs of vulnerable individuals, and provide additional measures that enhance the health and safety of the home. The program also established a partnership with vulnerable households to take actions to mitigate the environmental health risks in the home and refer clients to other services that can improve the health of the home and the clients who live there. APPRISE conducted a process and impact evaluation of this project.
PECO’s Low-Income Usage Reduction Program (LIURP) provides energy efficiency services and energy education to PECO’s low-income customers to reduce their energy usage and increase the affordability of their energy bills. The evaluations estimated the impact of LIURP on participants’ energy usage, energy bills, and payment compliance and surveyed customers to analyze the impact of energy education and client behavioral changes. In the 2012 program evaluation, APPRISE also conducted observations of service delivery and inspections on a sample of completed jobs. This research provided information on potential changes to program services and implementation that may lead to increased program savings. The current evaluation of 2015 participants will assess the program process, examine the impacts of program treatments on energy usage and affordability, and survey customers to assess program satisfaction.
PPL Electric Utilities (PPL) implemented the Winter Relief Assistance Program (WRAP) in 1984 to help reduce electric bills and improve home comfort for low-income customers. The objectives of the WRAP are to reduce energy usage and bills of low-income customers and to increase low-income customers’ ability to pay their electric bills, resulting in reduced arrearages. The program also aims to increase health, safety, and comfort for low-income occupants; create and maintain partnerships with community based organizations and contractors; and make referrals to other low-income assistance programs. APPRISE conducted a comprehensive Process and Impact evaluation of PPL’s WRAP. The research consisted of administrative interviews, review of program procedures, on-site observation of service delivery, customer interviews, service delivery provider interviews, and usage and payment impact analyses.
Sacramento Municipal Utility District’s (SMUD) Low Income Weatherization Program provides no-cost energy services to qualified low-income residential customers. The goals of the program are to reduce electric bills, improve safety and comfort, provide energy education to reduce usage and lower bills, and partner with contractors and community organizations in providing these services. APPRISE’s comprehensive evaluation included needs assessment, interviews with program administrators and contractors, a participant survey, on-site inspections of completed jobs, and billing analysis to measure the impacts on energy usage.
UGI’s Low-Income Usage Reduction Program (LIURP) provides no-cost energy efficiency services to low-income customers in its service territory. APPRISE conducted a Process and Impact Evaluation of this program. The research included review and assessment of program materials and procedures, on-site interviews with UGI managers and staff, in-depth telephone interviews with a sample of program participants, and analysis of the program’s impact on the energy usage of participating customers.

APPRISE Process Evaluations – Market Transformation Programs

Columbia Gas implemented a portfolio of Demand Side Management (DSM) programs in 2009 for its residential and small commercial customers. The residential programs include a Residential Home Performance retrofit incentive program and a rebate program for customers who purchase programmable thermostats and energy-efficient showerheads. APPRISE conducted a process evaluation of these programs to help refine the programs and improve program effectiveness. The evaluation research included document review, analysis of program statistics, interviews with Columbia Gas managers and staff, interviews with program implementers, and participant and nonparticipant surveys.
Columbia Gas implemented a portfolio of Demand Side Management (DSM) programs in 2009 for its residential and small commercial customers. The Small Business Energy Solutions program is an energy efficiency improvement rebate program for small general service commercial customers. APPRISE conducted a process evaluation of this program to assess program performance issues and determine whether modifications to program design could result in improved performance. The evaluation research included review of program documents, analysis of market statistics, interviews with Columbia Gas managers and staff, interviews with program implementers, and review of other ratepayer-funded programs for the same customer segment.
The Connecticut Small Business Energy Advantage (SBEA) Program provides energy-saving services for small commercial and industrial customers. APPRISE worked with KEMA on this evaluation research and developed a better understanding of how the program can reach a broader and wider set of customers.

The Data Mining Research included analysis of the program database and the eligible customer database. This research assessed which market sectors are highly represented among the SBEA participants over the last five years, which sectors are underrepresented and which are missing; determined how the market sector distribution of program participants compares to those of nonparticipants; characterized the mix of measures installed in the program; and explored predicted savings by sector.

The Low-Income and Limited English Study explored how to increase participation by small business owners who are low-income or have limited English fluency. The study contacted organizations in the state who work with these types of small businesses, characterized the businesses and assessed eligibility for the program, and determined whether the organizations were willing to facilitate participation of these businesses in the SBEA.

APPRISE worked with a team to assess the performance of thirteen residential and small business energy efficiency programs offered by the Consolidated Edison and Orange & Rockland utility companies. The evaluation assessed how each energy efficiency program is operating and made recommendations for improvements to the implementation process. Specific information gathered via program participant and non-participant surveys included participant satisfaction, effectiveness of program marketing and delivery, barriers to participation, satisfaction with measures offered, and satisfaction with trade ally training programs.
New Jersey Natural Gas operates energy efficiency programs that are complementary to the New Jersey Clean Energy Programs (NJCEP) as the SAVEGREEN project. The residential program provides grants or on-bill repayment plans to assist with the purchase and installation of furnaces/boilers and water heaters. The commercial program provides NJCEP Direct Install participants with a no-interest on-bill repayment plan for the value of the project not covered by the NJCEP incentive. APPRISE conducted an evaluation of the SAVEGREEN project to assess program management, analyze the incremental impact on energy efficiency activity, determine the extent to which the program influences residential customer implementation of whole house improvements, and measure the program’s energy savings.
NYSERDA’s Economic Development Growth Extension (EDGE) program develops partnerships with local businesses and organizations to identify potential energy and economic development projects and to link those projects with available NYSERDA funding opportunities. To support the 2015 process evaluation of the program, APPRISE conducted a survey with EDGE program participants. This survey was designed to identify effective methods to reach potential NYSERDA customers and to collect information about participants’ experience with the program.
NYSERDA is supporting training for building code, design, and construction officials across New York State to help reach State Energy Code compliance goals. The training provides detailed information on current and planned New York State Energy Code requirements, building science principles, and technical guidance on appropriate application of these principles. APPRISE is designing the survey, coordinating with multiple training providers to administer the pre- and post-training survey to all training participants, and processing the survey data. The surveys collect information on knowledge and awareness of specific code requirements and feedback on the provided training. This research will help NYSERDA evaluate the impact of training courses and to identify future training opportunities.
PGW’s CIER provides prescriptive rebates to PGW customers who purchase large commercial and industrial natural gas boilers and commercial food service equipment to encourage the purchase of the most energy-efficient units available. APPRISE conducted an impact evaluation and a market study for the program. The impact evaluation estimated the change in energy usage for program participants due to the program, evaluated the application rejection rate, frequency of measure installation, and measure-specific savings. The market study interviewed program participants and nonparticipants, installers, and distributers to understand how participants learned about the program, incremental costs for high-efficiency equipment, availability of high-efficiency equipment, sufficiency of current rebate levels, and how the program could be modified to encourage increased participation.
PGW’s CIRI provides incentives to multi-family residential, commercial, and industrial customers to install comprehensive energy efficiency upgrades. APPRISE conducted a market study and impact evaluation for the program. The market study included interviews with program managers, technical advisors, and program participants and nonparticipants to identify how PGW could improve marketing, participation, measure selection, and financing availability. The impact evaluation assessed installed measures, estimated energy savings, and made recommendations for how greater savings may be achieved.
PGW’s HECI provides incentives to home builders and commercial developers who build properties that save a specified percentage of natural gas. PGW provides offerings for single-family residential properties and for commercial, industrial, and multi-family properties. APPRISE conducted a market study of the program to understand the marketing efforts that have been most successful, the barriers to current program participation, the adequacy of current incentive levels, and opportunities to encourage increased program participation. The market study included in-depth interviews with program administrators and technical advisors, participants and nonparticipants who expressed interest, and architects and design build firms. APPRISE also conducted an impact evaluation to estimate the impact of the program on natural gas usage.
PGW’s Home Rebate Program provides incentives to residential customers and participating Conservation Service Providers (CSPs) who complete comprehensive natural gas energy efficiency retrofits.  Financial incentives are designed to encourage customers and CSPs to pursue as many cost-effective energy efficiency measures as possible and achieve at least twenty percent energy savings on average for the program.  The market study and impact evaluation included interviews with program administrators and CSPs, in-depth interviews with partial and full program participants, analysis of the measures installed, assessment of the impact of the program on natural gas usage, and estimation of the program’s realization rate and cost-effectiveness.
PGW’s RHER provides rebates to customers who purchase residential-sized natural gas heating equipment to encourage purchase of the most efficient boilers and furnaces available. The program provides prescriptive cash rebates to offset most of the incremental cost of the higher efficiency equipment and installation. APPRISE previously conducted an impact evaluation and a market study for the program soon after the program was implemented. The impact evaluation assessed the impacts of the program on natural gas usage by measure type, efficiency, and pre-treatment usage, and compared estimated savings to deemed savings. The market study included interviews with the program manager, HVAC contractors, and program participants to assess the impact of the program on the market for high efficiency equipment, obstacles faced in program participation, and the level of financial incentives needed to encourage adoption of this equipment. APPRISE also conducted a second impact evaluation to assess the energy savings achieved by the program in the 2014 program year.
South Jersey Gas has been operating energy efficiency programs in coordination with the New Jersey Clean Energy Program (NJCEP) since 2009. The programs provide additional rebates and financing to customers who participate in the NJCEP residential and commercial and industrial programs. APPRISE conducted an evaluation of these programs to assess program design, implementation, and impacts. The research included interviews with program administrators, contractors, and participants; surveys with participants of three different programs; analysis of the impacts of the programs on natural gas usage; and assessment of the non-energy impacts of the programs.