RESEARCH ACTIVITIES – Survey Research 2017-10-04T15:22:40+00:00

Survey Research

  • Sample Frame Research: Research on potential sample frames.
  • Sample Frame Development: Compilation of data to construct the sample frame.
  • Sample Selection: Use of sophisticated statistical sample selection methods for selecting unbiased and efficient samples in single or multi-stage sampling.
  • Survey Design: Design of survey instruments and procedures for survey implementation.
  • Survey Testing: Cognitive testing of survey wording and design to ensure that results are meaningful.
  • Survey Implementation: Implementation and management of in-person, telephone, and mail surveys.
  • Survey Data Analysis: Analysis and interpretation of survey data.

 

Survey Research Outputs

  • Survey design plan
  • Survey respondent targets
  • Survey data
  • Survey analysis and results

APPRISE Survey Research – Market Transformation Programs

Commercial & Industrial

APPRISE conducted survey research to support process and impact evaluations of Alliant’s commercial and industrial energy efficiency programs. The programs included Nonresidential Custom, Nonresidential Prescriptive, and Agricultural Prescriptive. The surveys were designed to support program development and improvement.
The Commercial Buildings Energy Consumption Survey (CBECS), a quadrennial survey of commercial buildings and related energy consumption, is conducted for the U.S. Department of Energy. In this project, we designed sample frames for the 1995 and 1999 CBECS. We conducted frame design research activities; designed the area and list frame development (including a targeting sample of newly constructed buildings) and sampling procedures, direct final listing, screening, and sample selection activities; and supervised the sample selection of new buildings included in the survey.
The Commercial Buildings Energy Consumption Survey (CBECS), a quadrennial survey of commercial buildings and related energy consumption, is conducted for the U.S. Department of Energy. In this project, we consulted with Westat on the development of new procedures for the 2003 CBECS sample frame update.
APPRISE supported the Connecticut Energy Efficiency Board in research on barriers to commercial and industrial customer participation in energy efficiency programs. The research included a survey with Connecticut businesses that focused on reasons for incomplete program participation, use of financing for energy efficiency investments, and support that utilities can provide to encourage increased program participation. Information from the survey is intended to be used to refine program marketing and potentially provide additional options to reduce barriers to participation.
APPRISE worked with a team to assess the performance of thirteen residential and small business energy efficiency programs offered by the Consolidated Edison and Orange & Rockland utility companies. The evaluation assessed how each energy efficiency program is operating and made recommendations for improvements to the implementation process. Specific information gathered via program participant and non-participant surveys included participant satisfaction, effectiveness of program marketing and delivery, barriers to participation, satisfaction with measures offered, and satisfaction with trade ally training programs.
Consolidated Edison’s Small Business Direct Install Program offers free energy surveys to evaluate lighting, ballasts, fixtures, HVAC systems, and other equipment and make custom recommendations for improved energy efficiency. The program also provides free installation of low-cost energy-saving devices, including CFLs, low-flow aerators, high pressure rinse sprayers, and water heater thermostat setbacks. Up to 70% of the costs of other upgrades are covered by this program. In support of the Impact Evaluation, APPRISE managed the recruiting and scheduling of participating customer sites for inspections and lighting equipment logging.
The Federal Buildings Supplemental Survey collected building characteristics and energy data about 900 federally-owned buildings for the U.S. Department of Energy’s Energy Information Administration as a follow-up to the 1992 Commercial Buildings Energy Consumption Survey. In this project, we supervised all aspects of the survey including sample design, facility screening, survey implementation, and survey processing. The sample for the project was selected from a database of Federal Buildings. The sampling procedures included the implementation of a multistage PPS design. The interview was a complex mail/phone/mail/phone survey, where a CATI instrument was used in the final phone contact. The survey was managed with an automated project management and sample tracking system.
As part of the Green Lights program, participants agree to submit reports on lighting upgrades so that EPA can estimate the total impact of the program. There was concern by EPA that not all lighting upgrades were being reported and that the reporting system was a burden to program participants. In this project, we conducted research to determine the satisfaction of program participants with program procedures, determine the level of underreporting, make estimates of unreported program accomplishments, and design an alternative reporting system. This research showed that the recognition received by Green Lights program participants was important for some participants. It also showed that detailed reports were overly burdensome for some partners. We developed a simplified data collection system and statistical models to estimate energy savings from the more limited data.
EPA initiated the Green Lights program to reduce pollution by accelerating the adoption of energy-efficient lighting technologies. EPA provided workshops to Green Lights participants to train them on assessing potential energy savings, finding a vendor for the lighting products, and financing the purchases. Interviews with workshop participants provided information on the value of Green Lights services to participating businesses and organizations.
The Commercial and Industrial Evaluation Contract (CIEC) Team has collaborated with the Massachusetts Program Administrators (PAs), Energy Efficiency Advisory Council (EEAC) Consultants and the National Building Institute (NBI) to assess the levels of energy code compliance with various code provisions. The Code Compliance project was intended to perform code compliance assessment on active construction sites and to provide supporting data analysis for the development and implementation of energy code-based efficiency programs. APPRISE participated in the development of the sample design and managed the recruitment efforts for on-site inspections of active construction sites.
APPRISE managed a scoping study for the Supply-Side Market Actors Project. This study assessed the feasibility and efficacy of using Program Administrator (PA) program databases, commercially available business data sources, and targeted in-depth interviews to characterize the population of supply-side market actors, develop market actor sample frames, and implement market actor panels.
The Massachusetts Large Commercial & Industrial Evaluation is a multi-client, statewide project that aims to characterize adoption of energy efficient equipment and design and assess market effects. As part of the evaluation team, APPRISE provided advice, guidance, and expertise on the sample development, data collection, and analysis that was performed in characterization of the large commercial and industrial new construction market.
APPRISE supported the New Jersey Board of Public Utilities (BPU) in research to assess the economic impacts of New Jersey’s Clean Energy Programs. APPRISE’s work included a survey with participants in New Jersey’s Clean Energy program to assess market characteristics, qualify the nature and number of jobs created in New Jersey, and to better understand the State’s clean energy economy. The project also included in-depth interviews with companies that participate in New Jersey’s Clean Energy economy to develop additional information on the perception of these programs and future clean energy plans.
The Business Partners Program is an integration of three prior NYSERDA programs: Motors Systems, Commercial Lighting, and Building Performance and HVAC. The Business Partners program promotes the purchase and installation of energy efficient products and services by working with trade allies including building and systems contractors, distributors, vendors, designers and energy service providers. APPRISE conducted surveys of trade allies and end-use customers to assess market penetration and net program savings.
The New York State Energy Research and Development Authority (NYSERDA) is conducting a study about the energy management practices, priorities, and needs of companies that operate in the industrial and manufacturing sectors. APPRISE supported this work with a baseline Continuous Energy Improvement (CEI) Survey that gathered information about the penetrations of CEI in industrial and manufacturing facilities in New York State. APPRISE conducted telephone interviews with management-level contacts to collect information on energy management attitudes, planning and implementation of strategic energy management, barriers to and interest in strategic energy management, and drivers in business decisions. The survey helped NYSERDA understand how these companies and facilities manage their energy use, what resources they have and they need, and how NYSERDA can design programs to assist these facilities in achieving greater efficiency and reduced costs.
NYSERDA’s Economic Development Growth Extension (EDGE) program develops partnerships with local businesses and organizations to identify potential energy and economic development projects and to link those projects with available NYSERDA funding opportunities. To support the 2015 process evaluation of the program, APPRISE conducted a survey with EDGE program participants. This survey was designed to identify effective methods to reach potential NYSERDA customers and to collect information about participants’ experience with the program.
NYSERDA’s Emerging Technologies and Accelerated Commercialization (ETAC) Program provides funding and support to building owners and technology developers who implement successful demonstrations of new energy-saving technologies. NYSERDA’s ETAC program is designed to increase adoption of emerging technologies in New York State, including energy efficient lighting and networked lighting controls. APPRISE conducted a survey with lighting end-users and a survey with lighting installation contractors to understand and characterize the adoption of solid-state lighting (SSL) in New York State. Both surveys collected information on awareness and use of new lighting technologies and the impact of several factors on lighting selection decisions. APPRISE developed the sampling approach, conducted the interviews, and prepared the final data for analysis.
APPRISE conducted survey research to support NYSERDA’s process evaluation of their Enhanced Commercial Industrial Performance Program (ECIPP). The ECIPP provides incentives to businesses for energy efficiency projects. APPRISE’s survey research assessed how clients learned about ECIPP, previous client experience with energy efficiency projects, previous client experience with the ECIPP, and client satisfaction with the ECIPP process.
NYSERDA’s Environmental Monitoring, Evaluation, and Protection (EMEP) Program aims to facilitate the communication of scientific and technological information regarding the environmental impacts of energy choices to inform public policy. APPRISE conducted surveys of EMEP’s policy maker, legislative, and environmental group contacts in support of the Process Evaluation. This survey research will aid EMEP in enhancing the efficiency and effectiveness of the program.
The Existing Facilities Program (EFP) is a consolidation of two prior NYSERDA programs — the Peak Load Management Program (PLMP) and the Enhanced Commercial and Industrial Performance Program (ECIPP). The EFP offers prescriptive and performance-based incentives for energy-efficient upgrades to existing buildings. The EFP targets a range of customer sectors including commercial and industrial businesses, healthcare facilities, universities and colleges, state and local governments, and mission critical facilities such as data centers and communications facilities. APPRISE conducted surveys of Energy Service Companies (ESCOs) and end-users that will support estimates of program savings, free ridership, and spillover effects.
NYSERDA’s Flexible Technical Assistance Program (Flex Tech) is a consolidation of what were formerly the Flex Tech, Technical Assistance, and Energy Audit Programs. Flex Tech is designed to provide customers objective and customized energy information that will facilitate the implementation of cost-saving energy-efficient measures. APPRISE conducted survey research to support NYSERDA’s Market Characterization and Assessment, Impact, and Process evaluations of the Program. APPRISE conducted customer and contractor surveys for participating and non-participating market sectors to evaluate the program processes, assess the market climate, and establish estimates of energy savings that can be attributed to the program.
NYSERDA’s Industry and Process Efficiency Program (IPEP) provides incentives to offset the costs of projects focused on increasing productivity, increasing process efficiency, reducing waste, and reducing electricity consumption. The program focuses on key manufacturing sectors in New York. APPRISE surveyed both eligible end-use customers and technical service providers to gather information on customers’ and providers’ decision-making, awareness, and perceptions of energy efficiency opportunities.
NYSERDA’s Innovation Capacity and Business Development Program (ICBD) is designed to provide financial and professional resources to new businesses and entrepreneurs operating in New York State’s clean energy sector.  The ICBD program supports NYSERDA’s clean technology Incubators, special Proof-of-Concept centers, and Entrepreneurs-In-Residence Program, all of which provide business development services to clean energy business start-ups.  APPRISE conducted surveys with recipients of ICBD services and with other emerging market actors.  Both surveys studied perceptions of market barriers and drivers, market trends, and market stakeholders.  APPRISE designed the web instrument, conducted phone interviews, and prepared the final data for analysis.

NYSERDA’s New Construction Program provides non-residential customers with technical assistance services and capital-cost incentives for implementing energy efficiency improvements in new construction or in substantially renovated buildings. The technical assistance provides cost-shared analysis to customers and their design teams (Architecture and Engineering firms) to identify energy efficiency opportunities for their projects. The program is designed to encourage the incorporation of energy efficiency, renewable energy, and green building features in the design, construction, and operation of commercial, industrial, and institutional buildings.

APPRISE conducted survey research to support NYSERDA’s market characterization research for the New Construction Program (NCP) in 2008. This research gathered information from participating and non-participating building owners and participating and non-participating architecture and engineering firms to assess their views, experiences, and perceptions on a variety of topics related to program participation.

APPRISE conducted surveys in 2011 with non-participating building owners and design firms to assess awareness of NYSERDA’s program offerings and energy efficiency measures in the non-residential new construction sector in New York State, to develop baseline data for non-residential new construction in New York State, and to assess non-participant spillover effects of the New Construction Program.

NYSERDA is interested in working with Commercial Property Managers in the NYC region to identify ways to increase adoption of energy efficiency measures by building owners and building tenants in the NYC region. APPRISE conducted structured in-depth interviews with the major Commercial Building Property Managers working in the NYC region to develop a better understanding of their business models, document their current activities with respect to the delivery of energy efficiency services, and explore their interest in partnering with NYSERDA to reach out to their customers. Through this research, APPRISE was able to give NYSERDA a better understanding of the opportunities for and barriers to investment in energy efficiency measures by building owners and tenants, and the role that Property Managers could play in facilitating participation by their owners and tenants in NYSERDA programs.
NYSERDA is interested in working with Demand Response Providers in the NYC region to leverage the provider/customer relationship and increase the adoption of energy efficiency measures. APPRISE conducted structured in-depth interviews with the major Demand Response Providers working in the NYC region to develop a better understanding of their business models, document their current activities with respect to the delivery of energy efficiency services, and explore their interest in partnering with NYSERDA to reach out to their customers. Through this research, APPRISE was able to segment the Demand Response Provider market, identify those provider segments that presented NYSERDA the best access to new customers, and develop a strategy that NYSERDA could use to increase their penetration in the NYC market.
NYSERDA is interested in working with ESCOs (electric commodity suppliers) in the NYC region to leverage the ESCO/customer relationship and increase the adoption of energy efficiency measures. APPRISE conducted structured in-depth interviews with the major ESCOs working in the NYC region to develop a better understanding of their business models, document their current activities with respect to the delivery of energy efficiency services, and explore their interest in partnering with NYSERDA to reach out to their customers. Through this research, APPRISE was able to help NYSERDA segment the ESCO market and identify potential strategies for working with each market segment.
NYSERDA is conducting a review of program spillover effects generated by those not receiving incentives from NYSERDA programs. This cross-program evaluation effort will examine these effects in an attempt to quantify net-to-gross savings estimates. In support of this evaluation, APPRISE conducted surveys of program non-participants in the Commercial and Industrial sectors.
The New York State Energy Research and Development Authority (NYSERDA) supports initiatives to encourage commercial adoption of energy efficient technologies through demonstration and replication (R&D) projects. APPRISE conducted in-depth telephone interviews with R&D grant recipients to support a process and impact evaluation of the program. The in-depth interviews with Principal Investigators were designed to identify and estimate environmental benefits, cost-effectiveness, and other impacts. The interviews also collected process evaluation information including participant awareness of NYSERDA programs, effectiveness of program communication, adequacy of assistance provided by program staff, and efficiency of the funding process. The 2016 study examined projects completed between 2011 and 2013. A previous study conducted in 2013 examined projects completed between 2008 and 2010.
NYSERDA’s Research Development (R&D) Program supports the development and commercialization of innovative energy and environmental products, technologies, and processes. In 2008, APPRISE conducted a survey of firms that reported product sales and made recoupment payments based upon their R&D recoupment contracts with NYSERDA. The purpose of the survey was to obtain information about the products for which NYSERDA provided technical expertise and funding. The survey collected information on product benefits, sales revenue, sources and magnitude of investment capital, impact of NYSERDA funding on product development and commercialization, and total development and commercialization costs. The information will be used to evaluate the impacts and value of NYSERDA funding. In 2010, APPRISE conducted surveys of integrators, vendors and site owners involved in the development of R&D program-funded demonstration projects and worked on a Process Evaluation of the R&D program by conducting surveys of respondents and non-respondents to R&D program solicitations.
NYSERDA’s Transportation Program provides funding to support new transportation technologies and approaches that reduce greenhouse gas emissions, including renewable energy infrastructure, innovative public transportation designs, and traffic demand management approaches.  The Program targets financial support for transportation product development, demonstration, and deployment throughout New York State.  APPRISE conducted a survey with business participants and transportation companies to understand and characterize the New York State transportation sector.  This survey collected information on the types of technologies in development, major actors in New York’s transportation market, and experiences with NYSERDA.  APPRISE designed the web instrument, conducted phone interviews, and prepared the final data for analysis.

APPRISE Survey Research – Market Transformation Programs

Residential

APPRISE conducted survey research to support process and impact evaluations of Alliant’s residential energy efficiency programs. The programs included Appliance Recycling, Direct Load Control, Home Energy Audits, Residential Prescriptive, New Home Construction, and Energy Awareness and Education. The surveys were designed to support program development and improvement.
APPRISE is conducting a process evaluation to understand the contributions that Avangrid’s marketing programs make to encourage residential and non-residential customers to participate in Avangrid’s energy efficiency program offerings. The research will include in-depth interviews and surveys with program participants to assess the success of various marketing approaches.
The Commercial Buildings Energy Consumption Survey (CBECS), a quadrennial survey of commercial buildings and related energy consumption, is conducted for the U.S. Department of Energy. In this project, we designed sample frames for the 1995 and 1999 CBECS. We conducted frame design research activities; designed the area and list frame development (including a targeting sample of newly constructed buildings) and sampling procedures, direct final listing, screening, and sample selection activities; and supervised the sample selection of new buildings included in the survey.
Columbia Gas implemented a portfolio of Demand Side Management (DSM) programs in 2009 for its residential and small commercial customers. The residential programs include a Residential Home Performance retrofit incentive program and a rebate program for customers who purchase programmable thermostats and energy-efficient showerheads. APPRISE conducted a process evaluation of these programs to help refine the programs and improve program effectiveness. The evaluation research included document review, analysis of program statistics, interviews with Columbia Gas managers and staff, interviews with program implementers, and participant and nonparticipant surveys.
APPRISE is conducting research for the Connecticut Energy Efficiency Board to explore existing NEI values nationally, regionally, and for CT; gaps in existing CT NEI data; and survey research and data analysis that can address NEI measurement.  The research will include implementation of a program participant survey for one priority program or measure.  The study will explore and assess the current state of the art for measuring NEIs both nationally and regionally, where and how NEIs are used in calculating the cost-effectiveness of energy efficiency programs, how the inclusion of NEIs has impacted energy efficiency programs across the U.S., the best available estimates of the NEIs for CT programs and measures, and how NEIs could be measured and incorporated in future CT evaluations.
Consolidated Edison’s Appliance Recycling Program provides financial incentives to encourage residential electric customers living in one-to-four unit homes to recycle their freezers, second refrigerators, and room air conditioners. APPRISE supported the Impact Evaluation of this program by facilitating on-site data collection. Data collected included the location of the second refrigerator, how the second refrigerator is used, and metered energy usage for that refrigerator. Data from the site visits was used to analyze the impact of the program on energy use.
APPRISE worked with a team to assess the performance of thirteen residential and small business energy efficiency programs offered by the Consolidated Edison and Orange & Rockland utility companies. The evaluation assessed how each energy efficiency program is operating and made recommendations for improvements to the implementation process. Specific information gathered via program participant and non-participant surveys included participant satisfaction, effectiveness of program marketing and delivery, barriers to participation, satisfaction with measures offered, and satisfaction with trade ally training programs.
Consolidated Edison’s Multi-Family Energy Efficiency Program provides financial incentives for the installation of energy efficiency measures in common areas and within apartments, in multi-family buildings with 5-75 units. Prescriptive rebates are offered for common area measures including HVAC systems and controls, insulation, and lighting. No-cost unit-level measures include compact fluorescent lights, low-flow showerheads, faucet aerators and smart power strips. APPRISE supported the Impact Evaluation of this program by facilitating individual unit on-site data collection. Data collected included the location of the apartment within the building, the number of light fixtures in the apartment, and metered CFL and smart strip usage. Data from the site visits was used to analyze the impact of this program on energy use.
Consolidated Edison’s Residential HVAC Program provides financial incentives to customers to encourage the installation of select high-efficiency equipment and home energy improvements. APPRISE conducted research to assess the impact of this program. This work included recruiting, scheduling, and conducting site visits to meter central air conditioners and ductless heat pumps installed through this program. Resulting usage data will be analyzed to estimate the impact of this program on energy use.
Duquesne Light’s Upstream Lighting program works with residential lighting retailers to offer discounts on CFLs and LEDs at the point of purchase. Duquesne Light is conducting an enhanced net-to-gross analysis of their upstream lighting program. This research includes in-store intercepts with customers, in-depth interviews with lighting manufacturers and retailers, and telephone interviews with Duquesne Light’s residential customers. APPRISE conducted the general population telephone survey of a random sample of Duquesne Light’s residential customers to ascertain the net-to-gross ratio, in service rate, residential versus non-residential installation allocation, and low-income versus non low-income household savings allocation.
EPA initiated Energy Star programs to reduce pollution by accelerating the adoption of new and underutilized energy-efficient technologies. In this evaluation we conducted surveys with home builders and home buyers. We conducted a home builder mail survey with participating Energy Star builders on their satisfaction with services received through the program. The survey also collected information to assess the impact of the program on the behavior of home builders. In the home buyers survey, we collected baseline and follow-up information from a sample of Energy Star home buyers and a comparison sample of other new home buyers. We obtained energy billing data for responding households and developed estimates of energy savings resulting from the Energy Star Homes Program.
The Energy Star HVAC Program works to overcome barriers associated with the purchase of higher efficiency replacement HVAC equipment in the residential market. EPA works with HVAC equipment manufacturers, distributors, dealers, utilities, and financial institutions to overcome the market barriers. In this evaluation, we conducted in-depth interviews with all participants in the HVAC program, including trade associations, utilities, finance partners, manufacturers, regional distributors, dealers, and consumers.
APPRISE was a subcontractor on the evaluation of the Massachusetts ENERGY STAR Lighting Program. This statewide program provides incentives for residential customers to purchase ENERGY STAR qualified lighting products. APPRISE conducted telephone surveys with comparison residential customers in South Dakota to assess the market effects of the program.
The New Jersey Board of Public Utilities manages a portfolio of energy efficiency programs. These programs include the Residential HVAC, Residential New Construction, Energy Star Products, Commercial and Industrial New Construction, Commercial and Industrial Retrofit, Combined Heat and Power, and Customer On-Site Renewable Energy. As part of a comprehensive evaluation of the portfolio of programs, APPRISE conducted an evaluation of the Residential New Construction Program, known as the NJ Energy Star Homes Program. Under this program, builders receive incentives to develop homes that are at least 15 percent more energy efficient than a standard home. APPRISE conducted research to update how estimates of the program’s energy savings are made and to estimate actual program energy and demand savings from 2001 through 2006. The research included a survey with Energy Star homebuyers and non-participating homeowners and analysis of electric and gas energy usage data for these households.
New Jersey Natural Gas operates energy efficiency programs that are complementary to the New Jersey Clean Energy Programs (NJCEP) as the SAVEGREEN project. The residential program provides grants or on-bill repayment plans to assist with the purchase and installation of furnaces/boilers and water heaters. The commercial program provides NJCEP Direct Install participants with a no-interest on-bill repayment plan for the value of the project not covered by the NJCEP incentive. APPRISE conducted an evaluation of the SAVEGREEN project to assess program management, analyze the incremental impact on energy efficiency activity, determine the extent to which the program influences residential customer implementation of whole house improvements, and measure the program’s energy savings.
Under New Jersey restructuring legislation, all utility DSM programs are implemented through a statewide utility collaborative. The New Jersey RNC Working Group was responsible for designing a statewide RNC program to encourage builders to construct and home buyers to purchase energy-efficient homes. APPRISE staff directed the RNC Baseline Study, including surveys with market actors to assess awareness of and attitudes toward RNC programs, and the development of Residential New Construction statistics and information on the Affordable Housing market.
NYSERDA is supporting training for building code, design, and construction officials across New York State to help reach State Energy Code compliance goals. The training provides detailed information on current and planned New York State Energy Code requirements, building science principles, and technical guidance on appropriate application of these principles. APPRISE designed the survey, coordinated with multiple training providers to administer the pre- and post-training survey to all training participants, and processed the survey data. The surveys collected information on knowledge and awareness of specific code requirements and feedback on the provided training. This research helped NYSERDA evaluate the impact of training courses and to identify future training opportunities.
The goal of NYSERDA’s ENERGY STAR Homes Program is to implement and promote an enhanced version of the U.S. Environmental Protection Agency’s ENERGY STAR Program in New York State. The program provides technical assistance and targeted financial incentives to residential builders and income-eligible homeowners to encourage the adoption of energy efficient design features and the selection and installation of energy efficient equipment. APPRISE conducted survey research to support NYSERDA’s market characterization research for the ENERGY STAR Homes Program (ESH) in 2008. APPRISE gathered information from homeowners who purchased an Energy Star Home and homeowners who purchased a non-Energy Star Home to examine their views, experiences, and perceptions on a variety of topics related to program participation. APPRISE also surveyed home builders who participated and homebuilders who did not participate in the ESH to assess their views, experiences, and perceptions on a variety of topics related to program participation. APPRISE conducted additional surveys with participating homeowners, non-participating homeowners, participating builders, non-participating builders, and formerly-participating builders to supplement energy savings estimates and provide information on potential program free-ridership and spillover effects.
NYSERDA’s Energy Star Products Program (NYESP) seeks to increase sales of residential energy-efficient appliances, lighting, power management, home electronics, heating, ventilation and air conditioning equipment. The goals of the program are to increase the supply of products through partnerships with retailers, manufacturers and distributors and to create demand for high-efficiency and ENERGY STAR products through increased consumer awareness and understanding of the ENERGY STAR label. APPRISE surveyed recent purchasers and appliance retailers of refrigerators, dishwashers, clothes washers, or room air conditioners and assessed the purchases, sales, availability, attitudes, and awareness of Energy Star products.
NYSERDA’s Green Jobs – Green New York (GJGNY) Program provides New Yorkers with access to energy audits, installation services, low-cost financing, and pathways to training for various green-collar careers. Evaluation activities were designed to assess the effectiveness, progress, and outcomes related to the program and to provide recommendations for program refinement. APPRISE conducted surveys with program participants and non-participants to develop a comprehensive understanding of current and emerging markets and to provide baseline data to enable NYSERDA to define, deliver, and evaluate programs for these target markets.
NYSERDA’s Home Performance with Energy Star (HPwES) Program provides low-cost training to contractors and financial incentives to homeowners to help improve the energy efficiency of the housing stock in New York. APPRISE conducted survey research to support NYSERDA’s market characterization research for the program in 2008. APPRISE’s research gathered information from homeowners who received a Comprehensive Home Assessment from a participating BPI-certified technician to examine homeowner views, experiences, and perceptions on a variety of topics related to program participation. APPRISE also surveyed program participating and non-participating contractors to examine contractor views, experiences, and perceptions on a variety of topics related to program participation. APPRISE also completed the HPwES Impact Evaluation, and conducted surveys of homeowners and contractors to supplement program net-to- gross savings estimates.
APPRISE conducted RDD telephone surveys in NY State, New York City, Ohio, Washington DC, and Houston in 2009 to assess the saturation of CFLs. APPRISE then recruited the RDD survey respondents for in-home audits of lighting fixtures, existing bulbs, and home electronics bundles in the summer of 2009. In the summer of 2010, APPRISE conducted another RDD telephone survey in NY State, New York City, and Houston, and recruited respondents for a follow-up in-home audit. The data are being used by CFL Program staff to identify marketing opportunities and by NYSERDA Evaluation staff to measure program impacts.
NYSERDA conducted a market characterization study of the lighting market in New York to inform a possible Enhanced Lighting Design Program under SBC IV. APPRISE conducted quantitative surveys with retailers, homeowners, and the general population to develop a better understanding of the residential lighting market and identify opportunities for optimizing NYSERDA’s future investments in the lighting market.
PGW’s RHER provides rebates to customers who purchase residential-sized natural gas heating equipment to encourage purchase of the most efficient boilers and furnaces available. The program provides prescriptive cash rebates to offset most of the incremental cost of the higher efficiency equipment and installation. APPRISE previously conducted an impact evaluation and a market study for the program soon after the program was implemented. The impact evaluation assessed the impacts of the program on natural gas usage by measure type, efficiency, and pre-treatment usage, and compared estimated savings to deemed savings. The market study included interviews with the program manager, HVAC contractors, and program participants to assess the impact of the program on the market for high efficiency equipment, obstacles faced in program participation, and the level of financial incentives needed to encourage adoption of this equipment. APPRISE also conducted a second impact evaluation to assess the energy savings achieved by the program in the 2014 program year.
Puget Sound Energy (PSE) began mailing Opower Home Energy Reports (HERs) to approximately 40,000 customers in November 2008. These reports deliver customized, household-level data on energy usage in the home, and encourage energy-efficient behaviors and measures. Savings for participants in the HER Program may be a direct result of the HER report or may result from other energy efficiency programs, such as CFL buy downs. To identify potential double-counting of energy savings, APPRISE conducted survey research as part of a three-year Home Energy Report (HER) Impact Evaluation. Fifteen hundred interviews were completed with current HER recipients, past recipients, and a comparison group of customers who never received a report. The survey collected data on purchases made and energy usage behaviors adopted by respondents across all three groups to enable estimation of the impact of other energy programs and avoid double counting of these savings.
APPRISE was a subcontractor on the evaluation of the Rhode Island ENERGY STAR Lighting Program. This statewide program sponsored by National Grid provides discounts for residential customers to purchase ENERGY STAR-qualified lighting products. APPRISE conducted telephone surveys with residential customers in Rhode Island to assess the market effects of the program.
APPRISE partnered with two natural gas utilities in the Northeast to study current thermostat technology, setback behavior, and opportunities for smart thermostats.  A survey was conducted with a random sample of 385 residential customers to develop a better understanding of which households may benefit from smart thermostats and assess the opportunities for energy savings.  The survey found that there is a great opportunity for smart thermostats, as only four percent of respondents had a Wi-Fi connected thermostat, but 74 percent had the technology needed for a smart thermostat.  Lower income households may present the greatest opportunity for savings with smart thermostat technology.  Over half of these customers had the needed technology, but they were less likely to have programmable thermostats.  Households show opportunities for benefitting from smart thermostats that adjust to their usage preferences, as about half of the respondents reported frequent manual adjustment of programmable thermostats.  Many reported that they would make extensive use of smart thermostat features if they were available.  Programs and incentives could potentially have a large impact on the adoption of smart thermostats.
South Jersey Gas has been operating energy efficiency programs in coordination with the New Jersey Clean Energy Program (NJCEP) since 2009. The programs provide additional rebates and financing to customers who participate in the NJCEP residential and commercial and industrial programs. APPRISE conducted an evaluation of these programs to assess program design, implementation, and impacts. The research included interviews with program administrators, contractors, and participants; surveys with participants of three different programs; analysis of the impacts of the programs on natural gas usage; and assessment of the non-energy impacts of the programs.
Sustainable Princeton introduced the EnergySmart Homes Campaign to help Princeton residents reduce their energy usage. Through this program, Sustainable Princeton partnered with a BPI-certified contractor to provide low-cost home energy audits and encourage the installation of energy saving measures. APPRISE conducted a survey with EnergySmart Homes participants to explore the quality of energy education and services delivered by the auditor, why customers did or did not install measures, and what actions could be taken to help customers overcome barriers to measure installation.

APPRISE Survey Research – Payment Programs

APPRISE survey research on payment program evaluations includes survey design, survey implementation, and survey data analysis.

APPRISE conducted a comprehensive evaluation of Allegheny Power’s Universal Service Programs. The evaluation research included interviews with Allegheny and agency program managers and staff, surveys with participating and nonparticipating customers, analysis of program databases, and analysis of the impacts of the payment assistance program on energy affordability, payment compliance, arrearages, and collections costs.
Ameren introduced a pilot of a low-income payment assistance program that provides electric bill payment assistance to low-income customers. Participants receive monthly bill credits toward their electric bill and additional credits toward their arrearages. APPRISE provided an evaluation of this program in 2012 that included interviews with local agencies that conduct intake for the program, a survey of participating customers, analysis of program data, and analysis of program impacts on affordability, payment coverage, and terminations. Ameren made program modifications in response to findings from the initial evaluation. APPRISE conducted a follow-up Process and Impact Evaluation of the program in 2014 that documented the impacts of the program changes. APPRISE conducted a third evaluation of this program in 2016 that included interviews with defaulted customers, analysis of program data, and analysis of the impacts of the program on affordability and bill payment. Findings from this research were used to further refine the program.
Baltimore Gas and Electric (BGE) designed and implemented pilot programs to determine the effect of energy bill discounts and payment counseling on the payment timeliness of limited-income customers. The Graduated Rate Discount (GRAD) Program provided a range of usage-based discounts. Some of the customers also received a home energy audit or payment counseling phone calls. The Customer Assistance Maintenance Program (CAMP) pilot tested changes to BGE’s CAMP including increased bill credits, payment counseling, or both. APPRISE’s process and impact evaluation assessed the effectiveness of these pilots in increasing customers’ on-time bill payment and energy conservation behavior.
APPRISE conducted research to assist the Colorado’s Governor’s Commission on Low-Income Energy Assistance to develop recommendations to improve the effectiveness and efficiency of the state’s low-income energy assistance services. The research included analysis of American Community Survey data, the Colorado LIHEAP database, and the Energy Outreach Colorado database; review and assessment of current LIHEAP procedures and implementation; surveys with participating and non-participating low-income households; and interviews with program managers at the state and agency level.
Duquesne Light implemented Universal Service Programs to improve affordability for low-income customers. The programs include the Customer Assistance Program (CAP), which provides reduced payments and arrearage forgiveness; Smart Comfort, which provides energy efficiency and energy education services; CARES, which provides outreach and referral services; and the Hardship Fund, which provides emergency assistance. APPRISE conducted an evaluation of the performance of these programs and the integration among the different programs. The research included interviews with program and agency staff, a participant survey, and analysis of the impacts of the program on affordability and payment compliance.
FirstEnergy implemented Universal Service Programs to help low-income customers maintain electric service and protect customers’ health and safety. The programs include the CAP Program, which provides reduced payments and arrearage forgiveness; LIURP/WARM, which provides energy efficiency and energy education services; CARES, which provides outreach and referral services; the Hardship Fund, which provides emergency assistance; and Gatekeeper, where field personnel recognize and report customers who may be in distress. APPRISE conducted an evaluation of the performance of these programs, their ability to help low-income customers maintain affordable service, and the integration among the different programs. This evaluation included analysis of program databases, interviews with FirstEnergy managers and staff, interviews with staff at organizations that implement the programs; and analysis of the impact of CAP on bills, customer payments, and affordability.
The Illinois Department of Healthcare and Family Services (HFS) has designed the Percent of Income Payment Plan (PIPP) pilot to help low-income households maintain energy service throughout the year and reduce the health and safety risks associated with energy service disruption. The PIPP program offers LIHEAP recipients arrearage forgiveness and a fixed payment plan in the place of a one-time LIHEAP grant. It also tests the effectiveness of offering weatherization and energy conservation, as well as counseling and education services to program participants. It is important for HFS to have information on the appropriateness of the program development, the effectiveness of the program implementation, and the program impact on participating clients and the participating utility. APPRISE conducted a process and impact evaluation of this program that assessed whether the program was implemented effectively and whether the program achieved its intended effect on clients.
Louisville Gas and Electric Company and Kentucky Utilities Company offer a Home Energy Assistance (HEA) Program to assist the poorest households in their service territories least able to afford to pay their energy bills. Funded by ratepayers and shareholders, the programs (structured differently by company) provide fixed monthly credits to LIHEAP participants. APPRISE conducted a comprehensive HEA assessment to analyze the program management, availability, and customer participation; assess impacts on payments, arrearages, collections actions, and weatherization participation; and make recommendations for program improvement.
APPRISE conducted surveys of LIHEAP recipients in 2003, 2005, 2008, 2009 and 2011. APPRISE recently conducted a survey of 2011 recipients to update information about LIHEAP-recipient households that was collected in the previous surveys. These surveys documented changes in the affordability of energy bills, the need for LIHEAP, and the choices that low-income households make when faced with unaffordable energy bills. The 2011 Survey interviewed recipients in 13 states. Stratified samples of fiscal year 2011 LIHEAP recipients were chosen from each of the 13 state LIHEAP databases. The study characterized the LIHEAP population for 2011 and documented the challenges that these households face in addition to their energy bills, including unemployment, lack of health insurance, and medical issues.
APPRISE assessed the fiscal integrity and operational efficiency of the NJ LIHEAP and USF Programs. This research included an assessment of the performance of the New Jersey Department of Community Affairs (DCA), Division of Housing and Community Resources (DHCR), Office of Home Energy Assistance (OHEA) and an assessment of the operations of the LIHEAP and USF programs. We reviewed program documents, and conducted interviews with HHS LIHEAP compliance staff, DCA management, USF managers at the NJ Board of Public Utilities, and managers at Community Based Organizations that administer the programs. We also reviewed program databases and statistics, conducted on-site interviews and observations with a sample of local agencies, and conducted a survey with LIHEAP recipients.
The New Jersey Statewide Heating Assistance and Referral for Energy Services (NJ SHARES) is a nonprofit corporation organized to provide assistance to individuals and families living in New Jersey who are in need of temporary help paying their energy bills. APPRISE conducts an annual impact evaluation of the New Jersey SHARES program. The evaluation includes analysis of administrative and utility data to assess the program’s impact on maintenance of electric and gas service and on energy bill payment.  Some of the previous evaluations also included surveys and in-depth participant interviews to assess whether the crises had been resolved, what additional assistance was needed, and the interest and potential for energy efficiency program participation.   APPRISE also serves as an advisor to the NJ SHARES Board of Directors.
The New Jersey Universal Service Fund (USF) program provides fixed credits to eligible low-income families to limit electric and gas bills to six percent of household income. APPRISE conducted a process and impact evaluation of this program. This evaluation provided information on the benefits and costs of the USF program and alternatives for program design and operations. Program evaluation research consisted of administrative interviews, client surveys, program and utility data analysis, and financial data analysis.
Ohio’s Percentage of Income Payment Program (PIPP) was revised as the PIPP Plus in November 2010. The program helps low-income households maintain utility service by providing a reduced bill based on a percentage of the household’s annual income. APPRISE conducted an evaluation of this program to assess the effectiveness of the changes to PIPP that aimed to provide incentives for regular, on-time payments and consequences for failure to adhere to program rules. The research included administrative interviews, participant surveys, and analysis of program and utility data.
PECO implemented Universal Service Programs to help low-income customers maintain energy services and protect customers’ health and safety. The programs include CAP which provides reduced payments and arrearage forgiveness, LIURP which provides energy efficiency and energy education services, CARES which provides outreach and referral services, and MEAP which provides emergency financial assistance for overdue energy bills. APPRISE conducted the 2006 and the 2012 evaluations of PECO’s Universal Services Programs. The research included review of program procedures and implementation; customer needs assessment; interviews with program managers and implementation partners; customer surveys; and an analysis of the impact of the program on energy affordability, bill payment, and collections. The work also included preparation of the Universal Services Programs Evaluation Report, submitted to the Pennsylvania Public Utilities Commission.
APPRISE conducted an evaluation of Peoples Natural Gas Universal Service Programs.  Peoples’ Universal Service Programs provide reduced natural gas bills, arrearage forgiveness, emergency bill payment assistance, energy efficiency services, and case management services to low-income households.  The evaluation assessed the efficiency and effectiveness of these programs through background research, needs assessment research, program administrator interviews, customer surveys, program database analysis, and analysis of the impacts of the program on affordability, bill payment, and collections actions and costs.
PG Energy implemented their Customer Assistance Program (CAP), called the PG Energy Partners Program (PGEPP), to assist eligible low-income residential customers to pay their gas bills and reduce arrearages. Participating households pay a monthly bill that equals a percentage of their income, and receive monthly arrearage forgiveness when bills are paid on time and in full. APPRISE evaluated this program to determine how to increase efficiency of administration and to measure the impacts of the program on affordability, bill payment, collections activity, and gas usage.
APPRISE conducted a Process and Impact Evaluation of PGW’s Customer Responsibility Program in 2006 and 2012. This program provides low-income PGW customers with a reduced bill to increase energy affordability. The evaluations assessed the characteristics of program participants, payment compliance by program participants, and arrearage forgiveness received by program participants. The research estimated the impact of the program on affordability, payments, and collections actions and costs. Evaluation activities included interviews with program managers and staff, a survey of participating and non-participating customers, and analysis of customer billing and payment data.
APPRISE conducted a Process and Impact Evaluation of PPL Electric Utilities’ (PPL) Universal Services Programs in 2008 and in 2014. The programs provide energy efficiency services and bill payment assistance to PPL’s low-income customers. The 2014 evaluation included a needs assessment, interviews with program managers and staff, interviews with service delivery agencies, surveys with participating and non-participating customers, and analysis of the impact of the programs on energy affordability and bill payment practices.
T.W. Phillips established a customer assistance program called the Energy Help Fund (EHF) as part of its Universal Services Programs. The EHF provides low-income customers with rate discounts and arrearage forgiveness. APPRISE conducted a Process and Impact Evaluation of the program. The evaluation included administrative interviews, customer surveys, and analysis of payment and program data.
UGI has a Customer Assistance Program (CAP) to improve energy affordability for low-income customers. The program provides a reduced natural gas bill based on the customer’s poverty level and energy usage. APPRISE conducted a Process and Impact Evaluation of this program. The research included an assessment of the program design; review of the program costs and implementation; participant and non-participant feedback; and analysis of the impacts of the program on bill payment, arrearages, and collections costs.
APPRISE conducted research to assist Washington’s Department of Community, Trade, and Economic Development to assess low-income household needs for energy conservation and energy assistance services. The research included an analysis of the fuels used, energy costs, and energy burden for low-income households in WA; documentation of current energy prices and assistance programs offered by utility companies in WA; and recommendations for the types of energy assistance programs that may best meet the needs that are documented in the study.

APPRISE Survey Research – Usage Reduction Programs

APPRISE survey research on usage reduction program evaluations includes survey design, survey implementation, and survey data analysis.

AmerenUE partnered with the Missouri Department of Natural Resources Energy Center and other utilities to provide weatherization services to low-income households through the Low Income Weatherization Program (LIWP). The goal of the program is to provide energy efficient improvements to low-income customers to reduce their utility bills and help them prepare for the heating and cooling seasons. APPRISE conducted a process and impact evaluation of the weatherization program. The goals of this research were to assess the effectiveness of the program and to identify opportunities for program improvement.
APPRISE conducted an evaluation of California’s Single-Family Affordable Solar Homes and Multi-Family Affordable Solar Homes programs. The programs provide solar incentives to customers of three investor-owned utilities in California. The programs aim to stimulate the adoption of solar power in affordable housing, improve the quality of affordable housing, increase energy affordability for low-income households, and increase awareness and knowledge of the benefits of solar power among affordable housing occupants and developers. As a subcontractor in this research, APPRISE conducted interviews with the program administrators and conducted surveys with participating homeowners, participating building managers, and nonparticipating building managers.
Colorado’s Governor’s Office of Energy Management and Conservation implemented new energy efficiency initiatives in 2006 to improve the energy efficiency of low-income homes. The initiatives include a First Response Program to provide low-cost, cost-effective energy efficiency measures and education and an E$P Plus Initiative to provide heating system and other appliance replacement. APPRISE provided evaluation research on these new programs and additional research on Colorado’s ongoing E$P weatherization services.
The 2004 Colorado REACh grant developed a comprehensive cooling program to assist Colorado’s low-income households. Local agency partners implemented cooling program services, including equipment installation and client education, and the state LIHEAP office furnished community-wide education and notification services. APPRISE conducted a comprehensive Process and Impact Evaluation of this program.
The New Hampshire REACh Furnace Cleaning Program aimed to decrease energy usage and reduce home energy insecurity for low-income households in northern New Hampshire by cleaning furnaces and identifying furnaces in need of repair and replacement. Extensive furnace repairs and furnace replacements were also provided in those homes where the need was identified. APPRISE’s evaluation included interviews with program managers, community agency staff, and furnace cleaners; and analysis of the impact of the program on energy usage, energy insecurity, and furnace safety and efficiency indicators.
The goal of the New Hampshire Weatherization Assistance Program Evaluation is to analyze the impact of weatherization activity on energy consumption and individual household energy security as well as the efficacy of the program management and delivery of services. In this project, we provided: an assessment of the implementation of the program; an analysis of the program tracking database; identification of program deficiencies and how they might affect results; a description of how evaluation results may be used for program design and management; development of recommendations; identification of missed opportunities to install measures and why they were missed; and an assessment of the impact of the program on participant energy burden and their perception of energy burden.
The Comfort Partners Program is a statewide collaborative utility residential low-income program, providing joint delivery of services through electric and gas utilities. Participating customers receive direct installation of energy efficiency measures, energy education, and payment plans aimed to reduce arrearages. APPRISE conducted a comprehensive evaluation of this program, including a Process Evaluation and analysis of the program’s data tracking system, the comprehensiveness of service delivery, and the impacts on energy usage and bill payment.
The NJ Comfort Partners Program provides no-cost energy efficiency services to low-income utility customers throughout the state. APPRISE conducted a comprehensive evaluation of this program. The Data Tracking Research assessed the comprehensiveness, sufficiency, and accuracy of the program database. The Process Evaluation included review and assessment of materials and procedures; interviews with utility managers; interviews and on-site observations at service delivery contractors; on-site observation of audits, installations, and third party inspections; and post-completion inspections of program installations. The Affordability Evaluation included a telephone survey with program participants and analysis of the program’s impact on affordability of energy bills and participation in the NJ Universal Service Fund program. The Impact Evaluation included analysis of program data, engineering impact analysis, usage impact analysis, and savings realization analysis. The Energy Savings Protocols Research included review and assessment of protocols currently in use to estimate program savings.
Niagara Mohawk’s Low-Income Customer Assistance Program (LICAP) was designed to address the problems of low-income customers who are unable to pay their full energy bills. Payment-troubled customers may receive an affordable payment arrangement, energy use management education, and a variety of energy efficiency services including weatherization and energy efficient appliance replacement. APPRISE conducted Process and Impact Evaluations of this program.
NYSERDA’s EmPower Program provides cost-effective electric reduction measures to low-income households. These measures include lighting and refrigerator replacements, insulation, air sealing, heating system repair and replacement, and health and safety measures. The program provides in-home energy use education to customers to assist them with strategies for managing their energy costs. Participants are also invited to energy-use management and financial management workshops held in communities across the state. APPRISE conducted surveys with EmPower Program participants to assess pre- and post-program energy use and potential opportunities to refine the program. APPRISE conducted surveys with nonparticipants to assess barriers to program participation.
The Ohio Department of Development, Office of Energy Efficiency, has implemented programs funded by the Universal Service Fund. Participating customers receive cost-effective energy efficiency services and education targeted at reducing electric usage. APPRISE conducted a Process Evaluation of this program. The work included review of all program documents; interviews with program managers, staff, and contractors; observation of service delivery; customer surveys; contractor surveys; and economic analysis of the impacts of the program on Ohio’s economy.
Ohio’s Department of Development, Office of Community Services, was awarded a REACh grant to research how to provide additional resources to low-income weatherization services in a way that addresses the more comprehensive needs of low-income households. These funds were used to assess the in-home environmental health risks for vulnerable individuals, prioritize usage reduction measures to meet the energy and health needs of vulnerable individuals, and provide additional measures that enhance the health and safety of the home. The program also established a partnership with vulnerable households to take actions to mitigate the environmental health risks in the home and refer clients to other services that can improve the health of the home and the clients who live there. APPRISE conducted a process and impact evaluation of this project.
PECO’s Low-Income Usage Reduction Program (LIURP) provides energy efficiency services and energy education to PECO’s low-income customers to reduce their energy usage and increase the affordability of their energy bills. The evaluations estimated the impact of LIURP on participants’ energy usage, energy bills, and payment compliance and surveyed customers to analyze the impact of energy education and client behavioral changes. In the 2012 program evaluation, APPRISE also conducted observations of service delivery and inspections on a sample of completed jobs. This research provided information on potential changes to program services and implementation that may lead to increased program savings. The evaluation of 2015 participants assessed the program process, examined the impacts of program treatments on energy usage and affordability, and surveyed customers to assess program satisfaction.
PPL Electric Utilities (PPL) implemented the Winter Relief Assistance Program (WRAP) in 1984 to help reduce electric bills and improve home comfort for low-income customers. The objectives of the WRAP are to reduce energy usage and bills of low-income customers and to increase low-income customers’ ability to pay their electric bills, resulting in reduced arrearages. The program also aims to increase health, safety, and comfort for low-income occupants; create and maintain partnerships with community based organizations and contractors; and make referrals to other low-income assistance programs. APPRISE conducted a comprehensive Process and Impact evaluation of PPL’s WRAP. The research consisted of administrative interviews, review of program procedures, on-site observation of service delivery, customer interviews, service delivery provider interviews, and usage and payment impact analyses.
Sacramento Municipal Utility District’s (SMUD) Low Income Weatherization Program provides no-cost energy services to qualified low-income residential customers. The goals of the program are to reduce electric bills, improve safety and comfort, provide energy education to reduce usage and lower bills, and partner with contractors and community organizations in providing these services. APPRISE’s comprehensive evaluation included needs assessment, interviews with program administrators and contractors, a participant survey, on-site inspections of completed jobs, and billing analysis to measure the impacts on energy usage.
UGI’s Low-Income Usage Reduction Program (LIURP) provides no-cost energy efficiency services to low-income customers in its service territory. APPRISE conducted a Process and Impact Evaluation of this program. The research included review and assessment of program materials and procedures, on-site interviews with UGI managers and staff, in-depth telephone interviews with a sample of program participants, and analysis of the program’s impact on the energy usage of participating customers.