APPRISE Projects

Residential Market Transformation Research and Evaluation

APPRISE conducted survey research to support process and impact evaluations of Alliant’s residential energy efficiency programs. The programs included Appliance Recycling, Direct Load Control, Home Energy Audits, Residential Prescriptive, New Home Construction, and Energy Awareness and Education. The surveys were designed to support program development and improvement.

APPRISE conducted a process evaluation to understand the contributions of Avangrid’s marketing programs to residential and non-residential customer participation in Avangrid’s energy efficiency programs. The research included in-depth interviews and surveys with program participants to assess the success of various marketing approaches.

Columbia Gas implemented a portfolio of Demand Side Management (DSM) programs in 2009 for its residential and small commercial customers. The residential programs include a Residential Home Performance retrofit incentive program and a rebate program for customers who purchase programmable thermostats and energy-efficient showerheads. APPRISE conducted a process evaluation of these programs to help refine the programs and improve program effectiveness. The evaluation research included document review, analysis of program statistics, interviews with Columbia Gas managers and staff, interviews with program implementers, and participant and nonparticipant surveys.
Consolidated Edison’s Appliance Recycling Program provides financial incentives to encourage residential electric customers living in one-to-four unit homes to recycle their freezers, second refrigerators, and room air conditioners. APPRISE supported the Impact Evaluation of this program by facilitating on-site data collection. Data collected included the location of the second refrigerator, how the second refrigerator is used, and metered energy usage for that refrigerator. Data from the site visits was used to analyze the impact of the program on energy use.
APPRISE worked with a team to assess the performance of thirteen residential and small business energy efficiency programs offered by the Consolidated Edison and Orange & Rockland utility companies. The evaluation assessed how each energy efficiency program is operating and made recommendations for improvements to the implementation process. Specific information gathered via program participant and non-participant surveys included participant satisfaction, effectiveness of program marketing and delivery, barriers to participation, satisfaction with measures offered, and satisfaction with trade ally training programs.
Consolidated Edison’s Multi-family Energy Efficiency Program provides financial incentives for the installation of energy efficiency measures in common areas and within apartments, in multi-family buildings with 5-75 units. Prescriptive rebates are offered for common area measures including HVAC systems and controls, insulation, and lighting. No-cost unit-level measures include compact fluorescent lights, low-flow showerheads, faucet aerators and smart power strips. APPRISE supported the Impact Evaluation of this program by facilitating individual unit on-site data collection. Data collected included the location of the apartment within the building, the number of light fixtures in the apartment, and metered CFL and smart strip usage. Data from the site visits was used to analyze the impact of this program on energy use.
Consolidated Edison’s Residential HVAC Program provides financial incentives to customers to encourage the installation of select high-efficiency equipment and home energy improvements. APPRISE conducted research to assess the impact of this program. This work included recruiting, scheduling, and conducting site visits to meter central air conditioners and ductless heat pumps installed through this program. Resulting usage data will be analyzed to estimate the impact of this program on energy use.
Duquesne Light’s Upstream Lighting program works with residential lighting retailers to offer discounts on CFLs and LEDs at the point of purchase. Duquesne Light is conducting an enhanced net-to-gross analysis of their upstream lighting program. This research includes in-store intercepts with customers, in-depth interviews with lighting manufacturers and retailers, and telephone interviews with Duquesne Light’s residential customers. APPRISE conducted the general population telephone survey of a random sample of Duquesne Light’s residential customers to ascertain the net-to-gross ratio, in service rate, residential versus non-residential installation allocation, and low-income versus non low-income household savings allocation.
Vermont has a number of energy programs that make important contributions to reductions in electricity usage in all sectors and reductions of fossil fuel usage among low-income households. However, Vermont is missing important opportunities for energy savings because there is a program gap with respect to fossil fuel usage by households that are not low-income and fossil fuel usage in the commercial and industrial end-use sectors. This 2006 memo presents the status of energy usage in Vermont, describes existing programs, and discusses opportunities for future programs.
EPA initiated Energy Star programs to reduce pollution by accelerating the adoption of new and underutilized energy-efficient technologies. In this evaluation we conducted surveys with home builders and home buyers. We conducted a home builder mail survey with participating Energy Star builders on their satisfaction with services received through the program. The survey also collected information to assess the impact of the program on the behavior of home builders. In the home buyers survey, we collected baseline and follow-up information from a sample of Energy Star home buyers and a comparison sample of other new home buyers. We obtained energy billing data for responding households and developed estimates of energy savings resulting from the Energy Star Homes Program.
The Energy Star HVAC Program works to overcome barriers associated with the purchase of higher efficiency replacement HVAC equipment in the residential market. EPA works with HVAC equipment manufacturers, distributors, dealers, utilities, and financial institutions to overcome the market barriers. In this evaluation, we conducted in-depth interviews with all participants in the HVAC program, including trade associations, utilities, finance partners, manufacturers, regional distributors, dealers, and consumers.
APPRISE was a subcontractor on the evaluation of the Massachusetts ENERGY STAR Lighting Program. This statewide program provides incentives for residential customers to purchase ENERGY STAR qualified lighting products. APPRISE conducted telephone surveys with comparison residential customers in South Dakota to assess the market effects of the program.
The New Jersey Board of Public Utilities manages a portfolio of energy efficiency programs. These programs include the Residential HVAC, Residential New Construction, Energy Star Products, Commercial and Industrial New Construction, Commercial and Industrial Retrofit, Combined Heat and Power, and Customer On-Site Renewable Energy. As part of a comprehensive evaluation of the portfolio of programs, APPRISE conducted an evaluation of the Residential New Construction Program, known as the NJ Energy Star Homes Program. Under this program, builders receive incentives to develop homes that are at least 15 percent more energy efficient than a standard home. APPRISE conducted research to update how estimates of the program’s energy savings are made and to estimate actual program energy and demand savings from 2001 through 2006. The research included a survey with Energy Star homebuyers and non-participating homeowners and analysis of electric and gas energy usage data for these households.

New Jersey Natural Gas operates energy efficiency programs that are complementary to the New Jersey Clean Energy Programs (NJCEP) as the SAVEGREEN project. The residential program provides grants or on-bill repayment plans to assist with the purchase and installation of furnaces/boilers and water heaters. The commercial program provides NJCEP Direct Install participants with a no-interest on-bill repayment plan for the value of the project not covered by the NJCEP incentive. APPRISE conducted an evaluation of the SAVEGREEN project to assess program management, analyze the incremental impact on energy efficiency activity, determine the extent to which the program influences residential customer implementation of whole house improvements, measure the program’s energy savings, and measure the non-energy impacts of the programs.

New Jersey Natural Gas’ (NJNG) SAVEGREEN Project offers rebates and financing incentives to fund energy efficiency projects for residential and commercial customers.  APPRISE is on a contractor team to evaluate NJNG’s operations and ensure that the programs are meeting the energy savings reduction targets outlined in New Jersey’s 2018 Clean Energy Act.  APPRISE is responsible for the Home Performance with Energy Star and Moderate Income Weatherization Process and Impact Evaluations.  The research will include NJNG and trade ally interviews, participant surveys, and analysis of the natural gas and electric usage reduction that is attributed to these programs.

Under New Jersey restructuring legislation, all utility DSM programs are implemented through a statewide utility collaborative. The New Jersey RNC Working Group was responsible for designing a statewide RNC program to encourage builders to construct and home buyers to purchase energy-efficient homes. APPRISE staff directed the RNC Baseline Study, including surveys with market actors to assess awareness of and attitudes toward RNC programs, and the development of Residential New Construction statistics and information on the Affordable Housing market.
NYSERDA is supporting training for building code, design, and construction officials across New York State to help reach State Energy Code compliance goals. The training provides detailed information on current and planned New York State Energy Code requirements, building science principles, and technical guidance on appropriate application of these principles. APPRISE designed the survey, coordinated with multiple training providers to administer the pre- and post-training survey to all training participants, and processed the survey data. The surveys collected information on knowledge and awareness of specific code requirements and feedback on the provided training. This research helped NYSERDA evaluate the impact of training courses and to identify future training opportunities.
The goal of NYSERDA’s ENERGY STAR Homes Program is to implement and promote an enhanced version of the U.S. Environmental Protection Agency’s ENERGY STAR Program in New York State. The program provides technical assistance and targeted financial incentives to residential builders and income-eligible homeowners to encourage the adoption of energy efficient design features and the selection and installation of energy efficient equipment. APPRISE conducted survey research to support NYSERDA’s market characterization research for the ENERGY STAR Homes Program (ESH) in 2008. APPRISE gathered information from homeowners who purchased an Energy Star Home and homeowners who purchased a non-Energy Star Home to examine their views, experiences, and perceptions on a variety of topics related to program participation. APPRISE also surveyed home builders who participated and homebuilders who did not participate in the ESH to assess their views, experiences, and perceptions on a variety of topics related to program participation. APPRISE conducted additional surveys with participating homeowners, non-participating homeowners, participating builders, non-participating builders, and formerly-participating builders to supplement energy savings estimates and provide information on potential program free-ridership and spillover effects.
NYSERDA’s Energy Star Products Program (NYESP) seeks to increase sales of residential energy-efficient appliances, lighting, power management, home electronics, heating, ventilation and air conditioning equipment. The goals of the program are to increase the supply of products through partnerships with retailers, manufacturers and distributors and to create demand for high-efficiency and ENERGY STAR products through increased consumer awareness and understanding of the ENERGY STAR label. APPRISE surveyed recent purchasers and appliance retailers of refrigerators, dishwashers, clothes washers, or room air conditioners and assessed the purchases, sales, availability, attitudes, and awareness of Energy Star products.
NYSERDA’s Green Jobs – Green New York (GJGNY) Program provides New Yorkers with access to energy audits, installation services, low-cost financing, and pathways to training for various green-collar careers. Evaluation activities were designed to assess the effectiveness, progress, and outcomes related to the program and to provide recommendations for program refinement. APPRISE conducted surveys with program participants and non-participants to develop a comprehensive understanding of current and emerging markets and to provide baseline data to enable NYSERDA to define, deliver, and evaluate programs for these target markets.

NYSERDA’s Clean Energy Fund includes multiple initiatives designed to support adoption of heat pump technologies in residential buildings.  NYSERDA is conducting a comprehensive study to understand the impact of current efforts, to characterize the heat pump market in New York State, and to measure energy and cost savings due to electrification.  APPRISE will be conducting detailed surveys with participating building owners and contractors for each Heat Pump initiative.

NYSERDA’s Home Performance with Energy Star (HPwES) Program provides low-cost training to contractors and financial incentives to homeowners to help improve the energy efficiency of the housing stock in New York. APPRISE conducted survey research to support NYSERDA’s market characterization research for the program in 2008. APPRISE’s research gathered information from homeowners who received a Comprehensive Home Assessment from a participating BPI-certified technician to examine homeowner views, experiences, and perceptions on a variety of topics related to program participation. APPRISE also surveyed program participating and non-participating contractors to examine contractor views, experiences, and perceptions on a variety of topics related to program participation. APPRISE also completed the HPwES Impact Evaluation, and conducted surveys of homeowners and contractors to supplement program net-to-gross savings estimates.
APPRISE conducted RDD telephone surveys in NY State, New York City, Ohio, Washington DC, and Houston in 2009 to assess the saturation of CFLs. APPRISE then recruited the RDD survey respondents for in-home audits of lighting fixtures, existing bulbs, and home electronics bundles in the summer of 2009. In the summer of 2010, APPRISE conducted another RDD telephone survey in NY State, New York City, and Houston, and recruited respondents for a follow-up in-home audit. The data are being used by CFL Program staff to identify marketing opportunities and by NYSERDA Evaluation staff to measure program impacts.
NYSERDA conducted a market characterization study of the lighting market in New York to inform a possible Enhanced Lighting Design Program under SBC IV. APPRISE conducted quantitative surveys with retailers, homeowners, and the general population to develop a better understanding of the residential lighting market and identify opportunities for optimizing NYSERDA’s future investments in the lighting market.

NYSERDA’s Low Carbon Pathways program provides incentives for multifamily buildings to install measure packages that include electric heat pump technology, balanced ventilation measures, building envelope improvements, and energy efficient domestic hot water systems.  As part of the Impact Evaluation, APPRISE is designing and conducting a quantitative survey with building residents to document home conditions before and after the building upgrades.  This survey will provide NYSERDA and participating buildings with valuable information about resident perceptions and satisfaction, non-energy impacts resulting from building upgrades, and variations in resident experience based on location or unit type. Measured non-energy impacts will include resident perceptions of unit comfort, indoor air quality, hot water quality, unit affordability, and household health and well-being.  APPRISE is also analyzing building-level results and highlighting key findings for building managers.

NYSERDA’s Residential Energy Audit Program provides no-cost home energy assessments to New York homeowners. The home assessments include recommendations for building envelope measures, high efficiency heating and cooling equipment, and energy controls technology. APPRISE supported a study to estimate the adoption rate for each recommended measure, document the costs of measure installations, and calculate energy savings and non-energy benefits for participants. APPRISE was responsible for survey design consultation, survey testing, and outreach and data collection for the Participant Survey that documented the measures that participants have or have not installed since the home audit was completed.

PGW’s Home Rebate Program provides incentives to residential customers and participating Conservation Service Providers (CSPs) who complete comprehensive natural gas energy efficiency retrofits.  Financial incentives are designed to encourage customers and CSPs to pursue as many cost-effective energy efficiency measures as possible and achieve at least twenty percent energy savings on average for the program.  The market study and impact evaluation included interviews with program administrators and CSPs, in-depth interviews with partial and full program participants, analysis of the measures installed, assessment of the impact of the program on natural gas usage, and estimation of the program’s realization rate and cost-effectiveness.
PGW’s RHER provides rebates to customers who purchase residential-sized natural gas heating equipment to encourage purchase of the most efficient boilers and furnaces available. The program provides prescriptive cash rebates to offset most of the incremental cost of the higher efficiency equipment and installation. APPRISE previously conducted an impact evaluation and a market study for the program soon after the program was implemented. The impact evaluation assessed the impacts of the program on natural gas usage by measure type, efficiency, and pre-treatment usage, and compared estimated savings to deemed savings. The market study included interviews with the program manager, HVAC contractors, and program participants to assess the impact of the program on the market for high efficiency equipment, obstacles faced in program participation, and the level of financial incentives needed to encourage adoption of this equipment. APPRISE also conducted a second impact evaluation to assess the energy savings achieved by the program in the 2014 program year.
Puget Sound Energy (PSE) began mailing Opower Home Energy Reports (HERs) to approximately 40,000 customers in November 2008. These reports deliver customized, household-level data on energy usage in the home, and encourage energy-efficient behaviors and measures. Savings for participants in the HER Program may be a direct result of the HER report or may result from other energy efficiency programs, such as CFL buy downs. To identify potential double-counting of energy savings, APPRISE conducted survey research as part of a three-year Home Energy Report (HER) Impact Evaluation. Fifteen hundred interviews were completed with current HER recipients, past recipients, and a comparison group of customers who never received a report. The survey collected data on purchases made and energy usage behaviors adopted by respondents across all three groups to enable estimation of the impact of other energy programs and avoid double counting of these savings.

APPRISE and MaGrann conducted a study of residential thermostat settings and temperature preferences based upon a survey of 239 households in the Northeast about thermostat settings, setbacks, and type of thermostat in the home.  The study documented thermostat setpoints and setbacks; the penetration of programmable thermostats and smart thermostats; and use of smart thermostat features.

Based on the research, the study recommended that the greatest setback opportunities for non-elderly households were nighttime in the winter and daytime in the summer.  Different protocols are needed for elderly households.

  • For households without an elderly member, consider a default winter setting of 69 during the day with a setback of three degrees to 66 at night; and 72 degrees at night in the summer with a setup of 2 degrees to 74 during the day.
  • For households with an elderly member, consider a default winter setting of 70 degrees in the winter and 74 degrees in the summer, but without any predetermined setbacks.

We recommend always providing the opportunity for a resident interview in combination with the installation of a new thermostat so that default settings can be modified accordingly.  Similarly, providing the opportunity for “live” training empowers residents to modify default settings with an understanding of the energy and comfort implications.

Smart thermostats represent a good opportunity for energy savings based on a number of factors evident from this survey.

  • Low existing penetration rate.
  • Many households still do not practice thermostat setbacks.
  • Of the households practicing setbacks, most are doing so manually.
APPRISE was a subcontractor on the evaluation of the Rhode Island ENERGY STAR Lighting Program. This statewide program sponsored by National Grid provides discounts for residential customers to purchase ENERGY STAR-qualified lighting products. APPRISE conducted telephone surveys with residential customers in Rhode Island to assess the market effects of the program.
APPRISE partnered with two natural gas utilities in the Northeast to study current thermostat technology, setback behavior, and opportunities for smart thermostats.  A survey was conducted with a random sample of 385 residential customers to develop a better understanding of which households may benefit from smart thermostats and assess the opportunities for energy savings.  The survey found that there is a great opportunity for smart thermostats, as only four percent of respondents had a Wi-Fi connected thermostat, but 74 percent had the technology needed for a smart thermostat.  Lower income households may present the greatest opportunity for savings with smart thermostat technology.  Over half of these customers had the needed technology, but they were less likely to have programmable thermostats.  Households show opportunities for benefitting from smart thermostats that adjust to their usage preferences, as about half of the respondents reported frequent manual adjustment of programmable thermostats.  Many reported that they would make extensive use of smart thermostat features if they were available.  Programs and incentives could potentially have a large impact on the adoption of smart thermostats.
South Jersey Gas has been operating energy efficiency programs in coordination with the New Jersey Clean Energy Program (NJCEP) since 2009. The programs provide additional rebates and financing to customers who participate in the NJCEP residential and commercial and industrial programs. APPRISE conducted an evaluation of these programs to assess program design, implementation, and impacts. The research included interviews with program administrators, contractors, and participants; surveys with participants of three different programs; analysis of the impacts of the programs on natural gas usage; and assessment of the non-energy impacts of the programs.
Sustainable Princeton introduced the EnergySmart Homes Campaign to help Princeton residents reduce their energy usage. Through this program, Sustainable Princeton partnered with a BPI-certified contractor to provide low-cost home energy audits and encourage the installation of energy saving measures. APPRISE conducted a survey with EnergySmart Homes participants to explore the quality of energy education and services delivered by the auditor, why customers did or did not install measures, and what actions could be taken to help customers overcome barriers to measure installation.