Use of Market Research Panels in Transit (2013)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

5 ments and cost of the research; hiring a consultant team, if any; and working with the consultant team to develop a final research plan. Step 2—Designing the research. This step includes deter- mining the data collection technique, developing the sampling plan, and designing the questionnaire. The data collection technique will be influenced by the goals of the research and resources available, as established in Step 1. Data collection techniques include qualitative techniques (e.g., focus groups) and quantitative techniques (e.g., surveys). The sampling plan outlines the population of who is to be surveyed (the respon- dents), the sample design (for example, stratified random sample), the sample size (number of respondents required) and the procedure for contacting the individual respondents from the population. The questionnaire is developed and tested to ensure it is obtaining the desired results and that the cost of implementing the survey is in line with the estimates. An analysis plan is developed at this point of the process to ensure that the sampling and data collection meet the goals of the project. Step 3—Conducting the research. This step includes train- ing the field supervisors and interviewers, data collection, data coding, data entry, and quality checking of data. Step 4—Analyzing data and reporting results. This step includes developing the analysis programs, establishing data weighting plans if needed, running the statistical analysis routines, analyzing the results, and writing the final reports and presentations. There are two types of market research panels, as defined by the American Marketing Association (AMA): • True panel: A sample of respondents who are measured repeatedly over time with respect to the same variables; • Omnibus panel: A sample of respondents who are mea sured repeatedly over time but on variables that change from measurement to measurement [http:// www.marketingpower.com/_layouts/Dictionary.aspx? dLetter=P (accessed Mar. 11, 2012)]. The shared element is that a sample of respondents is sur- veyed repeatedly over time, regardless of the content of the questionnaire. Thus, the key concepts for panels are in Step 2 of the market research process, designing the research. This synthesis defines the various types of market research panels, identifies critical issues that the researcher needs to be aware of when engaging in market research and panel sur- veys, and provides examples of successful market research panel programs. Understanding common pitfalls and suc- cessful techniques will allow transit market researchers to make the best use of funds without jeopardizing the quality of the data or the validity of the results. MARKET RESEARCH PROCESS Market research is the gathering and evaluation of data regard- ing consumers’ preferences for products and services. It is as valuable for the provision of public services as it is in the development and sales of private sector products. In the transit industry, market research supports decision making for all aspects of planning and operations, including service routing and scheduling, fare policy and implementation, advertising and promotion of transit services, vehicle and customer amenities, customer information, and long-term financing and planning. The private sector has used market research extensively for more than 50 years and has devel- oped a wide variety of tools to collect and analyze customer information. These tools have assisted transit agencies in understanding customer needs, resulting in improved transit services and effective use of public funds. Information collected through market research is used to identify and define marketing opportunities and problems; generate, refine, and evaluate marketing actions; monitor per- formance; and improve understanding of marketing as a pro- cess. Marketing research specifies the information required to address these issues, designs the method for collecting infor- mation, manages and implements the data collection process, analyzes the results, and communicates the findings and their implications. Market research requires a four-step process: (1) planning; (2) designing the research; (3) conducting the research; and (4) analyzing data and reporting results (Elmore-Yalch 1998). Although there are variations in this process, it remains essen- tially the same for all market research studies. The activities associated with each of these steps are summarized here: Step 1—Planning. This step includes the preliminary tasks of defining the research problem; determining resource require- chapter one INTRODUCTION

6 Random Survey Sampling A traditional random sample is designed to provide results that can be projected to a target population with a specified level of accuracy, whether the population is all transit riders, the gen- eral population, or some other target population. Collecting data using a traditional random sample will often take longer, driving up implementation time and cost of the research. A non-probabilistic sample is not designed to tap into a representative population, and therefore the results cannot be projected to a target population. This simplifies data col- lection, leading to faster completion times. Online surveys (e.g., links from agency websites) and omnibus panels typi- cally have a non-probabilistic sample that can provide fast, inexpensive survey results. The Internet, smart phones, and social media have dra- matically changed how society communicates. This has had a significant impact on sampling and the underlying assump- tions of a random and representative sample. This synthesis will touch on the critical issues in sampling as they relate to panel surveys. Limitations of the Research Market research is a broad subject and while most issues related specifically to panel surveys will be covered, other topics will not be covered or may only be touched upon in the synthesis. It should be noted that: This synthesis is not intended as a guidebook for creat- ing a transit market research program. There are many refer- ence books and resources available on market research and transportation-specific research which provide information beyond the scope of this synthesis. There are many areas of study in market research that relate to all types of surveys, not specifically to panels; and therefore are only touched on in this synthesis. These topics include types of error in sampling and surveying, the use of interviewer versus self-administered surveys, and the emer- gence of dual or multi-frame sampling. Specifically excluded are non-research activities such as public outreach and public involvement, although they may use some of the same online and panel techniques. Further reading is advisable for those wanting an in-depth understanding of market research methods and the full breadth of transportation research methods. PROJECT PURPOSE This synthesis provides a literature review of the science and standards of market research panels, results of a survey of transit and transportation agencies, and case examples of selected transportation agencies currently using market research panels. Areas explored include: • Which transit agencies are currently using market research panels and how these are defined; • When and why agencies choose to use this methodology; • Qualitative and quantitative uses of panel research; • Statistical and analytical techniques and tools utilized; • Advantages and limitations of panel research compared with other market research methods; • Range of costs associated with panel research compared with other market research methods; • Demonstrated usefulness of data; • Sampling, recruitment, maintenance, and incentives used to build and maintain the panels; • Ethical and legal issues; • Lessons learned; and • Gaps in information. The results of the survey and the case examples point out that larger agencies, which typically have more resources, are more likely to experiment with new research techniques. The experiences gained from the larger agencies provide valuable instruction for smaller agencies and those without the resources to risk experimenting with new techniques. TECHNICAL APPROACH TO THE PROJECT This synthesis includes a literature review, an industry sur- vey of transit and transportation agencies, and case examples demonstrating the range of applications of market research panels. The literature review includes documents on panel surveys within the broader context of market research techniques. Sources included market research reference books, TCRP and FHWA reports, and documents available through market research industry trade organizations, such as the AMA, Amer- ican Association for Public Opinion Research (AAPOR) and the European Association for Opinion and Market Research (ESOMAR). A web-based survey was developed to determine the extent that technology-based communication tools were used, cur- rent market research activities by transportation agencies, and their use of and experience with panel research techniques. Agencies invited to participate in the survey were known to have conducted panel survey research, have an active market research program, or be a leader in the field in other areas. The survey provided an opportunity for respondents to suggest other agencies that may have conducted panel survey research. The sample was designed to provide insights into use of market research panels, and is not indicative of the transit industry as a whole. Respondents included 29 transit agencies, one metro- politan planning organization (MPO), and one state department of transportation. The response rate was 76%. Appendix A provides the industry survey questionnaire. Appendix B lists the agencies who responded to the industry survey.

7 • Implementation, analysis, and reporting; • Benefits, cost, and concerns; • Legal, ethical, and privacy issues; and • Lessons learned/elements for success. Case example interviews were conducted by telephone, and case example write-ups were reviewed, edited, and approved by the individual case example respondents. REPORT ORGANIZATION The synthesis is organized into five chapters. This introduc- tion is followed by the literature review in chapter two, survey results in chapter three, and case examples in chapter four. Chapter five provides conclusions and identifies potential further research topics. Three agencies were singled out for a brief profile of their panel experience, which had specific elements of interest but did not warrant a full case example. They are provided along with the results of the industry survey. Based on the infor- mation provided on the surveys, the consultant’s knowledge of the industry, and input from this topic’s advisory panel, four agencies were identified for full case examples: Regional Transportation District (RTD), Denver, Colorado; Minneapolis DOT (MnDOT); Metropolitan Transportation Authority (MTA) (New York City); and Washington State Ferries (WSF). The agencies were selected to demonstrate the broad range of ways in which panel survey research is being implemented. The case examples highlight the following elements: • Overview of the panel survey; • Panel sampling, recruitment, and maintenance;