National Academies Press: OpenBook

Review of the Marine Recreational Information Program (2017)

Chapter: 5 Framework for Continued Scientific Evaluation, Review, and Certification

« Previous: 4 Sampling and Statistical Estimation for the Angler Intercept Survey
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×

5

Framework for Continued Scientific Evaluation, Review, and Certification

INTRODUCTION

In its 2006 Review of Recreational Fisheries Survey Methods the National Research Council (NRC) identified the need to develop an expanded and better structured framework for continued scientific evaluation, review, and certification of the methods, protocols, and procedures used for recreational fisheries data collection. Implicit in this advice was the notion that the Marine Recreational Information Program (MRIP) would need to be flexible and adaptive if it were to become the umbrella framework under which recreational fisheries data for assessment and management at the national level are provided. In other words, an improved program for collecting recreational fisheries data would need to account for the wide-ranging regional differences in U.S. recreational fisheries—and, in many cases, the differences among fisheries within the same region. It would need to develop the capacity to continually assess the strength of the scientific process, including the engagement of external scientific and technical expertise, to develop, test, review, and certify new sampling and estimation procedures. More specifically, the 2006 NRC report recommended that a permanent and independent research group should be established and funded to continuously evaluate the statistical design and adequacy of recreational fishery surveys and to guide necessary modifications or new initiatives. The report went further to say that “human dimensions” expertise should be included as well. This chapter discusses the extent to which these recommendations have been addressed and evaluates whether the framework for continued scientific evaluation, review, and certification established by the MRIP is sufficient and adequate for effective implementation of U.S. marine recreational fisheries surveys.

Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×

DEVELOPMENT OF TECHNICAL EXPERTISE

The 2006 committee specifically recommended that a survey office devoted to the management and implementation of marine recreational surveys be developed. This recommendation was based on the recognition that coordination and implementation of marine recreational surveys nationwide is a large and complex task that requires focused and dedicated attention and that expanding staff capacity and developing technical expertise were key to implementing the recommended programmatic improvements. Below the committee discusses the degree to which this recommendation was addressed.

Staffing Improvements

The MRIP has made significant progress in expanding the number of staff dedicated to the program. Since 2006, the number of full-time MRIP staff has increased from 6 to 12, and the program is supported by six full-time contractors. The program has also invested in formal training of existing staff. For example, most of the current MRIP staff have taken graduate-level courses on high-level technical topics such as survey methodology, sampling theory, and survey operations through the Joint Program in Survey Methodology (JPSM).1 Furthermore, two MRIP staff have completed a master’s degree in survey methodology through JPSM. Our discussions with regional and state partners revealed that these staff improvements have greatly increased the MRIP’s ability to expand technical support and achieve better regional coordination. In this regard, the MRIP has achieved a desired level of competency and, in conjunction with consulting of external experts, has made excellent progress since 2006.

The MRIP Communication and Education Team (CET) could still benefit from additional staffing. This team is currently supported by only one full-time and two part-time contractors and is tasked with the development, implementation, and coordination of the MRIP’s communications strategy nationwide. The success of the MRIP depends to a large degree on clear, accurate, and timely communications and on engagement of all stakeholder groups, including anglers. Therefore, the MRIP should consider expanding its communications staff to better address this important task. Chapter 7 provides a more complete review and discussion of the MRIP’s communications and outreach program.

___________________

1 Founded in 1993, JPSM is the nation’s oldest and largest program offering graduate training in the principles and practices of survey research. JPSM is sponsored by the Federal Interagency Consortium on Statistical Policy and located at the University of Maryland. To date, it has more than 240 graduates working in government agencies, academic settings, and private survey research firms.

Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×

Workshops, Conferences, and Symposia

The MRIP has either organized or been closely involved in the organization of several recreational fisheries workshops or symposia conducted either in coordination with regional partners (e.g., the regional Fisheries Information Networks [FINs] and the Atlantic Coast Cooperative Statistics Program [ACCSP]) or conducted as part of national or international scientific conferences (Table 5.1). These workshops and symposia have been attended by highly trained specialists and experts in the field and, therefore, facilitated review and discussion of MRIP issues by a broad range of scientists, promoted cross-pollination and exchange of ideas, and exposed MRIP technical staff, as well as regional and state partners, to a variety of recreational fisheries issues under different scenarios.

Consultants

The 2006 NRC report recommended that a permanent and independent research group should be established and funded to continuously evaluate the statistical design and adequacy of recreational fishery surveys and to guide necessary modifications or new initiatives. In response, the MRIP has developed and maintained a high-end cadre of statistical consultants who have greatly advanced survey revisions and improvements, facilitated faster and broader implementation of the MRIP certification process, and increased the efficiency of providing technical advice and guidance to regional and state partners.

An issue still to be addressed is the need for additional consultant support. The committee received feedback during its regional meetings that indicated that, despite the high quality of the current consultants’ input and advice, the review and certification process remains relatively slow, mainly because many regional and state partners have requested advice on survey improvements or submitted documentation for certification of new surveys. At the same time, this same pool of consultants assists the MRIP in other capacities, for example, by creating and testing new survey methods and improving existing survey designs. Expanding the pool of experts serving as MRIP consultants would allow for a faster, more efficient review process, as well as ensure a continuous infusion of energy and ideas.

DEVELOPING, TESTING, AND IMPLEMENTING NEW TECHNIQUES

In 2008 the MRIP established a Pilot Studies Program for developing, testing, reviewing, and eventually certifying new sampling and estimation procedures to be applied under the MRIP umbrella. Most of these pilot studies have been developed and implemented in collaboration with state and regional partners, usually by efforts coordinated through the regional Interstate Marine Fisheries Commissions and their associated FINs (see Box 5.1).

The MRIP Pilot Studies Program is implemented in three concurrent phases: (1) evaluation of current methods, (2) innovation to identify and test new meth-

Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×

TABLE 5.1 List of Workshops and Symposia Organized by the MRIP on Recreational Fisheries Issues

Name of Event Date
MRIP workshop: Recreational Fisheries Statistics Requirements Management Framework Workshop September 2006
MRIP workshop: Marine Recreational Information Initiative Operations Team Workshop August 2007
International Council for the Exploration of the Seas (ICES) Annual Science Conference Session on Survey Methods for Recreational and Artisanal Fisheries September 2008
MRIP workshop: Review of For-Hire Recreational Fisheries Surveys March 2008
ICES Workshop on Survey Methods for Recreational Fisheries June 2009
New Zealand Workshop on Recreational Fisheries Survey Methods August 2009
Workshop of ICES Planning Group for Recreational Fisheries Surveys (PGRFS) June 2010
Norway Workshop on Recreational Fisheries Survey Methods June 2010
MRIP workshop: Review of Oregon Recreational Fisheries Surveys July 2010
MRIP workshop: Review of Washington Recreational Fisheries Surveys November 2010
MRIP workshop: Addressing the Fishery Management Need for More Timely Recreational Data March 2011
Workshop of ICES Planning Group for Recreational Fisheries Surveys (PGRFS) April 2011
MRIP workshop: Review of California Recreational Fisheries Surveys June 2011
6th World Recreational Fisheries Conference Session on “New Methodological Tools to Survey and Assess Recreational Fisheries” August 2011
American Fisheries Society (AFS) Symposium: Improving Survey Methods for Monitoring Recreational Fishing Effort and Catch September 2011
MRIP workshop: Volunteer Angler Survey Workshop-Inventory Existing Programs and Assess Utility of volunteer angler surveys February 2012
MRIP workshop: MRFSS/MRIP Calibration Workshop March 2012
Workshop of ICES Working Group for Recreational Fisheries Surveys (WGRFS) April 2012
MRIP workshop: A Review of the Current Sampling and Estimation Methods of the Hawaii Marine Recreational Fishing Survey (HMRFS) July 2012
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Name of Event Date
MRIP workshop: Evaluation and Improvement of the Current Sampling and Estimation Methods for the Puerto Rico Recreational Fishing Survey September 2012
MRIP workshop: Development of a Survey Design(s) for Collecting Recreational Fishing Data in the US Virgin Islands September 2012
MRIP workshop: MRIP Methods and Data Seminar with NY Stakeholders April 2013
Workshop of ICES Working Group for Recreational Fisheries Surveys (WGRFS) April 2013
MRIP workshop: Design Effort Surveys for Shoreline Fishing in Highly Migratory Recreational Fisheries Surveys July 2013
MRIP workshop: Marine Recreational Information Program Executive Steering Committee Implementation Workshop July 2013
ICES Annual Science Conference Session on Marine Recreational Fisheries: Understanding Impacts and Consequences for Management September 2013
MRIP workshop: Gulf of Mexico Red Snapper Recreational Catch Accounting Methods Workshop November 2013
MRIP workshop: Gulf of Mexico Red Snapper Recreational Catch Accounting Methods Workshop II March 2014
Workshop of ICES Working Group for Recreational Fisheries Surveys (WGRFS) June 2014
MRIP workshop: Proportional Standard Error and Management Uncertainty in Recreational Data Collection on the Atlantic Coast September 2014
MRIP workshop: MRIP Calibration Workshop II September 2014
MRIP workshop: Gulf of Mexico Red Snapper Recreational Catch Accounting Methods Workshop III December 2014
MRIP workshop: Peer Review of Louisiana Creel Survey Program June 2015
AFS Symposium: Survey Methods for Monitoring Recreational Fisheries in Support of Stock Assessments and Fisheries Management August 2015
MRIP workshop: Peer Review of Alabama Snapper Check Survey Program December 2015
MRIP workshop: For-Hire Programs: Inventory, Certification, and Integration Planning May 2016
MRIP workshop: Peer Review of Mississippi Tails ‘n Scales Survey Program June 2016

SOURCE: Committee.

Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×

ods, and (3) implementation of proven methodologies. Funding—about $14.7 million from 2008 to 2015—has been largely provided by the National Marine Fisheries Service (NMFS) through a grants program coordinated through the regional FINs and ACCSP. The MRIP Operations Team solicits and reviews research proposals and provides recommendations for funding based on resources available (Table 5.2).

Table 5.3 provides examples of recently funded MRIP pilot projects and the various data or methodological needs they have addressed. A total of 105 regionally based pilot projects have been conducted since the program was implemented in 2008 in response to the NRC 2006 review.

Our review of the MRIP pilot study program indicates that it constitutes an appropriate and effective mechanism for providing highly specialized technical and scientific support (including access to technical consultants) for the development, review, and certification of regional- or state-specific surveys. This increased capability has greatly enhanced the MRIP’s capability to address regional and state needs for stock assessment and fisheries management.

Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×

TABLE 5.2 Summary of the Processes Led by or Functions Provided by Different Organizations, Institutions, and Entities Involved in the Development and Implementation of MRIP Pilot Projects

Institution or Organization Function or Process
MRIP Team (Executive Steering Committee, NMFS Office of Science and Technology)
  • Develop and conduct pilot studies
  • Manage peer reviews
  • Identify and recommend best practices
  • Facilitate regional implementation
  • Manage implementation (certain regions)
Regional Partners (States, Interstate Fisheries Commissions, RecFINs, Councils, NMFS Regional Offices and Science Centers)
  • Identify region-specific data needs and priorities
  • Adapt certified methods to meet regional needs, and, as necessary, secure additional resources
  • Manage implementation
Stakeholders (Anglers, Charter boat owners/operators, Nongovernmental organizations)
  • Work with MRIP team to identify research needs and, as appropriate, lead or participate in pilot studies
  • Work with regional partners to identify data needs and priorities, and, as appropriate, assist with resource acquisition

SOURCE: Committee.

TABLE 5.3 Examples of Recent Pilot Projects Funded by the MRIP

Pilot Project Title Summary Description
Alaska For-Hire Electronic Logbook Census Implementing new electronic logbook program targeting fishing guides and guide businesses in Alaska.
Hawaii On-site Private Boat Catch Survey Building on work that has taken place in the Gulf of Mexico and Atlantic Coasts, design and test an appropriate onsite survey to estimate catch rates from private fishing boats in Hawaii.
North Carolina and South Carolina For-Hire Electronic Logbooks Developing NC and SC for-hire industry logbook reporting programs, with supporting validation survey designs that can be used to meet any existing federal logbook reporting requirements.
Alabama Private Boat Electronic Red Snapper Fishing Census Refining the reporting methods and field validation protocols for reporting recreational red snapper landings by Alabama private recreational vessels.
Testing the Impacts of One-Month Waves Assessing the potential for bias resulting from measurement error in the Fishing Effort Survey and evaluating the impact of 1-month versus 2-month reference waves on the precision and timeliness of estimates on the Gulf of Mexico and Atlantic Coasts.

SOURCE: Committee.

Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×

USE OF NEW TECHNOLOGIES

Traditionally, recreational fishing surveys have been conducted by trained interviewers who collect specific data about anglers’ fishing trips and the nature of the species they landed and released on paper-and-pencil survey forms. However, there has recently been a growing interest throughout the recreational fishing community in identifying scientifically sound, statistically robust methods to report recreational fisheries data electronically (e.g., via smartphones and tablets). The use of these new technologies could potentially improve the timeliness and accuracy of recreational fisheries data, as well as reduce costs and paperwork burdens (e.g., nightly submissions of data to increase the timeliness of recreational catch and effort estimates to the public).

The MRIP has been evaluating and testing the use of these new technologies primarily through four distinct initiatives:

  1. For-hire electronic logbooks. The MRIP indicated that it has developed a comprehensive road map for implementing electronic reporting in the for-hire sector. It incorporates the work of three separate pilot studies and identifies the minimum requirements for ensuring success. These requirements include compliance measures, the need for hardware and software development standards, and the need for statistically sound designs for combining electronic logbook data with data collected in independent dockside and/or at-sea surveys for the purposes of validation.
  2. Angler electronic data reporting. With increases in smartphone use and Internet access, intense interest in the use of electronic reporting technologies by individual anglers has arisen. The MRIP seems engaged in further developing and expanding the electronic reporting to provide angler-provided catch data that are usable and statistically valid, exploring options that can be incorporated into, as well as supplement, existing surveys. Building on previous work in this area, the MRIP is funding several separate pilot studies to examine electronic reporting options to allow anglers to self-report data electronically.
  3. Sampler electronic data capture and submission. The MRIP has also been testing the use of tablets and other electronic data-collection platforms to allow dockside samplers to capture and submit data electronically. Such platforms could accelerate the provision of more timely data, reduce or eliminate the potential for data transcription errors, and facilitate implementation of real-time checks of data ranges and corrections at the data-collection stage (i.e., automated quality assurance and quality control [QA/QC]).
  4. Use of electronic monitoring for validation purposes. As noted previously, the validation of self-reported data on discards of recreationally caught fish at sea is important to the provision of accurate estimates of total removals by the recreational fishery. While observers can provide
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×

both validation and biological sampling of discarded fish, the recreational fishery is not well suited to deployment of observers. The use of cameras to validate total catch is becoming more common in commercial fisheries (Wallace et al., 2015) and may improve estimates of discards within recreational fisheries. Pilot studies have deployed electronic monitoring systems on even small charter vessels (< 30 ft) with good success. The committee believes that the MRIP could benefit from increased consideration and application of electronic monitoring for some sectors of recreational fishing.

In practice, evaluation and testing of new technologies for MRIP fisheries data collection are being accomplished through implementation of several MRIP-funded pilot studies, often structured according to Regional MRIP Implementation Plans. For example, researchers and stakeholders in Florida are testing the use of a smartphone- and Internet-based electronic reporting tool called iAngler to collect and report data on recreational effort and catch. A similar project is being implemented in Texas to test the use of an electronic reporting tool called iSnapper for collection of self-reported recreational fisheries catch data.

Despite these efforts, stakeholders indicated to the committee that large portions of the private angler and for-hire sectors perceive that implementation of electronic reporting or adoption of smartphone applications for volunteer angler self-reporting is not happening fast enough. This seems to be largely a communications issue—that is, anglers and for-hire captains are acutely aware of the potential for developments in smartphone, tablet, and other portable technologies, and see the potential to optimize data-collection methods. The MRIP and its partners see the advantages of electronic reporting but, given their technical roles, consider this to be primarily a statistical issue—that is, they are being cautious because they do not want to compromise the statistical robustness of survey estimates in the name of increased sampling efficiency and participation. Thus, the MRIP’s implementation of electronic reporting has been mostly exploratory and focused on testing different formats and platforms, and developing statistical techniques for integrating electronic-based data-collection programs into the existing survey estimation procedures (see Table 5.3 above for some examples).

Input from regional partners, anglers, and for-hire operators suggests that the delay in implementing electronic platforms for collection of recreational fisheries data is perceived differently by different groups. Fisheries scientists and managers as well as regional partners (i.e., regional coordinating staff associated with the RecFINs and ACCSP) are more understanding of the MRIP’s decision to delay full implementation of electronic reporting until scientifically valid estimation techniques are fully developed and properly tested. A large component of the private angler and for-hire sectors feels differently, probably because the integration of electronically reported data into the standard MRIP estimation procedures

Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×

is highly technical and statistically complex and far from intuitive. Thus, there again appears to be an issue for the communications team.

Other challenges exist. The flexible and distributed MRIP model—that is, surveys are developed and conducted according to regional implementation plans and with close coordination and input from state partners—makes adoption of electronic reporting and electronic platforms for data capture by MRIP samplers (i.e., instead of pencil and paper) more complicated. Issues such as the willingness of state partners to comply with or accept the use of electronic platforms for data capture and the costs involved in implementing these platforms nationally still exist.

In general, given the many complexities and challenges involved, the MRIP has done a good job of evaluating the use of new and emerging technologies for electronic reporting of recreational fisheries data. However, the perception by many that the MRIP is moving too slowly in incorporating these technologies needs to be addressed. The committee also sees potential for gains in accuracy through electronic monitoring of discards. The MRIP should develop a strategy to better articulate the complexities, costs, and timelines associated with the implementation of new and emerging technologies in recreational fisheries data collection. This communication strategy should not only focus on regional and state partners but also address the questions and concerns expressed by private anglers and for-hire operators (see Chapter 7 of this report for a broader discussion of the MRIP’s communication and outreach).

THE MRIP CERTIFICATION PROCESS

The 2006 NRC report recommended that a permanent and independent research group should be established and funded to continuously evaluate the statistical design and adequacy of recreational fishery surveys and to guide necessary modifications of new initiatives. In response to this recommendation, the MRIP has established a rigorous and systematic peer-review process to ensure that new survey and estimation methods are scientifically sound before they are “certified” and made available for use in stock assessments and fisheries management. Furthermore, the MRIP certification process incorporates detailed QA/QC requirements to reduce errors in the estimates produced by these add-on or supplementary surveys. Once certified, methods are available for use by the MRIP and its partners.

The process for MRIP certification seems fair and appropriate. In general, the MRIP only supports projects (i.e., financial and logistical support, including access to technical staff and statistical consultants) that apply methods that have been MRIP certified. The MRIP may support the use of methods that have not been certified if a plan to certify those survey methods is in place and being followed. The certification process is focused on evaluating new or replacement survey and estimation methods, and modifications or recommended improvements to existing methods. To be granted final MRIP certification, survey methods must

Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
  • Adhere to applicable MRIP standards and best practices (specific documentation for these standards are available at the MRIP website),
  • Be peer reviewed and supported by the results of the review, and
  • Be approved by the MRIP Operations Team, MRIP Executive Steering Committee, and NMFS leadership.

Our review of the criteria and procedural steps involved in the MRIP certification process (Table 5.2), as well as input from and discussions with multiple MRIP regional and state partners, indicates that, in general, the process is working well. For example, in 2010 the MRIP funded a full review of the Oregon Department of Fish and Wildlife Marine Resources Program’s Ocean Recreational Boat Survey and the Washington Department of Fish and Wildlife’s Ocean Sampling Program and Puget Sound Sampling Program, and in 2011 the MRIP funded a review of the California Recreational Survey Program. The committee received feedback that the partners greatly appreciated the input and reviews. The process was very interactive, involved multiple consultations with MRIP staff and statistical consultants, and provided an opportunity for those state agencies to make significant progress on developing and testing potential survey improvements. In early 2015, both the Oregon Department of Fish and Wildlife Marine Resources Program and the Washington Department of Fish and Wildlife submitted materials detailing their survey designs and programs for MRIP review and certification. Hawaii and Louisiana are undertaking similar processes.

The only major concerns the committee heard regarding MRIP certification related to the length of time associated with the process (i.e., perceived delays, lack of timeliness) and the uncertainty regarding the level of funding required to implement review recommendations. For example, the base level of survey funding for Pacific Coast states has been flat for many years, representing less than 50 percent of the overall costs of the marine recreational angling surveys. Flat or reduced funding has made implementation of recommended survey improvements difficult and in some cases impossible for states to accomplish.

The lack of timeliness in the MRIP certification process seems to be related to a need for additional people to conduct technical reviews or serve as statistical consultants (e.g., statisticians, survey methodologists, data-collection experts, human dimension and cognitive scientists). As noted above (see “Development of Technical Expertise”), the significant expansion of MRIP staff since 2006 has greatly benefited the program and has been highly praised by the MRIP’s regional and state partners. In addition, engaging statistical consultants with high levels of expertise shown by the group of statistical consultants the MRIP has engaged in the review and certification process is highly commendable. The problem seems to be rooted in the need to expand the number of people involved in this process so that review and certification of multiple survey programs in different regions can occur simultaneously and consistently. Because most programs are implemented in part through funding/governance by state Marine Fisheries Com-

Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×

missions (e.g., PSMFC), any alterations required by certification review would need to be coordinated with such agencies, which presents a strong argument for the simultaneous certification of state programs.

The review committee also discussed whether mechanisms are in place to confirm that surveys are implemented according to certified protocols and that assessments of data quality occur before incorporation of state or regional survey data into the MRIP database. Fortunately, because the vast majority of surveys certified by the MRIP are implemented through Regional Implementation Plans, several mechanisms are in place (e.g., regional RecFIN meetings, MRIP Wave meetings, regional stock assessment panel meetings) to ensure that approved survey protocols and collected data meet quality standards.

TRAINING OF INTERVIEWERS

Chapter 3 discussed interviewer effects and interviewer variation and their contributions to nonsampling error. One way to minimize errors due to interviewer variation is to provide appropriate and rigorous interviewer training (Dahlhamer et al., 2010). Standardized training will also facilitate the comparison of data at the state, regional, and national levels. Several MRIP documents reflect concern on the part of the program to produce well-trained interviewers. These include an outline of training structure, coordination and quality control (NOAA, 2016b), and procedures for interviewer training (Procedures Manual: Access Point Angler Intercept Survey [APAIS], January 2014). These documents reflect a genuine concern for maintaining training standards and highlight the following areas: interviewer knowledge (e.g., fish identification), concept definitions, basic sample design, survey data-collection procedures, survey structure, conduct of the creel survey, administrative issues, and materials and supplies. Although there is some practical, albeit limited, information on dealing with, for example, possible respondent hostility onsite, the procedures manual and statement of work (SOW) focus primarily on the technical aspects of conducting the APAIS. The following topics have not been directly addressed in documents and require further consideration.

Not everyone is well suited to be an interviewer: Recruitment of competent interviewers is critically important and should be addressed from a human resource perspective.

Improve quality control through statistical assessment: The documents, particularly the SOW, discuss aspects of QC in relation to interviewer training, evaluation, and monitoring. They mostly concern QC with regards to interviewer knowledge (e.g., fish identification), onsite monitoring, telephone verification of intercept surveys, and data quality checking. However, with respect to data QC, the documents focus on the use of error-checking software, which generally seeks and flags outlier entries for further scrutiny. Although these methods can improve data quality by detecting recording errors and data entry errors, it does little to

Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×

detect and address errors due to the interviewers themselves. As discussed in Chapter 3, statistical methods that can detect and model interviewer error should be considered as a supplemental method for increasing data quality through the QC efforts.

Not all training is purely of a technical nature: Although the training materials are technically sound, they convey a limited amount of practical experiential knowledge. In part, interviewing is an art and thus necessitates training materials that can provide more qualitative insights and guidelines for interviewers in the training process. It might be useful to produce an “interviewing in the MRIP context handbook” that documents interviewers’ experiences and methods for dealing with the vagaries of interviewing in the field. This would constitute an interviewer’s field guide with information on potential problems and solutions for interviewing in the APAIS context. The committee judges that development of this handbook would greatly benefit from input from the MRIP CET.

CONCLUSIONS AND RECOMMENDATIONS

Conclusion: During the pilot study phase of the Fishing Effort Survey (FES), NMFS has benefited from access to an independent group of statisticians and survey methodologists with substantive experience. The ever-changing world of survey research will require continuous evaluation of the FES, including experimentation, to ensure high-quality estimates in a timely manner.

Conclusion: If NMFS could expand the existing capacity of this consultant pool, both in number and in expertise (e.g., experts in cognitive issues, including angler recall), duplication of effort would be reduced and the provision of technical and scientific support would be facilitated. In addition, the MRIP certification process would be streamlined. Any such group should be periodically refreshed to include new researchers with a variety of interests and expertise.

Conclusion: With the development of a certification process, the MRIP made substantial progress toward implementing relevant key recommendations of the 2006 National Research Council report. The MRIP has invested in the development of a well-structured process for continued scientific evaluation, review, and certification of the recreational fisheries surveys conducted under its umbrella. This process also affords a mechanism for providing highly specialized technical and scientific support (including access to technical consultants) toward the development, review, and certification of regional- or state-specific surveys.

Conclusion: The MRIP has made progress in evaluating and testing the use of new technologies (i.e., smartphones, tablets, and other electronic data-capture platforms) as ways to implement electronic reporting, avoid or decrease data transcription errors, and increase the timeliness and reliability of recreational

Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×

fisheries data collection. Still, the impression among many private anglers and the for-hire sector is that implementation of these technologies is not occurring quickly enough.

Recommendation: The MRIP should develop a strategy to better articulate the complexities, costs, and timelines associated with implementing new and emerging technologies in recreational fisheries data collection and monitoring. This communication strategy should not only focus on regional partners but also address questions and concerns expressed by private anglers and for-hire operators. It should involve both the MRIP communications team and the NMFS Office of Communications.

Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Page 81
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Page 82
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Page 83
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Page 84
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Page 85
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Page 86
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Page 87
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Page 88
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Page 89
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Page 90
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Page 91
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Page 92
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Page 93
Suggested Citation:"5 Framework for Continued Scientific Evaluation, Review, and Certification." National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine Recreational Information Program. Washington, DC: The National Academies Press. doi: 10.17226/24640.
×
Page 94
Next: 6 Degree of Coordination »
Review of the Marine Recreational Information Program Get This Book
×
Buy Paperback | $55.00 Buy Ebook | $44.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The National Marine Fisheries Service (NMFS) of the National Oceanic and Atmospheric Administration (NOAA) is responsible for collecting information on marine recreational angling. It does so principally through the Marine Recreational Information Program (MRIP), a survey program that consists of an in-person survey at fishing access sites and a mail survey, in addition to other complementary or alternative surveys. Data collected from anglers through MRIP supply fisheries managers with essential information for assessing fish stocks. In 2006, the National Research Council provided an evaluation of MRIP's predecessor, the Marine Recreational Fisheries Statistics Survey (MRFSS). That review, Review of Recreational Fisheries Survey Methods, presented conclusions and recommendations in six categories: sampling issues; statistical estimation issues; human dimensions; program management and support; communication and outreach; and general recommendations.

After spending nearly a decade addressing the recommendations, NMFS requested another evaluation of its modified survey program (MRIP). This report, the result of that evaluation, serves as a 10-year progress report. It recognizes the progress that NMFS has made, including major improvements in the statistical soundness of its survey designs, and also highlights some remaining challenges and provides recommendations for addressing them.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!