8

Plans for Maintaining Continuity

INTRODUCTION

The Marine Recreation Information Program (MRIP) developed improved methodologies for the estimation of both fishing effort (the Fishing Effort Survey [FES]) and catch per unit effort (the Access Point Angler Intercept Survey [APAIS]) by recreational anglers. The resulting estimates of catches differed from those produced by Marine Recreational Fisheries Statistics Survey (MRFSS) and created the need to link and calibrate previous information collected under MRFSS with the new information from the MRIP, to create a continuous time series of equivalent data. In this chapter, the committee describes and evaluates the methods developed for this linking process and its implications for the assessment and management of stocks utilized by recreational anglers.

THE NEED FOR CONTINUOUS DATA SERIES

The MRIP calibration workshops, presentations to the committee, and substantial public testimony have highlighted the need for continuity in the recreational fisheries data used for assessment, management, and allocation. The three different processes have differing capabilities to accommodate changes in historical estimates. The stock assessment process can use recreational catch and effort statistics in two ways: as part of the raw data inputs on removals, and as indices of relative abundance. Changes in time series resulting from design and estimation changes can generally be accommodated inside assessment models using temporal blocks with different catchabilities for the two components of the time series (MRFSS/MRIP). The alternative approach is to calibrate the two time series to each other external to the assessment model and use the calibrated

estimates directly. Each approach has its merits, and while the internal assessment model treatment is more robust to uncertainty, the need for a common time series to use in other applications (management, allocation) argues in favor of the external calibration approach.

For assessment and management programs where there is no statistical model used for assessment and where the Annual Catch Limits (ACLs) may be based on historical trends, the calibrated approach is essential as a consistent yardstick for calculation of long-term averages and their variance. The MRIP calibration workshops clearly identified that modifications of the survey methodology required historical estimates to be calibrated to current methodology, rather than the opposite. The implications of an adjusted time series of catch estimates could be significant in the allocation arena, and some aspects of this issue are detailed in following sections. Likewise, adjusted time series of catch or effort statistics can influence the development of control rules for fishery removals. For example, calculation of season lengths or bag limits designed to maintain historical angler success or access will be sensitive to the input data. Existing control rules used for input management control may need to be reassessed in light of the adjusted time series of catch estimates by time or area.

TRANSITION FROM PHONE-BASED TO MAIL-BASED EFFORT SURVEY

The Coastal Household Telephone Survey (CHTS) was an extremely problematic element of the MRFSS due to a number of potential and realized biases in a methodology based on random-digit dialing of landlines. The 2006 National Research Council (NRC) report noted the inherent difficulty of estimating fishing effort using such methodology, in the absence of an adequate list frame of anglers to increase the efficiency and accuracy of the effort estimation. The MRIP has clearly heeded the NRC advice and developed a dual-frame methodology using both a list frame of anglers and a secondary list frame based on the U.S. Postal Service address-based frame of households (Chapter 3). The MRIP undertook substantial design and testing of the new effort estimation methodology. The results of implementing the new procedures were different estimates of fishing effort, often by large amounts, for some areas and time periods. Andrews et al. (2014, Table 3, p. 18) document differences in fishing effort of approximately four times higher for the improved FES compared with the previous CHTS methodology. Because these estimates resulted in much higher estimates of total catch for species in these areas and times, the committee has invested considerable effort in examining their validity.

Chapter 3 of this report examines the MRIP effort estimation methodology in detail and makes several recommendations to address issues of nonresponse and recall biases, weighting of the strata responses, and correct incorporation of variance in the components of the ultimate estimates. These recommenda-

tions are significant to the estimation of fishing effort, catch per angler, and the ultimate calculation of accurate values for total catch. Although addressing these recommendations may change the scale of the MRIP time series of total catch amounts for some areas and species, the choice of a method for calibration of the MRFSS and MRIP time series is not likely to be sensitive to these changes. This is because the changes contemplated by this report will affect primarily the degree of offset between the two time series, which the calibration is designed to bridge. However, it is important that MRIP staff be cognizant of any changes in methodology that affect the determination of peak fishing effort periods, because all calibration methods currently contemplated involve the use of peak effort periods to calibrate MRFSS estimates to MRIP estimates.

DEVELOPMENT OF CALIBRATION AND BRIDGING AMONG DATA SERIES

The 2006 NRC report on the MRFSS program (NRC, 2006) recommended several improvements to the program that would reduce the potential for bias in both effort and catch estimation. Largely as a result of the NRC report, the National Marine Fisheries Service initiated a complete redesign of both the effort and catch components of the MRFSS program. The two elements of the MRIP (APAIS and FES) were implemented with different degrees of rigor, largely dictated by the relatively higher expense of the intercept survey. The APAIS was evaluated in a side-by-side comparison with the previous MRFSS methodology in only a single year and for a single area. As such, our knowledge of the relationship of the estimates arising from the two methods is somewhat limited. In contrast, a carefully staged implementation of an improved mail-based FES was more temporally and spatially extensive. Nonetheless, the combined MRIP methodologies resulted in estimates of recreational catches that differ from the previous MRFSS estimates, generally by small amounts but substantially for some species-area units (Andrews et al., 2014). These differences between MRFSS and MRIP estimates ranged from consistent biases to apparently random variation.

The new methodologies employed in calculating the MRIP estimates are more statistically valid than those used in the MRFSS program (Chapters 3 and 4). Many important components of recreational fisheries management are dependent on these catch and effort estimates, including stock assessment, development of harvest policies, in-season management, and catch allocation (Figure 1.2). In addition, the allocation of resources for the production of catch statistics is itself dependent on the estimates of catch produced by the MRIP. The historical time series of recreational catch and effort produced with the outdated MRFSS procedures therefore requires calibration to the estimation processes used in the MRIP, so that a combined time series of total removals may be used to inform these processes.

The MRIP convened two workshops to address the calibration issues. The first, in 2012, was designed to develop a method to calibrate 2004-2011 catch rate

estimates based on the unweighted MRFSS estimation methods to catch estimates based on a new MRIP weighted method, demonstrate its use in hind-casting estimates prior to 2004, and develop a plan for implementing the calibration into benchmark stock assessments. The workshop identified a simple ratio estimator (MRFSS/MRIP) using 2004-2011 data, which could be used as a constant for hind-casting data prior to 2004, or trended using auxiliary information. The second workshop, in 2014, was convened to revisit the calibration issue in light of changes to the APAIS made in 2013 and 2014. That workshop identified three potential alternatives for calibration (discussed below), an interim methodology to use while the three methods were evaluated fully, and procedures to follow if survey methodology were to change in the future.

Both workshops clearly recognized that calibration was critical in allowing stock assessments to differentiate true changes in stock status from changes in the estimation procedures producing the data used in the assessments. Both workshops also identified several issues that affect the sampling error of the catch estimates, based on changes to the survey designs of both the MRFSS and MRIP over time.

The committee reviewed the workshop reports and other MRIP documents to determine the current status of calibration and plans for updating or improving the calibration method. Appendices 1 and 2 of the 2014 Calibration Workshop report (Carmichael and Van Voorhees, 2014) outline the three suggested alternatives for calibrating pre-2013 estimates to the post-2013 estimates. Importantly, the workshop also considered the opposite calibration, that is, calibrating the post-2013 estimates to the historical time series. The workshop concluded that the former process (calibrating historical to present) was the preferred calibration method because harvest control methodology requires coherence with catch estimation methodology.

The three alternative methods were examined thoroughly by the 2014 workshop. Their construction and merits are detailed in Appendix 1 of the workshop report, and are only summarized here (Carmichael and Van Voorhees, 2014).

1. Direct catch ratio estimator. In basic concept, the simple ratio estimator takes advantage of the improved coverage of peak periods in the 2013 MRIP (C_{p, 2013}) and scales the catches prior to 2013 by the ratio of peak catches to total catches in 2013 (R₂₀₁₃ = C_{total 2013}/C_{p 2013}). The scaled estimate for total catch (C_tot,y) in prior year y is then based on applying the 2013 ratio to the peak catch in the prior year, y. Thus,

   C_{tot, y} = R₂₀₁₃*C_p,y.

   The scaling is based on post facto identification of peak periods prior to 2013 and makes no use of data for nonpeak periods.

2. Complex ratio estimator. Because the MRIP program produces estimates of effort distribution throughout the day, it provides an opportunity to

3. scale the effort distribution in 2013 to match the truncated effort estimate from the more limited sampling in prior years. This is achieved by adjusting the weighting of effort in temporal bins for 2013 to match the effort for the more restricted temporal bins that existed in previous years. The ratio of the catch in the truncated 2013 bins (C_{tr 2013}) to the total catch in 2013 (C₂₀₁₃) is defined as R_{c/tr, 2013}. Similar to the simple ratio method, the R_{c/tr, 2013} is then applied to the available catch estimate (C_y) from a prior year y, to obtain an adjusted estimate of catch for that year. Thus,

   C_{y, adj} = R_{c/tr, 2013} * C_y.

   This adjusted estimate is assumed to be the estimate that would have been obtained if more complete MRIP-style sampling had been conducted in previous years. This estimator assumes a constant distribution of catch and effort over time and area, relative to that in 2013. Furthermore, estimates of total effort for years prior to 2013 are obtained from the CHTS, which have unknown properties.

4. The regression-based estimator. This estimator is more involved than the ratio estimators and is in some measure the reverse of the complex ratio estimator. It uses 2013 data to estimate and predict the distribution of morning, peak, and evening categories of catch/effort for 2013, based on characteristics of the catch or demographics from the APAIS. This modeled relationship is then applied to target year data to derive a pseudo distribution of categories for that year, which matches the 2013 distribution. These pseudo proportions are used to produce adjusted estimates of catches for the target year. Several extensions to this method are outlined in the report.

   The primary assumption of this method, and it is a strong one, rests on the stationarity of the catch and effort process over time and space. In other words, it assumes that the effort and catch distribution throughout any given day can be captured by this single model relationship. The committee appreciates the conceptual investment in this approach and commends the workshop for its innovative thinking. However, the committee has strong concerns about the ability to validate such an approach, because the quantity being predicted, that is, the distribution of categories, cannot be observed. This quantity is defined by 2013 characteristics and imputed to the target years.

The committee notes that all three methods are actually model-based estimators—all involve an underlying estimation model and vary only in the influence of the assumptions involved for each. The workshop consultants recognize that the calibration was not straightforward due to the limited side-by-side estimation using previous and current methodology for almost all areas. The committee agrees with the consultants’ concern in this regard and believes that

uncertainty about process and observation error could be reduced if additional side-by-side comparisons were conducted. While the consultants also suggested that time-series or small-area spatial analyses might also be conducted, the committee is doubtful that such analyses would yield significant improvements in a general calibration method. Nonetheless, such analyses could be conducted with available data and would be worth some investment of analytical resources.

Appendix 2 of the 2014 workshop also identified an interim approach (the simple ratio) to be applied while a full evaluation of the three alternatives was conducted. The Appendix detailed the drawbacks to this method, notably that the relationship of peak period catch to total catch is constant, and that none of the data outside of the peak catch period for years prior to 2013 are used. Both the 2012 and the 2014 calibration workshops provided guidance to stock assessment scientists concerning the use of a calibrated time series for the combined MRFSS-MRIP data. In particular, they suggested increasing the assumed variance in the time series to account for uncertainty in the calibration process.

ANTICIPATING IMPACTS ON ASSESSMENT AND MANAGEMENT PROGRAMS

An accurate calibration of MRFSS data to MRIP data has implications for both assessment and management. Statistical catch-at-age (SCA) stock assessments, while not immune to differences, are the least sensitive to calibration issues because the assessment models can accommodate some imprecision in calibration through alternative catchability functions. Imprecise or biased calibration does affect the calculation of reference points related to unfished biomass, hence optimum harvest rates and control rules. In SCA frameworks, calibration issues may increase uncertainty in these quantities, although these influences will be less strong than in other assessment/management frameworks.

In non-SCA stock assessment frameworks, and particularly in data-poor assessments, where the time series of total catch is a prime determinant of harvest levels (ACLs) and reference points, the method of calibrating MRFSS to MRIP data is likely to be more influential than in SCA frameworks. In the former, both the trend and scale of stock changes are informed totally by the calibrated time series, and in turn, the understanding of stock status is similarly governed. In these instances, the calibration process will have a much larger influence on the understanding of current stock status and appropriate reference points for stock management. The committee notes that these influences will not be uniform and will affect recreational fisheries management much more strongly in some areas than others, directly linked to the nature of how ACLs are determined.

For data-poor assessments the estimation of common reference points for stock management, for example, unfished equilibrium biomass B₀, biomass depletion level, and target harvest rate, are not well determined, or may be precluded, by time series of catches alone. The estimated B₀ is a quantity of consid-

erable uncertainty for even technically sophisticated assessments. In turn, a poor understanding of target harvest rates will increase the uncertainty associated with ACLs. Calibration affects primarily the scale of estimated removals but may also influence its trend. In the absence of auxiliary information on trend, management is therefore critically dependent on simple catch time series. These issues are not uniquely associated with the methodology for calibrating data series resulting from changed estimation methodologies, but imprecision in calibration will increase uncertainty in fisheries management.

Future efforts to develop calibrated time series of recreational catches will be most useful if accompanied with advice on the implications of the calibration method to stock assessment and reference points for stock management. In particular, simulation analyses of alternative methods will be useful. As the time since a change in methodology for estimating recreational catches lengthens, the calibration method will have less influence on the understanding of current stock status. The understanding of stock status will be influenced more strongly by recent data than by historical shifts in estimation methodology for catch, when removals are substantial proportions of available yield. If removals are a small proportion of available yield, then the calibration will continue to influence understanding of stock status. However, because the calibration methodology does influence the understanding of reference points for management, the effect of the calibration will be a long-term element of fisheries management. This is an important element to consider when contemplating any changes in survey and estimation methodology and underscores the point that any such change should be thoroughly evaluated prior to implementation.

CONCLUSIONS AND RECOMMENDATIONS

Conclusion: The low number of side-by-side comparisons between the angler intercept portions of the MRFSS and MRIP methodologies limits the ability to develop a more precise calibration between the time series of data produced by the two programs.

Conclusion: None of the methods proposed to calibrate the two intercept time series is completely satisfactory because of the necessary assumptions and/or post hoc data stratifications that must be applied when using the methods.

Conclusion: For stocks with substantial removals, the calibration between the two intercept data sets will diminish in importance for some stock assessment purposes over time as more recent data dominate the determination of stock status. Nonetheless, uncertainty in the estimation of reference points for harvest policy determinations will remain sensitive to the calibration process.

Conclusion: The calibration of the two data time series is extremely important

to multiple aspects of fishery stock assessment, catch management, and allocation processes. For stock assessment modeling, the absence of a fully satisfactory calibration can be addressed through alternative estimates of catchability over the combined time series. For simpler stock assessments, the calibration may be more influential.

Recommendation: The MRIP should continue development of a statistically sound calibration methodology as improvements to the Access Point Angler Intercept Survey and Fishing Effort Survey methodologies are incorporated. In the interim, the existing ratio-based calibration should be continued. For statistical catch-at-age (SCA)-based assessments, scientists should employ alternative catchability functions applied to the combined time series as a means to accommodate potential imprecision in the calibration of MRFSS data to MRIP data. For non-SCA frameworks, assessment scientists should exercise caution in the interpretation of trends in catch data.