4

Findings and Recommendations

METHODOLOGICAL IMPROVEMENTS

Calculator Approach and Intended Use

Finding 1: The software has conceptual design flaws (e.g., lack of consideration of oil mass balance and lack of guardrails for some input values), is incomplete (e.g., lacks storage), and has software errors (e.g., interface does not always update calculation with changing entries; see Appendix B). The software and the documentation are inconsistent and, in some computer operating systems, incompatible.

Given the gap between intended outcome and actual outcome inconsistency in the provided documentation versus the actual software, combined with inability to get clarifications from USCG Inland ERSP Calculator developers, it was very difficult for the committee to do a comprehensive job of evaluating (i.e., reviewing and verifying) the calculator. An additional level of complexity arose depending on the user’s computer. Because the USCG Inland ERSP Calculator was programmed to produce an installable Windows Form Application, Apple users encountered challenges when trying to download and run the system.

Recommendation 1.1: The calculator should undergo a thorough verification/quality assurance testing regimen by independent third-party reviewers prior to consideration for release.

Ideally this review would be populated by individuals who have day-to-day experience in the spill response field and in planning for the use of skimmers in spill response operations. For example, an option for performing the independent, third-party review would be to reach out to the American Society for Testing and Materials (ASTM) F20 Committee on Hazardous Substances and Oil Spill Response. Other groups that could be part of the process include inland planners with experience with geographic response plans, oil spill removal organizations (OSROs), and equipment manufacturers. This process will also provide feedback on the value of the tool to the user community.

Developing the calculator as a browser tool would ensure that it works on all platforms that support a web browser, addressing the compatibility issue.

Recommendation 1.2: The documentation should be updated to ensure that it is consistent with the current version of the calculator.

Recommendation 1.3: The USCG Inland ERSP Calculator documentation should specify its intended use, assumptions and simplifications, and outputs, and explicitly state how each input is used. As discussed earlier, some examples include:

• The assumption that skimmer nameplate capacity can be simplified to three values despite an extensive input section on skimmer selection.
• The way that spill volume is used in the calculation should be clear in the user guide. A user may think that the output is based on the actual spill volume entered rather than a thickness determined from the Oil Behavior Module based on one of three predefined spill volumes.

It should also be clear to the user when inputs are not taken directly by the calculation, but only serve to guide the user. Examples of this include Planning Volume Classification, Latitude Zone and Season, and Source Type.

Recommendation 1.4: Any revisions to the documents that accompany the calculator should provide alternative descriptions and/or definitions of the terms “nearshore” and “inland” to ensure that they are not misunderstood for the regulatorily defined terms.

Finding 2: The USCG Inland ERSP Calculator, as presented, may not improve planning and response in inland areas to the extent intended.

The complexity of the inputs and user experience required to properly use the calculator is inconsistent with it being a planning tool. The complexity also makes it difficult for the user to operate the system and increases the probability that different users will arrive at different ERSP values for the same system. The lack of outputs in the USCG Inland ERSP Calculator limits a user’s ability to understand how to optimize a recovery system. The calculator focuses exclusively on feeding the skimmer, which is often only a portion of the overall response strategy on inland spills. The USCG Inland ERSP Calculator is neither tied to a regulatory planning standard nor is it site-specific in nature, making its importance to an inland planner or OSRO questionable.

Recommendation 2.1: USCG should clarify the capabilities of the Inland ERSP Calculator.

It will be important to make sure that users (e.g., inland planners) understand exactly what the inland calculator is and is not providing to them. Specifically, the calculator is designed for determining ERSP for only certain situations (or circumstances), which are often a small subset of an overall inland response. Planners should not lose sight of all other response measures and should also recognize that the output is not currently linked to the existing regulatory planning standards.

Finding 3: The USCG Inland ERSP Calculator is unable to properly characterize a response system.

Recommendation 3.1: The calculator should be expanded to allow users to include storage characteristics and system-appropriate nameplate recovery values.

The lack of ability to define storage characteristics is inconsistent with ASTM Standard F1780-18 on calculating ERSP, inconsistent with the methodology in the BSEE ERSP Calculator, inconsistent with existing modeling capacity, and it ignores a major component (i.e., storage) in the regulations governing system requirements (e.g., existing OSRO classification system). In remote locations, storage can be the limiting factor. Even in more accessible locations, the time required to change storage units is important to understanding a system’s potential performance. With only the daily ERSP being provided as an output, users are not able to estimate how often storage units would need to be changed. Understanding the role of storage in a recovery operation can also be a valuable learning opportunity for inexperienced planners.

The reduced ability to input a nameplate rating is also inconsistent with the BSEE ERSP Calculator. A dropdown menu allows for the selection of one of three recovery ranges rather than requiring the user to input the actual nameplate value on the skimmer to be simulated. While the values provided appear to cover a range of recovery rates, the calculations, such as Maximum Effective Swath Width (MES), use fixed values of 250, 1,000, and 2,500 gallons per minute (gpm). It was difficult to determine the values used as fixed recovery rates when the user provides a recovery range. It was also difficult to understand why so much detail was allowed for Throughput Efficiency and Recovery Efficiency, yet the Nameplate rating was reduced to three allowable values when Nameplate rating is easily discoverable and the three categories can lead to significant overestimation or underestimation of system potential. For instance, it is possible to

select a Minimax skimmer (manufacturer lists at 20-gpm recovery)¹ and the minimum recovery considered in the calculator is 250 gpm and the calculator allows the skimmer to be given a recovery rate of 2,500 gpm. This means the recovery using this skimmer is at least 10-fold and up to a 100-fold overestimation. Accurate values of the Nameplate capacity are critical for determining the MES and actual ERSP. The Nameplate capacity can be a user input rather than a group value.

Finding 4: An Oil Behavior Module is a significant departure from the BSEE ERSP methodology and a component of the USCG Inland ERSP Calculator that makes it more complex and less transparent. It creates an impression that the USCG Inland ERSP Calculator can estimate a potential system recovery capacity in a specific scenario under specific conditions. In fact, all the approximations embedded in the calculator may likely result in an estimation considerably different from the scenario envisioned by the user without giving them an indication of the uncertainty.

Recommendation 4.1: Compare the outputs of Oil Behavior Module to 40 C.F.R. Part 112, Appendix E as a way to understand the potential differences between these approaches and their implications.

Recommendation 4.2: If the intent of the calculator is to evaluate a maximum recovery potential of the recovery system in a nongeographic and not scenario-specific situation, the inputs related to the scenario and oil behavior should be limited to the minimum, similar to the BSEE ERSP Calculator, so that calculation is truly focused on the recovery system, not the environment. If the intent of the calculator is to predict potential recovery under a specific scenario and potentially a geographic location, the calculator should more accurately reflect the location and scenario-specific details of oil fate and behavior.

Such adjustments could be accomplished by linking the calculator with modern numerical models (e.g., SIMAP used by the project team to generate lookup tables for the Oil Behavior Module) to ensure that it benefits from current modeling advancements and best practices. Similar advancements could come from the application of real-time and historical data modules, such as those accessible in the U.S. Geological Survey’s National Water Dashboard (see Appendix C).²

Calculator Inputs

Reducing the number of variables not related to the recovery system would lower the expertise level required from the user, increase transparency in calculations, allow easier comparison between systems and configurations, and make the result relevant to any response location.

Finding 5: The number of inputs makes setting up the calculator tedious and confusing.

Recommendation 5.1: Reduce the number of inputs to the minimum number required to perform the calculation. Specifically, consider removing the following:

___________________

¹ See https://www.elastec.com/products/oil-spill-skimmers/drum-oil-skimmers/minimax/#specs, accessed May 11, 2022.

² See https://dashboard.waterdata.usgs.gov/app/nwd/?aoi=default, accessed August 2, 2022.

Recommendation 5.2: The following inputs should be reviewed to ensure that they are properly incorporated:

The USCG Inland ERSP Calculator has several places where there appear to be duplicative inputs and other inputs that appear to only provide suggested values. Because it appears that there are multiple ways to provide inputs, it is not clear which ones are used in the calculator, and it is easy to set up situations that appear to be conflicting, for example, Planning Volume Classification and Source Type, where one can be associated with a vessel and the other a facility.

While the Waterway Configuration and Current/Flow entries are required for the calculations, they do not provide the user with an intuitive sense of what type of system is being selected. Changing the waterway configuration to River and Lake rather than Open and Confined may be more intuitive for users. The committee also questioned the utility of the Number of Days for a continuous spill. Eliminating this input and using 3 days built directly into the calculation is a simplification that could be considered. This would be consistent with the BSEE ERSP Calculator, it would give an estimate of system capacity with both slick types, and it would simplify the input for the user.

The Spill Volume could be a dropdown that allows a user to only select values included in the calculator. Those spill volumes are 50 bbl, 2,500 bbl, and 155,000 bbl (USCG, 2021a). It is potentially misleading to a user to think they can select any spill volume when the calculations only allow for three and the documentation does not tell the user how the Spill Volume entered will trigger which of the three volumes is used.

Recommendation 5.2: The most critical inputs should be selected based on the sensitivity analysis of the input parameters (including those currently excluded from the calculator).

Recommendation 5.3: Restrict inputs to realistic values and provide additional warnings and explanations if the scenario is unrealistic (e.g., if currents are too high for the efficient recovery, or recovery, of Group I oil, which typically would not be recovered mechanically).

Recommendation 5.4: Provide suggested values in the documentation rather than making them part of the calculator interface.

Calculator Outputs

The limited outputs of the calculator make it very difficult to determine if it is providing realistic answers. The USCG Inland ERSP Calculator currently provides only the daily ERSP values unless the customization modules are used. Only having one output makes it difficult for a user to understand what may be limiting the ERSP and how changes in equipment affect the ERSP. It also makes it difficult for users to determine if the output number is realistic. More detailed output provided by the BSEE ERSP Calculator allows the user to learn more about how their recovery system is operating and what the limiting factors are during different time periods. Additional values such as oil thickness, total fluid recovery rate, and oil recovery rate are currently estimated in the calculator but not provided to the user. Providing an output like that provided by the BSEE ERSP Calculator should be an aim of improvements to the existing USCG Inland ERSP Calculator.

Finding 6: The outputs are not sufficient to allow a user to evaluate system performance in the USCG Inland ERSP Calculator.

Recommendation 6.1: The calculator outputs should be expanded to give a better indication of the parameters affecting the recovery and to be consistent with the outputs of the BSEE ERSP Calculator.

The user is given only information on the volumes of oil recovered by skimming operation but never an understanding of where the rest of the oil has gone or the recovery rates of water and oil. This limits the user’s ability to put the amount of oil recovered in context of the amount of oil released. The core of the modeling methodology underlying the calculator is the conservation of oil mass released. It is suggested that the output of the calculator provides the oil mass balance with time: oil mass released, oil mass evaporated, oil mass trapped along the shoreline, oil mass potentially available for skimming, oil mass recovered by skimming over time, and recovery rates of water and oil. This will allow the user to determine the percentage of oil that is recovered, and recovery rates, based on both the original spill volume and the amount of potentially recoverable oil. This helps the user put the calculator’s estimates in context of the spill event and the response option selected. It also makes the calculator a better learning tool since it is aimed at people with limited knowledge of oil spill response. The expansion of the outputs will improve the calculator’s consistency with the BSEE ERSP Calculator and model capabilities.

Recommendation 6.2: Add capability to explicitly address impact of uncertainty in model input on model output of predictions of oil recovery.

In reviewing the Conceptual Model, Design, and Verification and Validation documents for the USCG Inland ERSP, it is clear that there are uncertainties associated with the calculator’s inputs and predictions. These are inherent in the development of the tool, which is focused on planning and not designed for use in estimating results for actual spill events. With the addition of options made necessary to consider inland areas with open versus closed and stagnant versus flowing systems, the level of uncertainty has increased from the BSEE ERSP Calculator to the USCG Inland ERSP. This uncertainty has been noted in the verification and validation studies performed as part of the calculator’s development. The present committee has also experienced this repeatedly in their testing of the system.

A prior review of the offshore EDRC by the National Academies in 2013 had similar concerns that the program did not adequately address uncertainties in the input to the calculator. That review committee implemented a Monte Carlo procedure to allow the calculator to estimate the uncertainty in the calculator predictions based on user-specified uncertainties in the input. The 2013 committee provided several examples showing how it would work. Their report provides a template for the graphical user interface for the calculator, showing both the nominal value for each parameter as well as its uncertainty (see NRC, 2013, fig. 2B). Their report recommends the development of a user manual containing guidance on how to specify the uncertainty for each parameter of interest. Alternatively, the plan holder could use other sources to specify the uncertainties, provided that they can be verified and documented.

The implementation of the Monte Carlo method, or some similar procedure, may explicitly represent the uncertainty in the predictions of the inland calculator. The guidance on uncertainty will emerge from the user community as the calculator becomes available for use. Obtaining this type of information will also emerge from the proposed independent third-party evaluation as discussed in Recommendation 1.1.

Recommendation 6.3: Provide estimate of relative density difference between spill and receiving water.
In representing the oil that is spilled, the USCG Inland ERSP Calculator uses an oil group classification as summarized in Table E-1 of the Design document (USCG, 2021a). Group V oils are not included in the list since it is assumed that they are denser than seawater and thus would sink and not be recoverable by skimming systems.

One issue of concern is that the calculator does not consider the fact that the density of the receiving water can vary over the area of application. Freshwater has a nominal density of 1 g/cm³. The BSEE ERSP Calculator did not need to consider this effect since the density of seawater (20°C, 33.4 ppt, 1.024 g/cm³) at the sea surface offshore does not vary much over typical areas of application. For the USCG Inland ERSP Calculator, the density can vary from values characteristic of offshore waters to those where the receiving water is fresh. This variation can be at the surface but is also possible with depth as a result of salinity intrusion in tidally dominated rivers and streams. In implementing the calculator, the USCG might consider requiring the user to specify the density of the receiving water at the surface. The calculator could then estimate the relative density difference between the oil that is released and the receiving water. The larger this value, the more likely that skimming operations would be successful, and the smaller the value, the greater the uncertainty in the performance of skimming systems. For example, for spills in freshwater, the relative density difference at the receiving location for Group I oils is (1.000 g/cm³ − 0.800 g/cm³)/1.000 g/cm³ = 0.20 (20 percent) and for Group IV oils it is 0.025 (2.5 percent). For brackish waters (17 ppt; density of 1.011 g/cm³), the relative difference is 20.8 percent for Group I oils and 9 percent for Group IV oils. For typical oceanic waters (34 ppt; density of 1.024 g/cm³), the relative difference is 21.8 percent and 4.7 percent for Groups I and IV, respectively. The conditions that give the most favorable environment for skimming are low-density oils floating on oceanic waters, and the least favorable are-high density oils floating on freshwater.

OTHER CONSIDERATIONS AND NEW METHODOLOGIES

Finding 7: The USCG Inland ERSP Calculator intended use and users and their intended qualifications are not clear to the committee. The calculator currently sits in the center of the continuum (Figure 2.1), which makes it unsuitable for planning or prediction purposes. There are some improvements to the existing calculator that may be considered as necessary to allow the calculator to have value as either a planning tool or an educational or predictive tool.

Recommendation 7.1: If the intent of the USCG Inland ERSP Calculator is to serve as a planning tool, and be focused on evaluating a maximum recovery potential of the recovery system in a nongeographic and not non-scenario-specific situation, the inputs related to the scenario and oil behavior should be limited to the minimum similar to the BSEE ERSP Calculator, so calculation is truly focused on the recovery system, not the environment (as described in Recommendation 4.2).

Recommendation 7.2: If the intent of the calculator is to be an educational or predictive tool and be able to predict potential recovery under a specific scenario and potentially a geographic location, the calculator should more accurately reflect the location and scenario-specific details of oil fate and behavior (as partially noted in Recommendation 4.2) as well as response strategies and equipment deployment tactics.

Recommendation 7.3: Reach out to additional specialists for technological and methodological improvements. Select examples for potential consideration are summarized below and in the text following Recommendation 1.1.

Additional descriptors of the waterway size and geometry can improve capabilities of the USCG Inland ERSP Calculator. Knowing the width and length of a spill in a river is important in allowing the calculator to estimate shoreside recovery in a more appropriate manner. Since in a moving river the slick will pass by the point where the shoreside recovery system is located, the amount of time that the slick is passing can be determined from the length of the spill and the speed of the river. This allows the recovery potential to be limited by the time oil is available to the system before it must be recovered and moved to another location. As an educational tool, it allows users to understand if they are likely to need to have multiple deployments of shoreside recovery systems by shifting from the infinite spill volume assumed in the existing calculator to determining recovery from a fixed volume spill.

Another benefit that comes from having the waterway width is that the percentage of the waterway covered by the shoreside recovery system can be determined. If the swath width is 10 percent of the waterway width, most of the oil will pass outside of the reach of the skimming system. If the swath width is 100 percent of the waterway width, more of the slick can be recovered. A very knowledgeable user could imitate this capability by adjusting the throughput efficiency to account for the portion of the waterway not swept by the skimming system. Doing it properly is not something that should be expected of a relatively inexperienced user .

Adding the area of an enclosed system as input provides an opportunity to better estimate slick thickness and provide a means to have recovery from a stationary recovery system. A stationary recovery system could be allowed to recover oil from a single location until the slick reached some critical thickness at which point it would be assumed that the oil would stop spreading to the skimmer.

Another potential modification is to allow the user to provide the oil thickness and percent emulsification as user design inputs. This would allow use of a geographically specific Oil Behavior Module to derive the inputs required for the ERSP calculations or to examine changes in the ERSP under a full range of potential conditions.

Finally, the USCG might consider adding the ability to calculate the recovery by nonadvancing systems. The ability to collect oil at a fixed location with a nonadvancing system is a prevalent strategy in an inland response.