Summary

New York City’s water supply system is one of the oldest, largest, and most complex in the nation. It delivers more than 1.1 billion gallons of water each day from three upstate watersheds (Croton, Catskill, and Delaware) to meet the needs of more than eight million people in the City; one million people in Westchester, Putnam, Orange, and Ulster counties; and millions of commuters and tourists who visit the City throughout the year. Approximately 95 percent of the total water supply is delivered to the City by gravity (without pumping). The Catskill and Delaware portions, which make up about 90 percent of the supply, receive no filtration or treatment other than disinfection, except for rare instances of high turbidity when a coagulant is added to increase deposition of suspended solids. The remaining 10 percent of the supply comes from the Croton watershed and receives treatment via filtration.

The drinking water supply is managed by the Bureau of Water Supply within the New York City Department of Environmental Protection (NYC DEP). To continue to avoid filtration of the Catskill/Delaware portion of the water supply, in 1997 the NYC DEP established and implemented a comprehensive Watershed Protection Plan. In 2007, NYC DEP was required by the New York State Department of Health to further examine its control of turbidity in the Catskill portion of the water supply, including both structural improvements to the system and operational changes. The Operations Support Tool (OST) was developed as part of these efforts.

OST couples models of reservoir operations and water quality; it uses real-time data on streamflow, snow pack, water quality, reservoir levels, diversions, and releases; and it incorporates streamflow forecasts—all

in order to predict future reservoir levels, water delivery to customers, and water quality within the system. These predictions inform the system operators, who then make decisions based on the most current data and forecasts. By running OST, system operators can account for changing environmental conditions and streamflow forecasts, providing valuable guidance for reservoir operations.

An ad hoc committee of the National Academies was convened in 2017 to review the use of OST in current and future reservoir operations, including the following tasks:

1. The Committee will review the City’s use of OST for water supply operations, including managing elevated turbidity, and will consider potential ways in which the City can more effectively use OST.
2. The Committee will evaluate the performance measures the City uses to assess the efficacy of the Catskill Turbidity Control Program, making recommendations for additional performance measures, if necessary.
3. The Committee will review the City’s plan for use of OST in evaluating proposed modifications to the Catalum State Pollutant Discharge Elimination System (SPDES) permit, as well as alternatives to be considered in the associated environmental review.
4. The Committee will review NYC DEP’s existing studies of the potential effects of climate change on the City’s water supply to help identify and enhance understanding of areas of potential future concern with regard to the use of OST.

Chapter 2 describes OST, discussing its strengths and weaknesses and making suggestions for how it could be improved for decision-making purposes. Chapter 3 addresses the Catskill Turbidity Control Program, particularly metrics that could be used to determine the performance of measures undertaken as part of the program. Chapter 4 considers the City’s plan to use OST as it develops an environmental impact statement for modifications to the Catalum SPDES permit. Chapter 5 discusses how future climate change in the New York region will affect the ongoing use of OST in water supply operations. Finally, Chapter 6 considers future uses of OST that go beyond its use as a daily operations decision support tool. Each chapter has conclusions and recommendations that synthesize more technical and specific statements found within the chapters; the most important conclusions and recommendations are repeated in this summary. This report is intended to be useful to the NYC DEP, other water suppliers, the U.S. Environmental Protection Agency and state regulators, academic and consulting communities, and other stakeholders.

DESCRIPTION OF THE OPERATIONS SUPPORT TOOL

OST informs management decisions by formally quantifying the routing of water inputs through the water supply system to New York City and other users. The accounting of water routing for the New York City water system is complicated by multiple competing objectives such as meeting water demands, controlling floods, and maintaining downstream releases. OST simulates decisions that closely approximate the decisions that a knowledgeable operator would make after considering and processing all of the available data, constraints, and possible operational controls. Manual completion of this task for the complex New York City water supply system may not allow for a complete evaluation of operational options. Therefore, it is done by utilizing water quantity and water quality models that provide probabilistic simulation of flows, reservoir levels, and water quality under a range of possible future conditions for a future time period, which could be weeks, months, or years.

Figure S-1 summarizes the conceptual framework of OST. Near-real-time data (shown in green) describe the current status of the water system, providing “situational awareness” for NYC DEP as well as the initial system conditions for model forecasts. Also in green are the ensemble forecasts of future streamflows that are a major input to OST. Models (in blue) for water quantity (OASIS) and water quality (CEQUAL-W2) combine forecasts of future water inputs (precipitation and streamflow) and system attributes to simulate future water flows and reservoir storage conditions. The model produces an ensemble of outputs (in red), such as diversions, releases, reservoir levels, and water quality conditions. Managers can use the results from OST simulations to develop conditional probabilities for specific events that may occur, providing a basis for assessing system reliability, understanding metrics of system performance, and making day-to-day decisions about system operation.

OST is one of the most advanced and complex support tools for water supply operations of its kind in the world. The Committee strongly endorses the use of probabilistic forecast tools to inform NYC DEP’s management decisions, allowing it to explicitly balance the risks of water shortage against the risks of having too much water (and the turbidity issues that come with high-flow events). Moving forward, OST offers a dynamic water routing platform that is both adaptable and capable of continuous improvement if properly resourced with data, validation, and most importantly, continued maintenance and growth of institutional expertise. The following conclusions and recommendations are intended to improve the quality of data inputs to OST and consequently bring about improvement in OST outputs and the ability of NYC DEP to make better use of OST now and in the future.

The historical data used in the Hydrologic Ensemble Forecasting Service (HEFS) to create the ensemble streamflow forecasts need to be updated now to include the most recent 20 years, and this updating needs to occur on a regular schedule into the future. There are important trends from the last 20 years (see Appendix A) that are not being captured by use of data only from 1950 to 1997. In light of the changes in climate and resulting hydrology in the region that are already underway, it is crucial that the climate data that drive the ensemble forecasts be kept up to date. Updating these data to near present is imperative now, and a plan for a regular cycle of updates should be a part of the HEFS system inputs to OST.

NYC DEP and the National Weather Service, which operates the weather and hydrologic forecasting model components of the system, need to critically evaluate the appropriateness of each of the key components of the ensemble streamflow forecasts. The hydrologic model used by the Middle Atlantic River Forecast Center, Continuous Antecedent Precipitation Index (API), is not state of the art, and serious consideration should be given to replacing it with the Sacramento Soil Moisture Accounting Model that is used by all other River Forecast Centers. The Climate Forecast System version 2 appears to provide no precipitation forecast skill (beyond about ten days), and the Committee questions its use in OST. An approach driven by historical climatology would entail far fewer assumptions and much more straightforward processing. When longer-term weather forecast models can be shown to provide significant skill with respect to precipitation, then it would be justified to include such models in the system.

Probabilistic validation results for the ensemble streamflow forecasts need to be published for the New York City watershed region. The Committee does not recommend traditional validation on point estimates, but rather on the conditional distribution of the ensemble. The Committee understands that the National Weather Service has a set of appropriate, ensemble validation exercises underway at present. The Committee urges NYC DEP and the National Weather Service to publish these results and provide for a continuing set of validation products.

The NYC DEP should continue in its efforts to update the streamflow-turbidity relationships and include their uncertainty. The Committee recognizes that including additional ensembles of turbidity inputs, representing the uncertainty of turbidity level traces for any given streamflow trace, would add a great deal of computational time and complexity to the OST model. However, updating the streamflow-turbidity relationships to be based on the most recent data, to reflect recent trends, and to determine if

there is any seasonality in these relationships would be valuable and could result in some improvements to the OST approach.

The Committee encourages the NYC DEP to examine the sensitivity of OST simulations to the values of the weights and penalties assigned within the OASIS model. The capability of OASIS to simulate the decisions that would be made by expert system operators is likely to depend on the values of the weights assigned to different flows and reservoir volumes, which reflect relative priorities of system performance (e.g., water storage volume, water quality, and water releases). Understanding and communicating the sensitivity of model outputs to decision variable weights should improve model performance and the confidence of stakeholders in the model simulations.

The Committee encourages the NYC DEP to continuously improve OST so that it remains operationally relevant, particularly in the face of growing environmental pressures such as climate change and more stringent regulations. Since its inception, the institutional memory and expertise of NYC DEP staff have been critical to refining the numerous data inputs, representing the operational complexities of the water supply, and weighting the various model components within the OST framework. The biggest risk to OST failing to fulfill its role as a water management decision support tool is a lack of continuous improvement, knowledge transfer, and knowledge mobilization through succession planning to ensure that the best and most appropriate data and operational insights are used to inform the inputs, the models, and the other relationships that OST relies upon.

METRICS FOR THE CATSKILL TURBIDITY CONTROL PROGRAM

The Committee was asked to evaluate the performance measures the City uses to assess the efficacy of the Catskill Turbidity Control Program and to make recommendations for additional performance measures. The Catskill Turbidity Control Program is a collection of activities that fall generally into three categories: (1) turbidity source control measures within the Schoharie and Ashokan watersheds; (2) improvements to infrastructure, most notably along the Catskill Aqueduct; and (3) operational changes, some of which are facilitated by OST. Because of the diverse nature of the activities that make up the program, as well as the long time period over which they have been or are being implemented, it is challenging to determine whether the collection of activities undertaken to reduce turbidity loading to Kensico Reservoir has been effective. The following recommendations are therefore made.

NYC DEP should be putting substantially more effort into ongoing assessment of the overall effectiveness of the Catskill Turbidity Control Program by conducting more data analysis. Metrics that should be evaluated on a continuing basis include (1) turbidity levels in diversions from the reservoirs and in aqueducts, (2) the frequency and/or duration of alum treatment events, and (3) mass of alum used during treatment events. The NYC DEP’s excellent network of data collection systems provides a strong basis for such evaluations. NYC DEP needs to take the next step of analyzing the data to improve understanding of the dynamics of the system, which may be helpful in improving existing control strategies, optimizing the data collection systems (learning which sensors provide the most important information), and demonstrating the effectiveness of the Catskill Turbidity Control Program.

Data analysis should start with exploratory data analysis to help the analyst understand the sources of variability in the data and then move beyond that with formal statistical modeling that attempts to characterize the deterministic and random components of the variability. The aim is ultimately to describe the nature of trends that may be happening due to the Catskill Turbidity Control Program. In particular, analysis of covariance is a useful statistical tool that can help quantify and test for differences between the period prior to introduction of the Catskill Turbidity Control Program and the period after the program was put in place. Such techniques are ideal for detecting step changes when the variable of interest (turbidity) is strongly driven by a random variable such as precipitation or discharge.

NYC DEP should move forward with its plan to “compare actual operations and water quality over specified periods of time to a no-action scenario for the corresponding period.” The no-action scenario would be generated by a configuration of OST that simulates system behavior in the absence of the infrastructure and OST improvements. Prior to this exercise, it will be important to establish the ability of OST to represent system performance by comparing its predictions of performance to what has actually occurred.

USE OF OST WITHIN THE ENVIRONMENTAL IMPACT STATEMENT FOR THE MODIFICATIONS TO THE CATALUM SPDES PERMIT

NYC DEP has proposed modifications to its SPDES permit for adding alum to the Catskill Aqueduct just prior to Kensico Reservoir for the purpose of controlling high turbidity. These modifications include the following three proposed actions: operation of the Ashokan Release Channel, dredging of alum floc from Kensico Reservoir, and the addition of alum to

the Catskill Aqueduct. These proposed actions will soon be undergoing a formal analysis of their environmental impacts, including consideration of 19 impact areas and 18 structural, nonstructural, and operational alternatives to the proposed actions. The Committee was asked to review the City’s plan for using OST in the resulting environmental impact statement (EIS), which is scheduled to be completed six months after the publication of this report.

The NYC DEP’s plans to use OST in the EIS are systematic and appropriate. The EIS team is on a good path and is applying OST appropriately. OST is particularly well suited for providing information on water quality and hydrologic conditions needed in the EIS analysis and is capable of simulating the proposed actions and the 18 alternatives. OST can simulate a range of conditions for different actions and alternatives and can provide output that directly addresses the four performance metrics for screening alternatives in the EIS.

The EIS team needs to be more explicit and transparent in disclosing specifically how they intend to use OST in the EIS. For example, details are needed regarding how they will use the OST output as inputs to other tools to evaluate and quantify impacts for non-OST parameters and other considerations in the 19 impact areas. The EIS team is encouraged to critically review the models, tools, and methodologies that will be linked to the OST output for their applicability and reliability. The EIS team should also consider whether the specific goals, weights, and writing of the code within OASIS may introduce implicit bias when evaluating the various alternatives being considered in the EIS.

The EIS team needs to expand the range of hydrologic inputs it uses in OST and not be limited to historical data through only 2013. Input data should include all years up to the most recent available, because conditions are changing. In addition, it is important to add additional input scenarios that consider the potential impacts of climate change and sensitivity analyses. It is quite possible that the ranking or selection of an alternative might change under different climate conditions (e.g., more rain or extended droughts) or under different assumptions simulated in the sensitivity runs.

USE OF OST IN A CHANGING CLIMATE

Future climate conditions are likely to affect water supply operations for New York City. Changes in the seasonality, frequency, duration, and magnitude of precipitation events and streamflow in the Catskill and Delaware watersheds over the last several decades have been detected in

observational data to varying degrees. To adapt to future changes in precipitation and streamflow patterns, NYC DEP and other water utilities need statistical methods, simulation modeling, and other analytical tools to help them assess trends over recent decades, project a range of possible future conditions, and estimate impacts of these future conditions on water quantity and water quality. The Committee considered whether OST in its current configuration can continue to be used operationally as climate conditions change. It also discussed how OST, when used in simulation mode, can be applied effectively to assess vulnerabilities in the New York City water supply system that may arise from future climate change. Because NYC DEP is already planning for climate change, the following recommendations are meant to support and enhance the City’s use of OST so it can ensure the most reliable, highest-quality water supplies for the New York metropolitan region in the future.

Given the Committee’s review of the NYC DEP’s and other studies on climate change in the watershed region, there is every reason to expect that OST can continue to be used as an effective tool for operational support into the future if the Chapter 2 recommendation to update OST with the most recent data is taken. Regularly updating OST with the latest climate and hydrologic data so that the model parameterizations reflect current trends is essential to prepare for the future under changing climate conditions. Recent studies suggest that increases in summer precipitation and streamflow events and earlier snowmelt are already occurring in the NYC watershed region and will likely continue and be amplified in the near future.

As OST is used in simulation mode in future climate change studies, it will be important to consider a range of approaches as inputs to OST, including climate and hydrologic models, historical climate analogs, and current conditions and trends. Together, climate modeling and observational approaches provide effective and complementary inputs to near- and long-term water supply planning for the region. The combined results from these multiple approaches should be used to assist in decision making for the future.

NYC DEP should consider structuring future planning studies to identify the range of changes in hydrologic and water quality conditions that would trigger the need for operational changes, and then estimate the likelihood of such conditions. For example, analysis could focus on the level of turbidity in the Catskill system that would stress or exceed the ability of the current suite of infrastructure and operational practices to constrain it, and then estimate the likelihood of climate and hydrologic conditions that might bring this about.

NYC DEP should consider coordinating with other New York City and regional agencies to create and update a Climate Resiliency Indicator and Monitoring System for the New York metropolitan region and assess climate change. This will track, on an ongoing basis, the frequency of occurrence of weather conditions that pose the highest risks to the water supply system. In addition, NYC DEP should consider participating in a regular Climate Change Planning Study in conjunction with the New York City Panel on Climate Change. This could provide ongoing benchmarks for long-term climate change planning for NYC DEP, allow for advanced preparations, and improve the rationale of investments and operational changes. Such benchmarking studies could usefully employ decision making under uncertainty methods to inform planning.

FINAL THOUGHTS

Over ten years ago, NYC DEP undertook the development of OST as part of a suite of actions intended to improve management of turbid waters flowing out of the Ashokan Reservoir in the Catskill system. From its earliest stages of development, NYC DEP recognized that OST’s capability to account for the movement of waters of various quality through the tunnels and reservoirs of the Catskill and Delaware systems offered a tool for exploring a wide range of critical operational and planning questions. The following applications of OST, some of which are already being pursued, could support or enhance NYC DEP’s mission.

Use of OST to Capture Existing Staff Knowledge and Expertise and as a Training Tool

When used in position analysis mode, OST draws on the expertise of its engineers and operators regarding how the water supply system should operate. This expert knowledge is captured both within OST’s model code and through the use of specific operations control language (OCL) instructions that accompany each run of OST. NYC DEP would benefit from a more systematic, documented approach to capturing information gained from OST runs. Furthermore, by enabling retrieval of prior runs of OST and “reverse engineering” those runs, current and future operators would gain insights into how and why the system responded under, for example, varying streamflow and turbidity forcing conditions, management of extreme rainfall events, reservoir release strategies under persistent drought, and work-arounds to cope with major system shutdowns. The Committee encourages NYC DEP to continue to build on its growing base of understanding of system performance under a range of challenging conditions by making maximum use of the OST position analysis runs.

As an extension of OST’s use as a means of capturing staff knowledge and expertise in operating the water supply system, OST also could provide a sophisticated simulator for training staff to cope with “virtual” floods, droughts, and other off-normal or extreme events. One approach to using OST in a training mode could be to generate stressing scenarios (e.g., intense high flows or an extended period of low flows) using synthesized time series of streamflow and other input data to OST. Alternatively, NYC DEP could draw on historical input data, actual systems performance, and archived OST runs over the course of particularly notable historical events or time periods since OST’s inception in early 2011.

Use of OST to Manage Other Water Quality Metrics and Compliance with the Filtration Avoidance Determination

OST in theory could be exercised to estimate other water quality measures beyond turbidity. For example, with sufficient observational data, it might be possible to incorporate precursors of disinfection byproducts, pathogens, and other water quality parameters into the W2 models within OST. The expansion of OST’s scope in simulating other water quality metrics could enable its use beyond system operations to include analysis of potential impacts of modifying current practices or implementing new actions under the authority of the Watershed Protection Program. The Watershed Protection Program currently includes a wide range of land-use restrictions and other watershed practices to reduce the impacts of human and animal contaminants, such as fecal coliforms, Cryptosporidium, and Giardia, on water quality. OST could simulate water quality impacts of existing and new program elements under a range of assumptions regarding systems operations, future land use, and other changes within the watershed.

Use of OST to Help Illuminate and Frame Research Questions

OST serves as a repository of current understanding of the dynamics within the water supply system, but can also serve to highlight research questions that could improve its usefulness to NYC DEP. One example relates to structuring physical experiments to measure sediment fate and transport within the Catskills system to provide an improved understanding of how turbidity correlates with precipitation events of varying antecedent conditions, duration, and intensity. OST could be run in a mode in which turbidity is assumed to be an uncertain parameter rather than a deterministic input. This approach could provide some insight into the sensitivity of downstream reservoir operations and diversions to assumptions about the distribution of turbidity under given precipitation and streamflow forecasts.

The Committee encourages NYC DEP and its academic partners to consider structuring a more formal research program around OST and to take steps to improve documentation and timely publication of results in the peer-reviewed literature.

Communication About OST and How It Is Used

Given the complexity of OST and the many assumptions built into its code, NYC DEP is challenged to provide the public with a clear and compelling explanation of OST’s workings, and how it is used to inform decision making in accordance with NYC DEP’s mission of providing drinking water to New York City. As NYC DEP’s use of OST continues to mature and result in additional insights to a wider range of operational and regulatory questions, the need for transparency and clarity about what OST is and what OST does will only grow. As important as developing clear explanations of what OST does and how it is used are developing equally clear explanations of what OST does not do and the uses for which OST is not well suited, such as modeling water quality and ecological impacts outside the bounds of the calibrated model.