Climate Data Records from Environmental Satellites Interim Report (2004) / Chapter Skim
Currently Skimming:
4 Data Management Requirements
4 Data Management Requirements
Pages 63-80
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 63... ...
Because the ultimate legacy of long-term CDR programs is the data left to the next generation, the cost of data management and archiving must be considered as an integral part of every CDR program. For reference, other large science programs with multidecadal data access and preservation requirements can spend as much as 20 percent of their budget on data management (NRC, 2002)
|
From page 64... ...
. Long-term success for the CDRs also will depend critically on sufficient metadata, in standard formats, including metadata fully describing the prod uct line and metadata to discuss CDR limitations and to aid in data man agement (dataset lineage, version control, and unique identification pa rameters)
|
From page 65... ...
Data Quality To ensure that the FCDRs and TCDRs are of the best scientific quality, research scientists who understand the data and the meaning of changes in the data and use the data for their own research should play a major role in data management through active involvement in the development of data products and associated documentation. This practical engagement of scientists also allows for the infusion of their scientific perspectives as actual and prospective data users into the data production process.
|
From page 66... ...
This transfer, accomplished through a call for proposals, was a major reason for the success and timely delivery of the EOS standard data products and accounted for the high degree of scientific community acceptance of these products. Planned Evolutionary Upgrades.
|
From page 67... ...
Allowing users to gain ownership of requirements through sponsored workshops to reach community consensus and initiating processes to enable users to prioritize requirements allowed stakeholders to take an active role in the design and thus improve their level of comfort with the EOSDIS core system. Lessons Learned 1.
|
From page 68... ...
. The long, consistent datasets from reanalysis have been extremely valuable to an impressive range of scientific studies and applications, including climate monitoring, climate prediction, applied climatology such as prediction and monitoring of climate related health problems, stratospheric transports and chemistry, and boundary conditions for regional models.
|
From page 69... ...
3. Global use of the reanalysis data has been facilitated by free distribution of the data products on CD-ROM and online, and the provision of the data in formats that are commonly used (e.g., netcdf)
|
From page 70... ...
It is, however, important that CDRs be made available in interoperable formats with certain common standards, such as being self describing, and that there be periodic reconsideration of CDR formats as the underlying data technologies change. Many current format tools have been written by data users.
|
From page 71... ...
. Web service infrastructures are rapidly evolving to permit data access through such digital libraries as the Digital Library for Earth Science Education (DLESE)
|
From page 72... ...
The irreplaceable primary or FCDR data must be preserved in perpetuity, and data policies for superceded TCDR versions should be established so that they may be decommissioned and deleted and the storage resources recovered. Data Security Data management systems must ensure the security of stored data.
|
From page 73... ...
Each TCDR team should develop guidelines to manage the data stream throughout the data lifecycle.1 These guidelines will provide the NPP and NPOESS mission science teams with a roadmap for the orderly transition of the data from production to an active archive and ultimately on to an LTA facility. New operational satellite missions must plan for an orderly process that addresses data archiving, metadata collection, data access, and data delivery as the data progresses through its full lifecycle.
|
From page 74... ...
; · standards (an ISO standard reference model that provides a con ceptual framework and defines a consistent set of standards for all major archive functions and services) ; and · peer-to-peer file sharing (distributed, ubiquitous computer servers networked in a dependable infrastructure that can support nomadic data access and retrieval)
|
From page 75... ...
Such an archive could encompass multiple agencies and government and nongovernmental data centers. Many of the NASA DAACs, for example, not only archive data generated by external groups (including products derived from NOAA polar orbiters)
|
From page 76... ...
ingest with immediate verification of data integrity and quality, and routine ingest and verification of data quality and integrity without tight time constraints. Data processing options include such alternatives as operational products generated within two, seven, or thirty days of ingest or availability of required inputs.
|
From page 77... ...
. The increase in user requests and data volume accompanies an increase in the number of products stored at the NOAA data centers, from roughly 800 in 1998 to nearly 1600 in 2003 (Figure 4-3)
|
From page 78... ...
1800 NODC 1600 (MB) NGDC 1400 NCDC Users 1200 to 1000 800 Delivered Thousands 600 400 Volume 200 Data 0 Jan-96 Jan-97 Jan-98 Jan-99 Jan-00 Jan-01 Jan-02 Jan-03 FIGURE 4-2 Data volume delivered to users.
|
From page 79... ...
SOURCE: Compiled from NOAA data by S Drobot, National Research Council.
|
From page 80... ...
This must involve cooperation among researchers, data and archive managers, data collectors, and primary users. To assist in making decisions on data stewardship in a resource constrained environment, a process should be established for the science assessment of the long-term potential of data and data products.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.