Statistical Analysis of Massive Data Streams Proceedings of a Workshop (2004) / Chapter Skim
Currently Skimming:
Pages 16-27
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 16... ...
The National Climatic Data Center is the official repository in the United States of all atmospheric weather-related data. As such, we get things like simple data streams, the automatic observing systems that give you temperature, moisture, cloud height at the Weather Service field offices.
|
From page 17... ...
Right now, we have got a little over one pedabyte and daily we are probably ingesting something like a terabyte. The biggest data set coming in now is that we are getting the next red data -- this is the weather radar data -- from about 120 sites throughout the United States.
|
From page 18... ...
In the middle are those ovals that we want to actually work on, understanding the satellite data and then understanding the processes of climate, and then, in fact, improving modeling. As an operational service and product agency, NOAA is responsible for not just analyzing what is going on but, foolishly, we are attempting to predict things.
|
From page 19... ...
$ ~~' ~~ aim... So, the first application, detection of long-term climate trends using environmental satellite data, the issue of global warming has really surfaced in the last ~ 0 years.
|
From page 20... ...
~ ~ · ~ USA_ t - ... ..{ ~~ ~ Din - E faded .~ ~~_~ ~~ ~~ ~.~ ~~ ~~.~ ~~ So, creation of seamless time series, you have here three different channels of data from a satellite, channel 8, that is an infrared window, channel 10 is actually a moisture channel in the upper atmosphere, a so-called water vapor channel, and these channels -- 10, ~ I, 12 -- are all water vapor channels.
|
From page 21... ...
Then, this is a long-term sort of diagram of the health of the satellite data set again. These are just simple statistical quantities, but very helpful for scanning out bad data.
|
From page 22... ...
The top two panels are the spatial patterns of empirical orthogonal analysis of precipitation, and then this is water vapor. What we have done here is that we have subtracted the annual harmonics from the time series of the data sets, so that we can look at interregna!
|
From page 23... ...
Nino. So, this is sort of easy, when ~ see these beats of this time series.
|
From page 24... ...
This confidence interval is just computed at each god point time series, and it is both the fit to the linear trend, plus a red noise persistence term. That is just a simple lag one autocorrelation, and then fit to the significance in the length of the time series.
|
From page 25... ...
25 did this. This is how they predicted -- the first predictions were not numerical models, but were statistical techniques where we reduced the dimensionality of the data set, and then looked at the propagation speed.
|
From page 26... ...
We have used those now to look at onsets of changes in the monsoons and regimes that favor or suppress hurricane activity in both the Atlantic and Pacific Oceans. Just real quick, this is just data mining techniques.
|
From page 27... ...
~.~,¢~ ~ mt ~ ~~m ~~.~d,.~,~ ., ma$~ - ,;~., ~,~,:~.~,~ ., ~~,~,~.~,~ ,.,~,.~.~.~ . ~~' - - it - b em ~~ So, concluding, we have got these massive data streams that are going to continue increasing geometrically in the next ~ O to ~ 5 years.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.