RAWS/RTMA

The key to any useful scientific study is a thorough description of the analysis tools and methods used for data acquisition, processing, and analysis. As such, this section will first describe the main goal of the project, then the steps used to work toward that end.

The main purpose of this project was to determine biases in RAWS observation data at both individual stations and across all RAWS observing platforms within the Great Lakes and Northeast regions. RAWS are located throughout this domain, and do not have the same siting requirements (e.g., open field) that ASOS sites have. Thus, there may be inherent biases in the output from the RAWS data. In the end, we hope to help fire managers and others who rely on accurate surface observations by providing them with the best available information.

However, to validate and verify the RAWS observations, we needed to find a "truth" with which to compare the RAWS data. Operational models were quickly discarded, as their relatively poor resolution (both spatially and temporally) and lack of site-specific measurements rendered them insufficient for this study. Satellite observations were limited to only calculated temperatures, so we couldn't test all of the instruments at a RAWS site. Plus, clouds could obscure surface readings, so satellite-derived observations were similarly deemed insufficient.

The best dataset for comparison with RAWS turned out to be the Real-Time Mesoscale Analysis prepared by NOAA/NCEP. Given the high temporal and spatial grid spacing of the RTMA (hourly and 5-km, respectively, for this study), we could better determine where the two data sets differed.

However, the RTMA has plenty of issues itself. It is the best mesoscale analysis of surface parameters currently available, but lacks verification where surface observations are not available. Thus, discrepancies between RAWS and RTMA could be caused by errors in either (or both) datasets. This fact must be kept in mind throughout the remainder of the analysis.

Data acquisition for each of these products is limited to very recent years, as the RTMA has only been produced operationally for the last few years. Thus, this study will look at two years of data, from 1 August 2008 to 31 July 2010. Expansion of our results to include more years is near the top of our to-do list.

RAWS observations include many fields (see About RAWS at right). However, fire managers (and most end users) care most about temperature, wind speed, relative humidity, and precipitation. These fields were extracted and calculated from the raw RAWS data (apologies about the pun) every hour for the duration of the study.

Some of the stations are missing huge chunks of data. Stations missing more than 10% of possible observations have been excluded from the study to help make the results more robust.

Once the RAWS data were properly trimmed and quality-checked, the RTMA data was bilinearly interpolated to the RAWS grid point. Now, each qualifying observation site has an hourly time series of temperature, wind speed, and relative humidity from both RAWS and RTMA.

The next step required comparison of the two datasets. To accomplish this, we have calculated some basic error statistics for each station for the entire duration of the study period (e.g., bias, MAE, RMSE). While useful for an overview of the differences between RAWS and RTMA, we wanted to delve further into possible issues/problems on a site-by-site basis.

We wanted to determine the range and frequency of observations of each meteorological field at each of the RAWS stations, so frequency diagrams have been created at each station. Then, to see if the observed range matches well with the RTMA, a similar plot was created using the objectively-analyzed dataset.

We also wanted to determine if there were other systematic biases that could be found in the data. Since there are only two years of data available in this study, monthly or yearly metrics are of questionable validity. However, there are enough observations to determine if there is an hourly bias. So, another set of plots was made for each variable to compare the RAWS observation with the RTMA "expected" value. Some stations show clear biases at every hour, others show a changing bias depending on the time, and still others show little to no bias. Further investigation of these biases is planned in future work.

One other goal of our future work plan is worth mentioning here. In the coming months, we plan to collect two more years of RTMA and RAWS data. Increasing the number of observations will strengthen confidence in the output statistics.