Version 2 NSF network data:
Data Products

The following list gives an overview of Version 2 data products. Click on the links for further information and example files:

• Spectral irradiance at selected wavelengths
  
• Integrals and dose-rates
  
• High resolution spectra
  
• Plots showing comparison of measurement and model
  
• Total column ozone
  
• Cloud optical depth
  
• Albedo
  
• Flags
  
• Model parameters and model log
  
• Wavelength accuracy
  
• Results of Quality Control
  

Detailed description of data products:

• Spectral irradiance at selected wavelengths
  This data product was already part of Volume 0, and is known as "Database 2". The Version 2 style "Database 2" is very similar to the original version, but the relationship between time and wavelength is better resolved.

  Each "Database 2" is complemented by two other databases of identical structure, which provide clear-sky model values and model calculations that take cloud attenuation into account.

  Examples of "Version 2-style" Database 2 (South Pole, Volume 5):
  

  • SPO_v5.2_DB2_meas.csv: Measurements (1.66 MBytes)
  • SPO_v5.2_DB2_clear.csv: Clear-sky model (1.62 MBytes)
  • SPO_v5.2_DB2_cloud.csv: "Cloud" model (1.01 MBytes)
  Notes:
  1- "Left click" will open the example files in Microsoft Excel if that program is installed on your computer. To download the file, "right click" and choose "Save Target as ..." from the context menu.
  2- No "Cloud model" values are given for wavelengths below 340 nm, as total ozone is not an input parameter for the cloud model. "Cloud model" spectra also do not take the sphericity of the Earth into account; spectra for solar zenith angles larger than 75 degrees may therefore underestimate the true spectrum significantly.
  

• Integrals and dose-rates
  This data product was also already part of Volume 0, and is known as "Database 3". The Version 2 style "Database 3" has a much larger number of integrals and dose-rates, and a better assignment of wavelength and time. A description of action spectra implemented can be found here.

  Each "Database 3" is complemented by two other databases of identical structure, which provide clear-sky model values and model calculations that take cloud attenuation into account.

  Examples of Version 2 Database 3 (South Pole, Volume 5):
  

  • SPO_v5.2_DB3_meas.csv: Measurements (2.37 MBytes)
  • SPO_v5.2_DB3_clear.csv: Clear-sky model (2.47 MBytes)
  • SPO_v5.2_DB3_cloud.csv: "Cloud" model (1.29 MBytes)
  Notes:
  1- "Left click" will open the example files in Microsoft Excel if that program is installed on your computer. To download the file, "right click" and choose "Save Target as ..." from the context menu.
  2- No "Cloud model" values are given for wavelengths below 340 nm, as total ozone is not an input parameter for the cloud model. "Cloud model" spectra also do not take the sphericity of the Earth into account; spectra for solar zenith angles larger than 75 degrees may therefore underestimate the true spectrum significantly.
  

• High resolution spectra
  Version 2 high resolution spectra are based on "Composite Scans" that are part of the published Volume 0 data set. The file format of composite scans has evolved over the years of network operation. For example, the visible part of the spectrum was initially scanned in 5 nm steps. The resolution of later Volumes is 1 nm. Version 2 spectra have a uniform wavelength grid (0.2 nm steps between 280 and 340 nm; 0.5 nm steps between 340 and 400 nm, and 1.0 nm steps above 400 nm).

  Version 0 composite scans had three columns: wavelength, spectral irradiance and ancillary measurements of a filtered photodiode (TSI). Version 2 data file consist of 12 columns, which also include model values, time, and solar zenith and azimuth angles. A full description of the data format can be found here.

  Examples of Version 2 high resolution spectra:
  

  • CB940600.351: Clear-sky spectrum
       ... measured at the South Pole on 12/17/94 (Julian Day 351) at 06:00 UT.
  • CB940800.349: Cloudy-sky spectrum
       ... measured at the South Pole on 12/15/94 (Julian Day 349 at 08:00 UT. The cloud optical depth was 2.8.

• Plots showing comparison of measurement and model
  All Version 2 high resolution spectra are accompanied by a PDF-file showing a comparison of the measured, cosine-corrected Version 2 spectrum with clear-sky and cloudy-sky model spectra. These plots are an easy way to interpret measurements, for example to analyze the effect of clouds, and to inspect the measured spectrum for errors such as spikes, abnormal wavelength shifts, and noise.

  Examples of Version 2 plots:
  

  • CB940600-351.pdf: Clear-sky spectrum
  • CB940600-351a.pdf: Clear-sky spectrum with annotations
  • CB940800-349.pdf: Cloudy-sky spectrum
  • CB940800-349a.pdf: Cloudy-sky spectr. with annotations

• Total column ozone
  Total ozone is calculated from global irradiance spectra based on a new method that we have recently published in Journal of Geophysical Research. A copy of the paper can be found here. The method takes the actual ozone profile at the time of the measurement into account and is therefore accurate for solar zenith angles up to 85°.

  Example file:
  SPO_ozone_coincident_TOMS_EP_V8.csv:
  Total ozone, South Pole, coincident with overpass of NASA/TOMS Earth Probe satellite
  Notes:
  1- SUV-100 ozone values are in column 4.
  2- "Left click" will open the example files in Microsoft Excel if that program is installed on your computer. To download the file, "right click" and choose "Save Target as ..." from the context menu.
  

• Cloud optical depth
  Cloud optical depth is estimated from the reduction of the measured spectra irradiance at 450 nm from the clear-sky model value. For this calculation, a thin, homogeneous ice-cloud located at 5 km a.s.l. with an effective particle radius of 20 µm is assumed.

  Example file:
  SPO_v5.2_COD.csv: Cloud optical depth, South Pole, Volume 5 (135 KBytes)
  Notes:
  1- Cloud optical depth is specified in column 7. Column 5 is "measured cloud transmittance (unbiased)", which is the ratio of measured irradiance at 450 nm to the respective modeled clear sky value. As this ratio may deviate from unity under clear-skies (i.e. due to inappropriately chosen model input parameters) a bias is applied, and cloud optical depth is estimated from the biased cloud transmittance (column 6).
  2- "Left click" will open the example files in Microsoft Excel if that program is installed on your computer. To download the file, "right click" and choose "Save Target as ..." from the context menu.
  

• Albedo
  Ground albedo is calculated for sites with a significant annual cycle in albedo (McMurdo, Palmer, Ushuaia, Barrow). Albedo is estimated from the characteristic wavelength-dependent enhancement of global spectral irradiance. No data is provided for South Pole as the variation in albedo over the year is much smaller than uncertainty of the retrieval algorithm.

• Flags
  Several quality indicators ("flags") are calculated for every spectrum, allowing to quantify distortions in measured spectra. The flagging procedure is similar to that implemented in the European UV Database. The following flags are implemented:

  ---

  Flag	Description
  Sky condition	"CS" for clear-sky spectra
  Ratio Meas/Mod	Ratio of measurement and model at 340 nm. This flag can be used to identify spectra that are abnormally low, indicating, for example, that calibration equipment was still in place during a solar scan.
  Start wavelength	Shortest wavelength from which onward a spectrum can be used with confidence.
  Irradiance at start wavelength	Spectral irradiance at the first "trustworthy" wavelength. This flag can be used to identify periods with abnormally high noise, for example caused by electronic problems.
  Max-min-ratio Ca-lines	Variation of measurement/model in the region of the Calcium lines (391-397 nm). This flag can be used to identify wavelength shifts.
  Spikes	Change in the ratio of measurement/model from one wavelength to the next. This flag can be used to identify electronic problems or sudden obstructions of the collector, for example, caused by birds. Spikes are given for four different wavelength bands.
  Wavelength shifts	Wavelength shifts are calculated for every spectrum in the critical wavelength range of 290 - 340 nm by means of a algorithm that compares the Fraunhofer structure in measured spectra with the structure in a reference spectrum.
  Manual flag	Flags entered manually after reviewing the data.

  ---

  Example file:
  SPO_v5.2_flags.csv: Flags, South Pole, Volume 5 (872 KBytes)
  Note:
  "Left click" will open the example files in Microsoft Excel if that program is installed on your computer. To download the file, "right click" and choose "Save Target as ..." from the context menu.
  

• Model parameters and model log
  All parameters used for radiative transfer modeling and for the correction algorithms are logged in this file. The meaning of some of the parameters is very complex and will be described elsewhere. The right-most columns of the file include the ratio of measurement to the clear-sky model for 10-nm wide wavelength bands centered at 30 different wavelengths between 300 nm and 590 nm. These ratios offer an easy way for plotting the ratio of measurement and model as a function of time.

  Example file:
  SPO_v5.2_model_log.csv: Model log, South Pole, Volume 5 (4.14 MBytes)
  Notes:
  1- Columns with ratios of measurement and model are labeled "R300", "R310" ... "R590". For example, the column "R310" includes the ratio averaged over the spectral range 305 - 315 nm.
  2- "Left click" will open the example files in Microsoft Excel if that program is installed on your computer. To download the file, "right click" and choose "Save Target as ..." from the context menu.
  

• Wavelength accuracy
  The conversion from Version 0 to Version 2 includes a wavelength correction. The correction files are not calculated for every spectrum, instead, one file may be applied for periods ranging from one day to the entire season. The wavelength accuracy after applying the correction is tested for every spectrum and the residual shifts are documented by this data product.

  Example file:
  SPO_v5.2_wave_shift2.csv: Wavelength shift, South Pole, Volume 5 (1.22 MBytes)
  Notes:
  1- The left part of the file gives the residual wavelength shifts for wavelengths typically between 292.5 and 342.5 nm. The right part of the file gives the error-ratios of the shifts, which are a measure of the quality of the shift value returned by the correlation algorithm.
  2- "Left click" will open the example files in Microsoft Excel if that program is installed on your computer. To download the file, "right click" and choose "Save Target as ..." from the context menu.
  

• Results of Quality Control
  The conversion from Version 0 to Version 2 is done by published volume. The results are thoroughly quality controlled before processing the next volume. Quality control includes the review of all data products listed above, a comparison of Version 0 and Version 2 data, and a comparison of Version 2 data of all volumes (for example to detect possible level changes). Suspicious scans are flagged and some volumes may be reprocessed with slightly changed correction parameters. Files created during quality control are not standardized but are available on request.

Where is "Version 1"?
Version 1 does not exist. As the original (=first) version was called "Version 0" rather than "Version 1", we decided to call the new version "Version 2" to avoid confusion.

For more information contact us.