Data Format and Transfer Specifications using Community Conventions

APE participants need only provide the subset of APE diagnostics from which all others can be derived. This minimum set reduces the required diagnostics to the GT, SH, TR, ML, MF and PF tables (see APE diagnostic specifications). The diagnostics listed in GA, SZ, MZ, TE and VB will be derived later from the minimum set.

We are exploiting the efforts of many who have worked to develop data standards for the climate research community. While a little investment is required to become familiar with the conventions, it is important to realize the benefit: this community activity is working towards a common means of data exchange that will be applicable for many other WCRP projects (e.g., AMIP and CMIP).

The data standards described below presume that APE models have recti-linear longitude-latitude grids or are interpolated to such a grid. Gaussian grids are of course acceptable, unless the number of longitude points varies with latitude. To ensure that APE data can be efficiently managed and results rapidly analyzed, only data that meets the following specifications can be accepted:

Data must be provided in netCDF and compliant with minimum specifications defined by the CF convention

Note: Participants need not follow all CF specifications, only those listed below.

CF Convention:

Dimension Information

Dimension names:

time for the time dimension

plev (data on pressure levels) or lev (data on model surfaces - table PF)

longitude or lon for the longitude dimension

latitude or lat for the latitude dimension

Dimension units:

No time dimension for time-averaged fields

time: "days since 0000-1-1-0" for daily (mean or instantaneous) data

plev: "Pa"

longitude: "degrees_east"

latitude: "degrees_north"

Note: The year (0000), month (1) and day (1) have been arbitrarily chosen for the APE configuration. Integrations should begin 0000-01-01, meaning if you follow the recommendation for a 6 month spin-up the data submitted will begin 0000-07-01. For the 6hourly data (table TR), fractional days should be used (e.g., 0000-07-1.25) and the reference time should include 0-hours (e.g., "days since 0000-01-01-00") to anchor the diurnal cycle.

Dimension attributes:

time: calendar - "NoLeapCalendar" or "Calendar360"

latitude: latitude_bounds (see example netCDF output below)

Data on model surfaces (table PF only):

CF accommodates transform coefficients (the A's and B's)

Relevant references in CF documentation:

Dimensionless vertical coordinate (Section 4.3.2)

Dimensionless coordinate types (Appendix D)

Sample file (ncdump output) given below

Grid ordering and orientation: ( time[0, N] ; level[near-surface level, top level] ; latitude[Southernmost latitude,Northernmost latitude] ; longitude[0,eastern most point] ).
For lesser ranking fields, preserve the ordering, e.g., SZ fields (single level zonal means): (time, latitude).

Variable attributes

Each variable (identified by the attribute name) should have the associated attributes ape_name and units.

These required attributes, separated by a "\", are provided in the attached files (below). The contents of each file (e.g., gt-cf-info.txt) include: 1) the APE diagnostic (e.g., "GT01") ; 2) the APE variable name (e.g., "gt_sw_toai") ; 3) the ape_name (e.g., "toa_incoming_shortwave_flux") ; 4) units (e.g., "W m-2"):

gt-cf-info.txt

mf-cf-info.txt

ml-cf-info.txt

sh-cf-info.txt

tr-cf-info.txt

pf-cf-info.txt

Only the variables listed in these files need to be provided. The remainder of the diagnostics in the APE diagnostic specifications will be derived from this minimum set. The majority of the ape_names are consistent with the CF standard names.

Sample CF compliant netCDF output (using the ncdump command):

ncdump_ape_gt.txt

ncdump_ape_mf.txt

ncdump_ape_ml.txt

ncdump_ape_sh.txt

ncdump_ape_tr.txt

ncdump_ape_pf.txt

Global attributes:

Conventions: "CF-1.0"

Source: Model Version as defined by modeler

File Structure

Data files should be organized according to Analysis/Data (e.g., GT, SH, TR; see APE diagnostic specifications) and experiment. File names should include information about the modelling group, the data type, and the experiment with the following template for file names: modelvintage_data_experiment.nc, where the "nc" extension implies a netCDF file. All the variables of the same data type (e.g., GT, SH) should be stored in the same file, giving 6 files per experiment, giving 6x8 = 48 files per model. Example file names:

NCAR-03a_GT_control.nc (Global timeseries)

NCAR-03a_MF_3keq.nc (Multiple-level zonal mean fluxes)

NCAR-03a_ML_flat.nc (Multiple-level 3-D means)

NCAR-03a_SH_qobs.nc (Single-level 2-D means)

NCAR-03a_TR_control5n.nc (Transients)

NCAR-03a_PF_peaked.nc (Parameterization forcing)

The 8 experiments are identified by: control, peaked, qobs, flat, control5n, 1keq, 3keq, 3kw1

The 6 data types saved for each experiment are: MF, ML, GT, SH, TR, PF

It is recommended that the AMIP convention for "model id" be used to identify the model. This simply designates the group running the model (e.g., "ncar") and the year it was run (e.g., "03a"). The letter ("a") appended to the year is a place holder in case more than one run is submitted in a given year. Note, PCMDI more completely refers to a "model designation" that is unique for each "model vintage". See for example: http://www-pcmdi.llnl.gov/projects/amip/STATUS/incoming.html.

Before processing all of your experiments, we recommend that you prepare your control run according to these standards, and submit it for verification.

Last modified: 13 February 2017