Description of software

pp2nc is a fortran/C program designed to allow the user to extract selected fields from a PP-type file and to output these required fields into a series of (smaller) netCDF files. For example, scientists in the HRAA group at the University of Reading use the software to reduce the amount of data to be transferred to the university (we typically do not need all the content of all PP files), thereby drastically cutting down on sftp tranfer times, and also reducing the amount of remote storage needed. Since netCDF is more widely used as a file data format, it is convenient to use such files as input into our analysis and diagnostic software.

The software runs currently on the IBM (and also on Sun workstations at the university). A simple script has been developed to allow users to run the software, which is a three-stage process.

  1. Perform a preliminary run of the software on a single PP file, creating a text file fort.55 that defines the fields that are present in the PP file.
  2. Edit this fort.55 file to leave only those fields of interest.
  3. Run the software again, this time on all the input PP files to create the netCDF output.

On the first run through of the software a text file (fort.55) is created which contains one record per field/variable found in the first of a sequence of PP data files. Here it is assumed that this first file is one of a series of similar files, created by an ensemble run of the UM through a series of cycles (in out case). At this stage no netCDF files are created.

The user then needs to edit this fort.55 file to leave only the required field names remaining in the file. (If the user is confident of the PP file content and has run the software previuously then a previous fort.55 file can be used - in this case the software will skip steps 1 and 2.

Running the software again once the fort.55 file is present (albeit modified by the user) will cause the software to run through all the input files and create the netCDF output files.

Using the software

A script /home/nwp/da/frbr/pp2nc/convert_job (shown here) on the IBM can be used to run the software.

Note that a separate script /home/nwp/da/frbr/pp2nc/MOPS_convert_job on the IBM can be used to convert files with an extension pp_001.MOPS; such files need preprocessing with ieee -32e.

# Script to run Roger's pp2nc software.
#
# Instructions to install the program
#
# 1. Take a copy of this file and place it in the directory in which
#    you wish to run the program.
#
# Instructions for use
#
# 1. Modify the line cyclelist=... to include the experiment cycles of interest
# 2. Modify the line DIR=... to point to your directory containing the files
#    to be converted.
# 3. Insert arguments to the ls command in the line filelist=...
#    such that the required input files will be read.
#
# 4. The first time you run this program on your data set RUN=1 and then
#    run the script.
# 5. When the program stops edit the fort.55 file to include
#    only the stash codes corresponding to the required fields.
# 6. Then set RUN=2 and rerun the script.
#
# 7. Results will be placed in the directory from where the script was launched.
 
# Note - by default all the levels of the chosen fields will be selected (the
#        line -01 in the setup of file fort.1 created by this job. In the event that you
#        wish to select only a single level from a multi-level field simply change
#        -01 into a three digit integer (eg. 023) to select that level number
#        (in this case level 23).

startdir=`pwd`
RUN=2                                                                                       #USER CHANGE
cyclelist="107 108 109 110 111 112"                                                         #USER CHANGE
memberlist="00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23"
for cycle in $cyclelist
do
for member in $memberlist
do

DIR=/home/nwp/da/apis/data/ETKF/ETKFfor1p5km_SM03_noprecip/oper/oper${member}               #USER CHANGE
cd ${DIR}
filelist="`ls qwq${cycle}.oper${member}.pp_001`"                                            #USER CHANGE

cd $startdir
if [ $RUN -eq 1 ]
then
  /bin/rm -r fort.55
fi

for file in $filelist
do
  echo $file
  /bin/rm -r fort.1

cat << /EOF > fort.1
-01
${DIR}/
$file
${startdir}/
/EOF

cat fort.1
  /home/nwp/da/frbr/pp2nc/pp2nc_convert

  if [ $RUN -eq 1 ]
  then
    exit
    # MUST exit the first time until fort.55 is modified
  fi

done
done
done