How to take output from ensemble runs and generate covariance statistics

This document is very long as it is meant to be comprehensive. Many of the steps will not necessarily be needed as required files may already be available or a particular set of diagnostics may not be needed. Some of the steps are run on Monsoon, some on the Met Office supercomputer and others on a Met Office linux box.

The outline of the prodedure is as follows:

An ensemble is assumed to be available. The procedure described in this document helps to answer questions like, "What are the covariances according to this ensemble?", "What does the ensemble say about balances satisfied by the ensemble perturbations?".
The ensemble members are read into some code run on the supercomputer. This code does extracts some information from the ensemble members and outputs some files (sometimes these are fieldsfiles, sometimes text files). The code is built within the VAR system (VarProg _TestPFmodel). No DA (or testing of PF model) is done - this is done because it is a convenient way of accessing the code to read-in ensemble members, and to have access to the library of routines used by the VAR system (e.g. FFT routines). The VAR system thinks that the ensemble members are LS states at different times, so they are stored in the LS structure.
The output files are then transferred to a linux box for visualization via a suite of tidl routines. The tidl routines output an html file and some jpegs, so lots of output can be viewed via a 'web page'.

This document is taylored to the kinds of data structures used by the DIAMET people, although it can be adapted for more general use. The document is structured as follows:

Initial preparation (sections 1, 2).
Prepare a suitable orography file (section 3). [In order to be read into VAR, each ensemble member must have u, v, theta, exner, q and orog. The ensemble members from MOGREPS do not have orog so one must be prepared.]
Merge-in the orography field into each member (section 4).
Put the processed ensemble onto the Met Office supercomputer if it has been processed on Monsoon (section 5).
Run an approprate piece of VAR code on the supercomputer, transfer the output to a linux box and run some tidl code (section 6).

1. Create a directory to store your processed ensemble (MetO supercomputer or Monsoon)

The raw UM output files will need to be processed before computing the covariance statistics. Make a directory on hpc2f (MetO ) or ibm02 (Monsoon):

outputdir

2. Find the validity time of interest (MetO supercomputer or Monsoon)

Have a look in a sample ensemble member and find the date and time of interest:

<year>, <month>, <day>, <hour>, <minute>

3. Make an orography file with this date/time (MetO supercomputer or Monsoon)

The code that reads-in the ensembles is expecting an orography field at the correct time (even though orography obviously doesn't change in time, the code demands this time is correct!)

A sample orography file for the Southern UK domain (1.5 km resolution) is contained here on monsoon

/projects/enda/rnbann/SampleFiles/orog0.pp

or on the Met Office supercomputer

/data/nwp/fr/frxb/Ensembles

Orography files at a selection of other times are also available in these directories (for the NDP model domain). If one has already been prepared for the time that you require, you can skip the rest of section 3.

Stage 3.1 - copy the following into a script, change the date and time and run (on Monsoon and MetO supercomputer).

On Monsoon

cat > namelist << EOF
&Var_AlterHeaderNL
FixHdIndices(1:5) = 21,22,23,24,25
FixHdValues(1:5) = <year>,<month>,<day>,<hour>,<minute>
/
EOF
/home/rnbann/Var_UMFileUtils/Vn28.1/VarScr_UMFileUtils -u AlterHeader -v -n namelist <OrogInputFile> <OrogOut1>

where values inside <> are user set. The <OrogInputFile> is, e.g. the one specified above. The procedure is the same on the MetO supercomputer, except that the last line should be replaced by

 /home/nwp/da/frva/var0/build/Var_Stable28/v1.0/pwr7-xlf-serial/build/bin/Var_UMFileUtils/Vn28.1/VarScr_UMFileUtils -u AlterHeader -v -n namelist <OrogInputFile> <OrogOut1>

Stage 3.2 - copy the following into a script, change the date and time and run (on Monsoon and MetO supercomputer).

On Monsoon

cat > namelist << EOF
&Var_AlterHeaderNL
LookupIndices(1:2,1) = 1,-1,
LookupIndices(1:2,2) = 2,-1,
LookupIndices(1:2,3) = 3,-1,
LookupIndices(1:2,4) = 4,-1,
LookupIndices(1:2,5) = 5,-1,
LookupValues(1) = <year>,
LookupValues(2) = <month>,
LookupValues(3) = <day>,
LookupValues(4) = <hour>,
LookupValues(5) = <minute>
/
EOF
/home/rnbann/Var_UMFileUtils/Vn28.1/VarScr_UMFileUtils -u AlterHeader -v -n namelist <OrogOut1> <OrogOut2>

The <OrogOut1> is the one specified above in 3.1. The procedure is the same on the MetO supercomputer, except that the last line should be replaced by

/home/nwp/da/frva/var0/build/Var_Stable28/v1.0/pwr7-xlf-serial/build/bin/Var_UMFileUtils/Vn28.1/VarScr_UMFileUtils -u AlterHeader -v -n namelist <OrogOut1> <OrogOut2>

4a. For experiments with 24 members stored on Monsoon: Edit and run the script that extracts the fields from the appropriate directories and merge-in the orography file prepared before (Monsoon)

This procedure assumes that fields are stored according to the description on this page. Files from the 23+1 ensemble members have been calculated in three batches of eight. For each ensemble member some relevant fields are in one file and other relevant fields are in another. All the appropriate fields will be extracted and combined into a single file (for each ensemble member).

Copy the two scripts /home/rnbann/Scripts/CreateNL.script and /home/rnbann/Scripts/ExtractAndCombine.script to your own user space (on ibm00). Make sure that they are executable ( chmod +x <each_script_name>). Only the latter script will need to be edited. The following items will need to be set:

RUNID1, RUNID2, RUNID3 These are the names of the UM jobs that produced each batch.
BASEDIR This is the directory containing the three directories (containing UM output) with names RUNID1, RUNID2 and RUNID3.
SPECIAL This is the part of the ensemble member filenames that is special to this particular experiment (e.g. noRP).
OUTPUTDIR This is the directory that the processed ensemble members will be sent to.
DAY, HOUR The day number of hour of interest.
OROG The path of the orography file <OrogOut2> in 3.2.
TEMPDIR A temporary directory used to store temporary files.

Run the script ./ExtractAndCombine.script and wait for the job to finish. The script outputs a file called Status which details the input and output files that the script has dealt with.

4b. For members with 92 members stored on mass: Extract the fields from mass and merges in the orography file from above and merge-in the orography file prepared before (Met Office supercomputer)

Run the following commands (from the diectory /scratch/username)

moo get moose:/devfc/sitdf/adhoc.file/2011092018.pp1.full.tar .
tar xvf 2011092018.pp1.full.tar
rm 2011092018.pp1.full.tar

Next run the following script which will add orography to each member

export UM_UTILS_DIR=/home/nwp/da/frva/var0/build/Var_Stable28/v1.0/pwr7-xlf-serial/build/bin
MEMBERS="00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 
30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 
63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92"
for MEMBER in $MEMBERS
do
  echo "Dealing with " $MEMBER
  $UM_UTILS_DIR/VarScr_UMFileUtils -u MergeData -v oper${MEMBER}/qwq118.oper${MEMBER}.pp1.full $DATADIR/Ensembles/orog201109201900Z qwq118.oper${MEMBER}.pp1
done

Finally delete the directory structure output from the tar command.

5. Copy the ensemble files to the Met Office (Monsoon only)

This comprises a number of stages:

Get the data to the exeter linux box - either els037 or els042. Instructions are available here.
sftp the data to an appropriate place on the Met Office hpc.

6. Process the ensembles on the supercomputer to produce required statistics (Met Office supercomputer)

The procedure now depends upon what you want. Options are as follows:

Plot basic fields from ensemble data (e.g. derived fields like vorticity and divergence) (option 0).
Compute 3-D variance fields (cross sections), structure and correlation functions with a chosen reference point (option 1).
Compute vertical correlation functions (as a function of horizontal position) with a chosen reference level (option 2).
Compute balance diagnostics (with and without localization) (option 3).
Compute variances and vertical correlations in spectral space (option 4).
As option 2, but averaged over longitude (option 5).
Total pressure, balanced pressure, residual pressure diagnostics (option 6).

The job is launched via a VAR job (i.e. run using varui), even though no data assmilation is done. The code is hacked into the Test PF Model program. The reason for this is to allow Ross's covariance code to make use of the read/write/Fourier transform/etc routines that are present in the VAR system. A complilation job for the code is vexi.f. In all jobs, the variable $VAR_LSDUMP_DIR specifies the directory where the ensemble files have been placed. The code does not care what the files are called, but this directory must not contain any other files.

6.1 Compute 3-D variance fields (cross sections), structure and correlation functions with a chosen reference point (option 1)

Set-up and run the 'VAR' code to compute the diagnostics

Run Ross's VAR code. An example job is vexi.b (as it stands this VAR job may have been set up for another option, but the only difference is in the namelist settings described below). The compilation has already been done (at least on the IBM power 7), but if needed, the compilation job is vexi.b. Note that this stage does not plot the data - it just generates data. Instructions for plotting follows. Also note that this is a job that can run on only 1PE.

Namelist variable Type Default Used with Master Routine
Description

General variables
HRAA_MasterRoutine Integer 1 n/a
0 = Compute basic or derived fields from a specified ensemble member.
1 = Compute 3-D variance, structure and correlation functions.
2 = Compute vertical correlation functions (as a function of horizontal position) with a chosen reference level.
3 = Compute balance diagnostics (with and without localization).
4 = Compute variances and vertical correlations in spectral space.
5 = As option 2, but averaged over longitude.
6 = Pressure (balanced / unbalanced) diagnostics
7 = Compute maps of balance diagnostics
8 = Theta (balanced / unbalanced) diagnostics

UseHowManyMembers Integer 0
1, 2, 3, 4, 5, 6
Set the number of ensemble members in the covariance calculations. Set to 0 to use all ensemble members in the VAR_LSDUMP_DIR directory.
UseMember Integer 1 0 To specify which ensemble member is converted for later plotted. Note that the control member is 1.
PertFromMean Logical .FALSE.
3 at moment
Set to calculate perturbations from the mean. Unset to calculate perturbations from the control.

NMC Logical .FALSE.
6 and 8 at moment
Set to use NMC forecast differences instead of ensembles.

NMC_DIRS Character
6 and 8 at moment
Array of directories containing NMC data

Localization-related variables
Localization_type Integer 0 1, 2, 3

0 = No localization (raw covariances).

1 = Static localization.

2 = Super-static localization.

3 = Simplified SENCORP localization.

4 = ECO-RAP localization.

5 = Super-ECO-RAP localization.

HorizLenScale
Real 100.0 1, 3 Horizontal smoothing distance (grid length multiples) used to generate smoothed members in (super)ECO-RAP and SENCORP localizations. For (super)ECO-RAP localization this is relevant only when Nk>0 and HorizSpectype > 1.
VertLenScale Real 10.0 1, 3 Vertical smoothing distance (grid length multiples) used to generate smoothed members in (super)ECORAP and SENCORP localizations. For (super)ECO-RAP localization this is relevant only when Nkz>0 and VertSpectype > 1.
Nk Integer
0 1, 3
Number of horizontal wavenumbers. Set to -1 for no horizontal localization.
Nkz Integer 0
1, 3
Number of vertical wavenumbers. Set to -1 for no vertical localization.
HorizSpecType Integer 1 1, 3
Horizontal spectrum shape. 1 = square, 2 = function described by length1_horiz and length2_horiz
VertSpecType Integer 1
1, 3
Vertical spectrum shape. 1 = square, 2 = function described by length1_vert and length2_vert
length1_horiz Real 15.0 1, 3
Quasi flat distance in horizontal for moderation fn
length2_horiz Real 50.0 1, 3
Drop-off distance in horizontal for moderation fn
length1_vert Real 5.0 1, 3
Quasi flat distance in vertical for moderation fn
length2_vert Real 15.0 1, 3
Drop-off distance in vertical form moderation fn
SENCORPorder
Integer 1 1, 3 Order of simplified SENCORP method (m in Bishop paper, 0, 1 or 2).
ECORAPorder Integer 1 1, 3 Order of (super)ECO-RAP adaptive 'sub' matrix (q in Bishop paper, 1 or 2).
OnlyLocalization Logical .FALSE. 1
Set to .TRUE. to output only the localization function in the output fields (static and ECO-RAP localization methods only).
CQ_type Integer 1 1, 3

1 = Calculate each element of CQ explicitly (uses CQ_radius_horiz and CQ_radius_vert)
2 = Make ensemble of square-root members (uses CQ_radius_horiz) with large number of members (used for faster balance diagnostics)
3 = As 2, but uses smaller number of members (recommended for structure function calculations).
CQ_radius_horiz Integer 100 1, 3
Number of horizontal points (in x or y directions) to include in action of CQ matrix (ECORAP).
CQ_radius_vert Integer 50
1, 3
Number of vertical points to include in action of CQ matrix (ECORAP).

JacobiPairs Integer 0
1, 3
No. of rotations in localization schemes

JacobiSeed Integer 211541
1, 3
Random No. seed for Jacobi rotations

Parameter conversion switches
Theta2T Logical .TRUE. 0, 1, 2, 4, 5
Convert theta to temperature (still store in LS % theta in code)

hydro_p Logical .FALSE.
0, 1, 2, 3, 4, 5, 6
Replace exner with hydrostatic exner
Exner2p Logical .TRUE. 0, 1, 2, 4, 5
Convert exner pressure to pressure (still store in LS % exner in code)

uv2psichi Logical .FALSE. 0, 1, 2, 4, 5
Convert u and v fields to psi and chi (store in new variables LS % psi and LS % chi in code)

uv2vortdiv Logical .FALSE. 0, 1, 2, 4, 5
Convert u and v fields to vorticity and divergence (store in new variables LS % psi and LS % chi in code)

Flags to switch on/off covariance calculations

Calc_u_stats Logical .FALSE. 1, 2, 4, 5 Set to .TRUE. to calculate structure/correlation functions associated with u (or psi, vorticity).

Calc_v_stats Logical .FALSE. 1, 2, 4, 5 Set to .TRUE. to calculate structure/correlation functions associated with v (or chi, divergence).

Calc_theta_stats Logical .FALSE. 1, 2, 4, 5 Set to .TRUE. to calculate structure/correlation function associated with theta.

Calc_exner_stats Logical .FALSE. 1, 2, 4, 5 Set to .TRUE. to calculate structure/correlation function associated with exner.

Calc_q_stats Logical .FALSE. 1, 2, 4, 5 Set to .TRUE. to calculate structure/correlation function associated with q.

Calc_w_stats Logical .FALSE. 1, 2, 4, 5 Set to .TRUE. to calculate structure/correlation function associated with w.

Calc_ws_stats Logical .FALSE. 1, 2, 4, 5 Set to .TRUE. to calculate structure/correlation function associated with wind speed.
Details about the (fixed) source point (to compute covariances with)
Source_x Integer 180 1, 2, 3 Source point index in the x-direction.

Source_y Integer 144 1 Source point index in the y-direction

Source_z Integer 36 1, 2, 4, 5 Source point index in the z-direction.

CalcInPlanesOnly Logical .TRUE. 1, 2,3
Set to .TRUE. to calculate structure/correlation functions in the Source_x, Source_y, Source_z planes only. Set to .FALSE. to calculate structure/correlation functions at all positions (latter very expensive).

CalcInLinesOnly Logical .FALSE. 1, 2,3
Set to .TRUE. to calculate structure/correlation functions in the Source_x, Source_y, Source_z lines only.

Balance calculation variables
Bal_flag_alpha Logical .FALSE. 3 Set to include 'second order' terms in horizontal balance equation.
Bal_glag_beta Logical .FALSE. 3 Similar to above (can't remember difference)
y_from Integer 1
3 First latitude
y_to Integer
288
3 Last latitude
SmoothingScale_from Real 0.0
3
Starting scale for sweep of lengthscales to scale ensemble members before calculating balance diagnostics (units of grid lengths).
SmoothingScale_to Real 50.0
3
Ending scale for sweep of lengthscales to scale ensemble members before calculating balance diagnostics (units of grid lengths).
SmoothingScale_inter Real 0.0
3
Interval of sweep of lengths to scale ensemble members before calculating balance diagnostics (units of grid lengths).
Bal_CalcMatrix Logical .FALSE.
3 Set to calculate vertical covariance matrices for theta, thetaH, exner, exnerH

Bal_CalcTerms Logical .FALSE.
3 Set to calculate terms in balance equations

Pressure (balanced / unbalanced) diagnostics variables
Pressure_realsp Logical .TRUE.
6 Calculate pressure diagnostics in real-space

Pressure_specsp
Logical .TRUE. 6
Calculate pressure diagnostics in spectral-space

Pressure_domain_ens Logical .TRUE. 6
Calculate pressure diagnostics averaged over domain and ensemble

RemoveMeans Logical .TRUE. 6
Remove means from covariance calculations

degrade_lengthscale Real 0.0 6
Degrade lengthscale by this amount (km) - convolutes with a Lorentzian

Theta_realsp Logical .TRUE.
8 Calculate theta diagnostics in real-space

Theta_specsp
Logical .TRUE. 8
Calculate theta diagnostics in spectral-space

Vertical_regression Integer 0 6 Options to perform vertical regression on the balanced pressure
0 = No VR
1 = Read VR matrix from file
2 = Calculate VR matrix from data

Vertical_regression_file Character StandardRegression_70 6 Filename of VR matrix (in vertical modes directory)

Vertical_regression_write Logical .FALSE. 6 Set to write VR matrix to file (in vertical modes directory)

Vertical_regression_ip Integer 0 6 Inner product to use
0 = Identity
1 = As psi
2 = As chi
3 = As Ap
4 = As mu

GlobalMeanBal
Logical
.FALSE. 6
Consider global mean pressure as balanced

FFTbp Logical .FALSE. 6 If set the Fourier method is used to calculate balanced pressure, otherwise GCR is used.

MaxGCRIters
Integer
10 6 Maximum number of iterations for the Laplace equation solver

GCR_Restart_value
Integer
8 6 For GCR(k) method

GCR_precon_option
Integer
0 6 GCR preconditioning option (only 0 implemented successfully - no preconditioning)

deltat
Real
1.0 6 For ADI preconditioner (not implemented)

read_bp
Logical
.FALSE. 6 Set to read-in the balanced pressures instead of using solver

Z
Integer
10 6 Z

JustOutputBalp
Logical
.FALSE. 6
Set to output only the balanced pressure (zero bcs)

The files that are output from this program are the following.

Output files for HRAA_MasterRoutine = 0 (derived fields from a specified ensemble member)

DerivedFields .ff

Output files for HRAA_MasterRoutine = 1 (3-D variance, structure and correlation functions)

Variances of u, v (or psi, chi), theta (or T), exner (or p), q, w and wind speed are in HRAAVars_x.ff.
Structure functions associated with u (or psi, vorticity) are in StrFns_x_with_u.ff.
Structure functions associated with v (or chi, divergence) are in StrFns_x_with_v.ff.
Structure functions associated with theta (or T) are in StrFns_x_with_theta.ff.
Structure functions associated with exner (or p) are in StrFns_x_with_exner.ff.
Structure functions associated with q are in StrFns_x_with_q.ff.
Structure functions associated with w are in StrFns_x_with_w.ff.
Structure functions associated with wind speed are in StrFns_x_with_windspeed.ff.
Correlation functions associated with u (or psi) are in CorFns _x_with_u.ff.
Correlation functions associated with v (or chi) are in CorFns_x_with_v.ff.
Correlation functions associated with theta (or T) are in CorFns_x_with_theta.ff.
Correlation functions associated with exner (or p) are in CorFns_x_with_exner.ff.
Correlation functions associated with q are in CorFns_x_with_q.ff.
Correlation functions associated with w are in CorFns_x_with_w.ff.
Correlation functions associated with wind speed are in CorFns_x_with_windspeed.ff.

Output files for HRAA_MasterRoutine = 2 (vertical correlation functions (as a function of horizontal position) with a chosen reference level)

Vertical correlation functions of u (or psi, vorticity), v (or chi, divergence), theta (or T), exner (or p), q, and w (as specfied in the namelist) associated with u (or psi, vorticity) are in CorFns_Vertx_with_u.dat.
Vertical correlation functions of u (or psi, vorticity), v (or chi, divergence), theta (or T), exner (or p), q, and w (as specfied in the namelist) associated with v (or chi, divergence) are in CorFns_Vertx_with_v.dat.
Vertical correlation functions of u (or psi, vorticity), v (or chi, divergence), theta (or T), exner (or p), q, and w (as specfied in the namelist) associated with theta (or T) are in CorFns_Vertx_with_theta.dat.
Vertical correlation functions of u (or psi, vorticity), v (or chi, divergence), theta (or T), exner (or p), q, and w (as specfied in the namelist) associated with exner (or p) are in CorFns_Vertx_with_exner.dat.
Vertical correlation functions of u (or psi, vorticity), v (or chi, divergence), theta (or T), exner (or p), q, and w (as specfied in the namelist) associated with q are in CorFns_Vertx_with_q.dat.
Vertical correlation functions of u (or psi, vorticity), v (or chi, divergence), theta (or T), exner (or p), q, and w (as specfied in the namelist) associated with w are in CorFns_Vertx_with_w.dat.

Output files for HRAA_MasterRoutine = 3 (balance diagnostics)

The actual terms of the geostrophic, hydrostatic and anelastic equations are in BalanceData.dat.
The vertical covariance matrices for theta-theta, thetaH-thetaH, exner-exner and exnerH-exnerH are are in MatrixData.dat (if selected).

Output files for HRAA_MasterRoutine = 4 (variances and vertical correlations in spectral space)

Vertical correlation functions and variances (in spectral space) associated with u (or psi) are in VertSpecData_u.dat.
Vertical correlation functions and variances (in spectral space) associated with v (or chi) are in VertSpecData_v.dat .
Vertical correlation functions and variances (in spectral space) associated with theta (or T) are in VertSpecData_theta.dat .
Vertical correlation functions and variances (in spectral space) associated with exner (or p) are in VertSpecData_exner.dat .
Vertical correlation functions and variances (in spectral space) associated with q are in VertSpecData_q.dat .
Vertical correlation functions and variances (in spectral space) associated with w are in VertSpecData_w.dat .

Output files for HRAA_MasterRoutine = 5 (as option 2, but averaged over longitude)

Output files as MasterRoutine = 2.

Note that the specification of the output fields (choice of levels, etc) can be made inside the Field Output namelist.

Output files for HRAA_MasterRoutine = 6 (pressure (balanced / unbalanced) diagnostics)

Real-space field of total pressure variances are in RealVars_p.ff.
Real-space field of balanced pressure variances are in RealVars_pb.ff.
Real-space field of unbalanced (residual) pressure variances are in RealVars_pr.ff.
Spectral-space fields of above quantities are in PressureSpec.dat.

Note that the specification of the output fields (choice of levels, etc) for the real-space fields can be made inside the Field Output namelist.

6.2 Convert the files into a format readable by TIDL (on supercomputer and then on linux)

For each .ff output file on the supercomputer (if any for the particular run that you have done) run the following (which I think tranforms the files from 64 to 32-bit):

ieee -32 x.ff x32.ff

Transfer each x32.ff file to a linux box and on linux run the following (which transforms the fields files to post processing files):

~frum/vn6.0/linux/utils/convpp -32 x32.ff x32.pp

To save typing, these are the commands that you will use for various options (copy and paste into terminal).

# ===== OPTION 0 =================================
# Commands to be performed on the ibm for option 0
ieee -32 DerivedFields.ff DerivedFields32.ff
# cd to ensemble directory
ieee -32 Memberx.pp Memberx32.ff
# gzip converted files
gzip *32.*
# Transfer to linux box
rsync *32.*.gz rld036:/data/local/frxb/...
# Delete Memberx32.ff on ibm
rm *32.*.gz 
# Commands to be performed in the linux box for option 0
gzip -d *.gz
~frum/vn7.6/linux/utils/convpp -32 DerivedFields32.ff DerivedFields32.pp
~frum/vn7.6/linux/utils/convpp -32 Memberx32.ff Memberx32.pp


# ===== OPTION 1 =================================
# Commands to be performed on the ibm for option 1
# For variances and correlation functions (usually 1a)
# export EXP=Localization/InputData/NoRP_20Sep2011_1500Z/10Ap

ieee -32 Vars_x.ff Vars_x32.ff
ieee -32 CorFns_x_with_q.ff CorFns_x_with_q32.ff
ieee -32 CorFns_x_with_theta.ff CorFns_x_with_theta32.ff
ieee -32 CorFns_x_with_exner.ff CorFns_x_with_exner32.ff
ieee -32 CorFns_x_with_u.ff CorFns_x_with_u32.ff
ieee -32 CorFns_x_with_v.ff CorFns_x_with_v32.ff
ieee -32 CorFns_x_with_w.ff CorFns_x_with_w32.ff
ieee -32 CorFns_x_with_windspeed.ff CorFns_x_with_windspeed32.ff
gzip *32.ff
# Transfer to linux box
echo $EXP
rsync *.gz rld036:/data/local/frxb/$EXP
rm *.gz
 
# For structure functions (usually 1b)
ieee -32 StrFns_x_with_q.ff StrFns_x_with_q32.ff
ieee -32 StrFns_x_with_theta.ff StrFns_x_with_theta32.ff
ieee -32 StrFns_x_with_exner.ff StrFns_x_with_exner32.ff
ieee -32 StrFns_x_with_u.ff StrFns_x_with_u32.ff
ieee -32 StrFns_x_with_v.ff StrFns_x_with_v32.ff
ieee -32 StrFns_x_with_w.ff StrFns_x_with_w32.ff
ieee -32 StrFns_x_with_windspeed.ff StrFns_x_with_windspeed32.ff
gzip *.ff 
# Transfer to linux box
echo $EXP
rsync *.gz rld036:/data/local/frxb/$EXP
# Delete stuff on ibm
rm *32.gz

# Commands to be performed on the linux box for option 1
# Unzip the files first 
gzip -d *.gz 
~frum/vn7.6/linux/utils/convpp -32 Vars_x32.ff Vars_x32.pp 
~frum/vn7.6/linux/utils/convpp -32 CorFns_x_with_q32.ff CorFns_x_with_q32.pp 
~frum/vn7.6/linux/utils/convpp -32 CorFns_x_with_theta32.ff CorFns_x_with_theta32.pp 
~frum/vn7.6/linux/utils/convpp -32 CorFns_x_with_exner32.ff CorFns_x_with_exner32.pp 
~frum/vn7.6/linux/utils/convpp -32 CorFns_x_with_u32.ff CorFns_x_with_u32.pp 
~frum/vn7.6/linux/utils/convpp -32 CorFns_x_with_v32.ff CorFns_x_with_v32.pp 
~frum/vn7.6/linux/utils/convpp -32 CorFns_x_with_w32.ff CorFns_x_with_w32.pp
~frum/vn7.6/linux/utils/convpp -32 CorFns_x_with_windspeed32.ff CorFns_x_with_windspeed32.pp
~frum/vn7.6/linux/utils/convpp -32 StrFns_x_with_q32.ff StrFns_x_with_q32.pp 
~frum/vn7.6/linux/utils/convpp -32 StrFns_x_with_theta32.ff StrFns_x_with_theta32.pp 
~frum/vn7.6/linux/utils/convpp -32 StrFns_x_with_exner32.ff StrFns_x_with_exner32.pp 
~frum/vn7.6/linux/utils/convpp -32 StrFns_x_with_u32.ff StrFns_x_with_u32.pp 
~frum/vn7.6/linux/utils/convpp -32 StrFns_x_with_v32.ff StrFns_x_with_v32.pp 
~frum/vn7.6/linux/utils/convpp -32 StrFns_x_with_w32.ff StrFns_x_with_w32.pp 
~frum/vn7.6/linux/utils/convpp -32 StrFns_x_with_windspeed32.ff StrFns_x_with_windspeed32.pp
# Delete the ff 
rm *32.ff 

# ===== OPTIONS 2 & 5 ============================
# Commands to be performed on the ibm for options 2 & 5
gzip CorFns_Vertx*.dat
# Transfer files to linux
rsync *.gz rld036:/data/local/frxb/...

# Commands to be performed on the linux box for options 2 & 5
# Unzip the files
gzip -d *.gz
 

# ===== OPTION 3 =================================
# Commands to be performed on the ibm for option 3
# export EXP=Localization/InputData/NoRP_20Sep2011_1500Z/10Ap

gzip *.dat
# Transfer files to linux
echo $EXP
rsync *.gz rld036:/data/local/frxb/$EXP
rm *.gz

# Commands to be performed on the linux box for option 3
# Unzip the files
gzip -d *.gz

# ===== OPTION 4 =================================
# Commands to be performed on the ibm for option 4
gzip VertSpecData*.dat
# Transfer files to linux
rsync *.gz rld036:/data/local/frxb/...

# Open up ftp link to start transfer of files sftp rld036
# Commands to be performed on the linux box for option 4
# Unzip the files
gzip -d *.gz


# ===== OPTION 6 =================================
# Commands to be performed on the ibm for option 6
ieee -32 RealVars_p.ff RealVars_p32.ff
ieee -32 RealVars_pb.ff RealVars_pb32.ff
ieee -32 RealVars_pr.ff RealVars_pr32.ff
gzip *32.ff
gzip *.dat
# Transfer to linux box
rsync *.gz rld036:/data/local/frxb/...
# Delete stuff on ibm
rm *32.*.gz 

# Commands to be performed on the linux box for option 6
# Unzip the files first 
gzip -d *.gz 
~frum/vn7.6/linux/utils/convpp -32 RealVars_p32.ff RealVars_p32.pp
~frum/vn7.6/linux/utils/convpp -32 RealVars_pb32.ff RealVars_pb32.pp
~frum/vn7.6/linux/utils/convpp -32 RealVars_pr32.ff RealVars_pr32.pp
# Delete the ff 
rm *32.ff

6.3 Plot results using TIDL

The plotting routines are found in /home/mm0200/frxb/My_IDL_Library/Corr+Bal_Diags on the Met Office linux system. The IDL code works by taking the outputs processed in previous sections and generating the following:

Encapsulated postscript files of required data (put in a separate directory).
Jpeg equivalents of these postscript files.
Html file linking the jpegs together so that theu can be viewed conveniently.

The eps files (actually .ps) can be kept if required for use in publications if required.

The use of the idl programs should be evident from the comments in each file. The programs rely on the existance of input and output directories which should be set-up in advance. Directories may include the directory containing the input data, the directory to contain the ps files, the directory containing the html and jpeg files, etc.

Each different option (0-5) uses a different idl master program (only master programs need to be edited - not subroutines) as follows.

Option 0 (compute basic or derived fields from a specified ensemble member) uses MasterPlotControlFields.pro.
Option 1 (compute 3-D variance, structure and correlation functions) uses MasterPlotVar_x.pro and MasterPlotCor_x.pro.
Option 2 (compute vertical correlation functions with a chosen reference level) uses MasterPlotCor_Vert_x.pro.
Option 3 (compute balance diagnostics) uses MasterPlotCorr_Balx.pro.
Option 4 (compute variances and vertical correlations in spectral space) uses MasterPlotSpectral.pro.
Option 5 (compute vertical correlation functions with a chosen reference level, and averaged over longitude) uses MasterPlotCor_Vert_x.pro.
Option 6 (Pressure (balanced / unbalanced) diagnostics) uses MasterPlotPressSpec.pro.

Namelist variable	Type	Default	Used with Master Routine	Description
General variables
HRAA_MasterRoutine	Integer	1	n/a	0 = Compute basic or derived fields from a specified ensemble member. 1 = Compute 3-D variance, structure and correlation functions. 2 = Compute vertical correlation functions (as a function of horizontal position) with a chosen reference level. 3 = Compute balance diagnostics (with and without localization). 4 = Compute variances and vertical correlations in spectral space. 5 = As option 2, but averaged over longitude. 6 = Pressure (balanced / unbalanced) diagnostics 7 = Compute maps of balance diagnostics 8 = Theta (balanced / unbalanced) diagnostics
UseHowManyMembers	Integer	0	1, 2, 3, 4, 5, 6	Set the number of ensemble members in the covariance calculations. Set to 0 to use all ensemble members in the `VAR_LSDUMP_DIR` directory.
UseMember	Integer	1	0	To specify which ensemble member is converted for later plotted. Note that the control member is 1.
PertFromMean	Logical	.FALSE.	3 at moment	Set to calculate perturbations from the mean. Unset to calculate perturbations from the control.
NMC	Logical	.FALSE.	6 and 8 at moment	Set to use NMC forecast differences instead of ensembles.
NMC_DIRS	Character		6 and 8 at moment	Array of directories containing NMC data
Localization-related variables
Localization_type	Integer	0	1, 2, 3	0 = No localization (raw covariances). 1 = Static localization. 2 = Super-static localization. 3 = Simplified SENCORP localization. 4 = ECO-RAP localization. 5 = Super-ECO-RAP localization.
HorizLenScale	Real	100.0	1, 3	Horizontal smoothing distance (grid length multiples) used to generate smoothed members in (super)ECO-RAP and SENCORP localizations. For (super)ECO-RAP localization this is relevant only when Nk>0 and HorizSpectype > 1.
VertLenScale	Real	10.0	1, 3	Vertical smoothing distance (grid length multiples) used to generate smoothed members in (super)ECORAP and SENCORP localizations. For (super)ECO-RAP localization this is relevant only when Nkz>0 and VertSpectype > 1.
Nk	Integer	0	1, 3	Number of horizontal wavenumbers. Set to -1 for no horizontal localization.
Nkz	Integer	0	1, 3	Number of vertical wavenumbers. Set to -1 for no vertical localization.
HorizSpecType	Integer	1	1, 3	Horizontal spectrum shape. 1 = square, 2 = function described by length1_horiz and length2_horiz
VertSpecType	Integer	1	1, 3	Vertical spectrum shape. 1 = square, 2 = function described by length1_vert and length2_vert
length1_horiz	Real	15.0	1, 3	Quasi flat distance in horizontal for moderation fn
length2_horiz	Real	50.0	1, 3	Drop-off distance in horizontal for moderation fn
length1_vert	Real	5.0	1, 3	Quasi flat distance in vertical for moderation fn
length2_vert	Real	15.0	1, 3	Drop-off distance in vertical form moderation fn
SENCORPorder	Integer	1	1, 3	Order of simplified SENCORP method (m in Bishop paper, 0, 1 or 2).
ECORAPorder	Integer	1	1, 3	Order of (super)ECO-RAP adaptive 'sub' matrix (q in Bishop paper, 1 or 2).
OnlyLocalization	Logical	.FALSE.	1	Set to .TRUE. to output only the localization function in the output fields (static and ECO-RAP localization methods only).
CQ_type	Integer	1	1, 3	1 = Calculate each element of CQ explicitly (uses CQ_radius_horiz and CQ_radius_vert) 2 = Make ensemble of square-root members (uses CQ_radius_horiz) with large number of members (used for faster balance diagnostics) 3 = As 2, but uses smaller number of members (recommended for structure function calculations).
CQ_radius_horiz	Integer	100	1, 3	Number of horizontal points (in x or y directions) to include in action of CQ matrix (ECORAP).
CQ_radius_vert	Integer	50	1, 3	Number of vertical points to include in action of CQ matrix (ECORAP).
JacobiPairs	Integer	0	1, 3	No. of rotations in localization schemes
JacobiSeed	Integer	211541	1, 3	Random No. seed for Jacobi rotations
Parameter conversion switches
Theta2T	Logical	.TRUE.	0, 1, 2, 4, 5	Convert theta to temperature (still store in LS % theta in code)
hydro_p	Logical	.FALSE.	0, 1, 2, 3, 4, 5, 6	Replace exner with hydrostatic exner
Exner2p	Logical	.TRUE.	0, 1, 2, 4, 5	Convert exner pressure to pressure (still store in LS % exner in code)
uv2psichi	Logical	.FALSE.	0, 1, 2, 4, 5	Convert u and v fields to psi and chi (store in new variables LS % psi and LS % chi in code)
uv2vortdiv	Logical	.FALSE.	0, 1, 2, 4, 5	Convert u and v fields to vorticity and divergence (store in new variables LS % psi and LS % chi in code)
Flags to switch on/off covariance calculations
Calc_u_stats	Logical	.FALSE.	1, 2, 4, 5	Set to .TRUE. to calculate structure/correlation functions associated with u (or psi, vorticity).
Calc_v_stats	Logical	.FALSE.	1, 2, 4, 5	Set to .TRUE. to calculate structure/correlation functions associated with v (or chi, divergence).
Calc_theta_stats	Logical	.FALSE.	1, 2, 4, 5	Set to .TRUE. to calculate structure/correlation function associated with theta.
Calc_exner_stats	Logical	.FALSE.	1, 2, 4, 5	Set to .TRUE. to calculate structure/correlation function associated with exner.
Calc_q_stats	Logical	.FALSE.	1, 2, 4, 5	Set to .TRUE. to calculate structure/correlation function associated with q.
Calc_w_stats	Logical	.FALSE.	1, 2, 4, 5	Set to .TRUE. to calculate structure/correlation function associated with w.
Calc_ws_stats	Logical	.FALSE.	1, 2, 4, 5	Set to .TRUE. to calculate structure/correlation function associated with wind speed.
Details about the (fixed) source point (to compute covariances with)
Source_x	Integer	180	1, 2, 3	Source point index in the x-direction.
Source_y	Integer	144	1	Source point index in the y-direction
Source_z	Integer	36	1, 2, 4, 5	Source point index in the z-direction.
CalcInPlanesOnly	Logical	.TRUE.	1, 2,3	Set to .TRUE. to calculate structure/correlation functions in the Source_x, Source_y, Source_z planes only. Set to .FALSE. to calculate structure/correlation functions at all positions (latter very expensive).
CalcInLinesOnly	Logical	.FALSE.	1, 2,3	Set to .TRUE. to calculate structure/correlation functions in the Source_x, Source_y, Source_z lines only.
Balance calculation variables
Bal_flag_alpha	Logical	.FALSE.	3	Set to include 'second order' terms in horizontal balance equation.
Bal_glag_beta	Logical	.FALSE.	3	Similar to above (can't remember difference)
y_from	Integer	1	3	First latitude
y_to	Integer	288	3	Last latitude
SmoothingScale_from	Real	0.0	3	Starting scale for sweep of lengthscales to scale ensemble members before calculating balance diagnostics (units of grid lengths).
SmoothingScale_to	Real	50.0	3	Ending scale for sweep of lengthscales to scale ensemble members before calculating balance diagnostics (units of grid lengths).
SmoothingScale_inter	Real	0.0	3	Interval of sweep of lengths to scale ensemble members before calculating balance diagnostics (units of grid lengths).
Bal_CalcMatrix	Logical	.FALSE.	3	Set to calculate vertical covariance matrices for theta, thetaH, exner, exnerH
Bal_CalcTerms	Logical	.FALSE.	3	Set to calculate terms in balance equations
Pressure (balanced / unbalanced) diagnostics variables
Pressure_realsp	Logical	.TRUE.	6	Calculate pressure diagnostics in real-space
Pressure_specsp	Logical	.TRUE.	6	Calculate pressure diagnostics in spectral-space
Pressure_domain_ens	Logical	.TRUE.	6	Calculate pressure diagnostics averaged over domain and ensemble
RemoveMeans	Logical	.TRUE.	6	Remove means from covariance calculations
degrade_lengthscale	Real	0.0	6	Degrade lengthscale by this amount (km) - convolutes with a Lorentzian
Theta_realsp	Logical	.TRUE.	8	Calculate theta diagnostics in real-space
Theta_specsp	Logical	.TRUE.	8	Calculate theta diagnostics in spectral-space
Vertical_regression	Integer	0	6	Options to perform vertical regression on the balanced pressure 0 = No VR 1 = Read VR matrix from file 2 = Calculate VR matrix from data
Vertical_regression_file	Character	StandardRegression_70	6	Filename of VR matrix (in vertical modes directory)
Vertical_regression_write	Logical	.FALSE.	6	Set to write VR matrix to file (in vertical modes directory)
Vertical_regression_ip	Integer	0	6	Inner product to use 0 = Identity 1 = As psi 2 = As chi 3 = As Ap 4 = As mu
GlobalMeanBal	Logical	.FALSE.	6	Consider global mean pressure as balanced
FFTbp	Logical	.FALSE.	6	If set the Fourier method is used to calculate balanced pressure, otherwise GCR is used.
MaxGCRIters	Integer	10	6	Maximum number of iterations for the Laplace equation solver
GCR_Restart_value	Integer	8	6	For GCR(k) method
GCR_precon_option	Integer	0	6	GCR preconditioning option (only 0 implemented successfully - no preconditioning)
deltat	Real	1.0	6	For ADI preconditioner (not implemented)
read_bp	Logical	.FALSE.	6	Set to read-in the balanced pressures instead of using solver
Z	Integer	10	6	Z
JustOutputBalp	Logical	.FALSE.	6	Set to output only the balanced pressure (zero bcs)