GLOMAP

Methodology

Introduction

I've been asked to determine the time added to a standard UM run by adding GLOMAP-mode and see if there was anyway we could reduce this time. It was suggested that I use Dr Hook to profile the code, which is what I've done.

Dr Hook is software written by ECMWF to profile code. All new UM routines are required to have a call to dr_hook at the start and end of the routine (the only routines without dr_hook calls that I'm aware of is the COSP routines, which is apparently an old piece of code, which shouldn't be that important for this study). Dr Hook uses these calls to determine the time spent in each routine (self time), and the time spent in each routine plus all the routines it calls (total time), among other things for each PE - and stores this in files called drhook.prof.<PE number> where the <PE number> runs from 1 to the total number of PEs used. These files can be found in the main run directories, e.g. /data/cr/ukesm/mstringe/facec.

Reading these files

As far as I'm aware, there's no specific software designed to read these files, so I've written my own in perl. The perl code can be found in /home/cr/ukesm/mstringe/cgi-bin/drHook and can be run with the following commands

perl /home/cr/ukesm/mstringe/cgi-bin/drHook/drHook.pl --dir=<directory>
--nRoutines=<number of routines> --orderBy=<measure>:<stat> 
--html

where it should all be on one line and where

<directory> is the directory containing the Dr Hook files, e.g. /data/cr/ukesm/mstringe/facec. This is the only argument which is always required.
<number of routines> is the number of routines that are listed in the output. The routines to be listed is determined by the criteria which is defined with the --orderBy=<measure>:<stat> argument. The default option is the top ten.
<measure> can be
- self: the self time - the time spent just in this routine
- percent: the percentage of time spent in this routine (100 * self / absolute total)
- total: the total time - time in routine and the routines which it calls
- cumul: the cumulative time - the time spent in this routine and routine which call it
- numCalls: the number of calls to this routine
- selfPer: (self time) / (number of calls)
- totalPer: (total time) / (number of calls)
the default option is `self'.
<stat> can be
- min: the shortest time/number found for any PE in the Dr Hook files
- mean: taking the times/numbers for all PEs the mean time
- max: the longest time/number found for any PE in the Dr Hook files
- diff: max - min. Gives an indication of the most unbalanced routines, where this number will be high compared to their mean time.
--html is only included if you want the output in form for writing to a web page, when HTML is added to the output.

Note that the PE number system for the statistics produced by drHook.pl run from PE 0 to one less than the total number of PEs. In contract to the drhook.prof* files which run from PE 1 upwards, which is less standard, and so PE 5 refers to the data in drhook.prof.6.

The runs

The runs used for the profiling have originally come from Colin, Jane and Mohit

Backward-Euler (Colin's anyfl and copied to facec and many others).
Newton-Raphson (Jane's aofni and copied to faceb)
CLASSIC (Jane's aodmb and copied to faced)
N216 (Mohit's aivjg, copied to faceg and modified to include the read of offline oxidants for faceh).

These runs have had Dr Hook implemented as described below and all the GLOMAP testing is done under my folder face.

Adding Dr Hook

The add Dr Hook you need to select job from UMUI and make the following selections