GLOMAP

Offline oxidants at N216

Details of run

  • faceo (mstringe)
  • GLOMAP + dust
  • 256 (8 * 32) PEs
  • Grid size is 192*144*85
  • Runs for 10 days

10 days has been chosen to be a similar time to the run time for the Bakward-Euler run at N96 (5,000s before the improvements).

Calling tree

The times in the tables below are the total time, which is the time in each routine and the routines below it, except where `itself' is written to indicate time taken solely by that routine. Only the main routes - as given by time spent in them - are general shown.

Profiling for UM_SHELL

Routines
UM_SHELL (4,969s)
U_MODEL_4A (4,965s)
ATM_STEP_4A* (3,010s) UKCA_MAIN1 (1,611s)
ATMOS _PHYS- ICS1 (1,047s) EG_ COR- RECT_ TRAC- ERS (45s) ATMOS _PHYS- ICS2 (470s) EG_ SL_ HELM- HOLTZ (382s) TR_ SET_ PHYS _4A* (63s) EG_CORRECT _TRACERS _UKCA (208s) SL_ TRAC- ER1_ 4A (157s) EG_ SL_ MOI- STURE (88s) EG_SL_ FULL_WIND (161s)  ⇓  UP- DATE _M_ STAR (96s) ATM_ STEP_ STASH (30s)  ⇓   ⇓  See profiling for UKCA_ MAIN1 below
See profile for ATMOS_ PHYSICS1 and EG_CORRECT _TRACERS below See profile for ATMOS_ PHYSICS2 and EG_SL_ HELMHOLTZ below EG_SL_WIND_U, EG_SL_WIND_V & EG_SL_WIND_W (48 + 48 + 54 = 150s)  ⇓  EG_Q_ TO_MIX (97s)  ⇓  STASH (120s)
Itself (144s) EG_INTERPOLATION _ETA (283s) DEP- ARTURE_ POINT _ETA (105s) EG_SWAP_ BOUNDS_DP (219s) STWORK (120s)
EG_ CUBIC_ LAG- RANGE (115s, itself) EG_VERT_ WEIGHTS_ ETA (25s, itself) MONO_ ENFORCE (22s, itself) Itself (40s) See profile for SWAP_ BOUNDS _DP below SPA- TIAL (41s) PP_ HEAD (38s) EXP- PXI (24s, itself)
*should also link to SWAP_BOUNDS_DP, like many other returns.

After considering our fixes, the mean time EG_SL_HELMHOLTZ seems to be significantly larger and the mean time in ATMOS_PHYSICS1 seems to be significantly less than the N96 Backward-Euler run.

The profiling tables lower down suggest that around an extra 100s is spent in TRI_SOR_DP_DP at N216, which accounts for a large chunk of the extra time in EG_SL_HELMHOLTZ. While all the routines below RAD_CTL and MICROPHYS_CTL seem to be much quicker, which accounts for the reduced time in ATMOS_PHYSICS1.

Profiling for UKCA_MAIN1

Routines Total mean time
UKCA_MAIN* (1,611s) 1,611s
UKCA_AERO_CTL (1,006s) UKCA_ ACTIVATE (106s) UKCA_ CHEMISTRY_CTL (266s) UKCA_ EMISSION _CTL (42s) 1,420s
UKCA_AERO_STEP (963s) UKCA_ ABDULRAZZAK_ GHAN (97s) ASAD_CDRIVE (252s) 1,312s
UKCA_COAGWITHNUCL (455s) UKCA_ CONDEN (180s) UKCA_ CHECK_ MD_ND (81s, itself) UKCA_ CALCNUCRATE (79s) UKCA_ VOLUME_ MODE (44s) Itself (92s) ASAD_SPMJPDRIV (244s) 1,217s
Itself (375s) UKCA_ SOLVECOAGNUCL _V (80s, itself) UKCA_ COND_ COFF_V (111s, itself) Itself (69s) UKCA_ BINAPARA (74s, itself) Itself (22s) ASAD_SPIMPMJP (243s) 1,189s
SPLINSLV2 (151s, itself) SPFULJAC (76s, itself) 1,173s
*UCKA_MAIN also calls STASH

UKCA_CHEMISTRY_CTL is significant here, as oppose the Backward-Euler run at N96 (where UKCA_CHEMISTRY_CTL_BE was 1.23s). This was meant to be Backward-Euler run but clearly it isn't. And we could probably lose about 266s by running a Backward-Euler run.

Profiling for ATMOS_PHYSICS1 and EG_CORRECT_TRACERS

ATMOS_PHYSICS1* (1,047s) EG_CORRECT _TRACERS (45s)
Itself (275s) RAD_CTL (332s) MICROPHYS_CTL (122s) NI_GWD _CTL (300s)  ⇓   ⇓  EG_MASS_ CONSERVATION (41s)
LW_RAD (237s) SW_RAD (77s) LS_PPN (113s) G_ WAVE _5A (242s) GW_ USSP (58s) GLOBAL _2D_ SUMS (87s, itself) Itself (29s)
RADIANCE_CALC (309s) LS_PPNC (106s) SWAP_ BOUNDS (see table below)
UKCA_ RADAER _BAND_ AVERAGE (162s, itself) SOLVE_BAND_K_EQV (116s) UKCA_ RADAER_ COMPUTE _AOD (13s, itself) LSP_ICE (78s) Itself (28s)
MCICA_ SAMPLE (90s) LSP_ INIT (18s) LSP_ FALL (15s)
MONOCHR- OMATIC_ RADIANCE (78s) Itself (10s) Itself (13s)
MONOCHR- OMATIC_ RADIANCE _TSEQ (67s)
MCICA_ COLUMN (67s)
TWO_COEFF (46s)
TRANS_ SOURCE_ COEFF (27s) Itself (7s)
Itself (17s)
  
* ATMOS_PHYSICS1 also calls COSP_MAIN. However, the routines in atmosphere/COSP don't have Dr Hook calls. The TIMER output suggests that about ?s is spent in these routines.

Profiling for ATMOS_PHYSICS2 and EG_SL_HELMHOLTZ

ATMOS_PHYSICS2 (470s) EG_SL_HELMHOLTZ (382s)
NI_CONV_CTL (222s) NI_IMP_CTL (90s) SWAP_BOUNDS, SWAP_BOUNDS_2D_MV & SWAP_BOUNDS_MV (see table below) EG_BICGSTAB (253s) EG_HELM_RHS_STAR (92s)
GLUE_CONV_6A (165s) IMP_SOLVER (47s) EG_PRECON (173s) EG_SISL_INIT (69s)
Itself (79s) MID_CONV_6A (50s) TRI_SOR_DP_DP (173s) EG_SISL_INIT_UVW (66s)
Itself (20s) Itself (126s) Itself (39s)

Profiling for SWAP_* routines

Routines Total mean time
EG_SWAP_BOUNDS_DP (s) ATMOS_PHYSICS1, ATMOS_PHYSICS2, G_WAVE_5A, ... 219 + ...
SWAP_BOUNDS & SWAP_BOUNDS_DP (437 + 252 = 689s) SWAP_BOUNDS_MV (114s, itself) 803s
SWAP_BOUNDS_EW_DP (360s) SWAP_BOUNDS_NS_DP (328s, itself) 802s
SWAP_BOUNDS_EW_H1_DP (263s, itself) Itself (97s) 802s