Timings for offline oxidants

All these runs are on the Power 7.

N216

Not Backward-Euler and NTZS=20

These runs are different configurations of faceo, which is N216L85 with offline oxidants (faceo was meant to be Backward-Euler but clearly isn't - on the 32 * 8 run, we'd probably lose about 266s by running as a proper Backward-Euler run, so they would be a bit faster) . I'm using the elapsed time for the total time. Each timestep seems to be 15 mins (960 timesteps for 10 days).

NMPPE*NMPPN (total) Model run lengthTotal time (s) Speed (model years per day)
32 * 8 (256) 10 days4,991 0.481
32 * 16 (512) 10 days2,681 0.895
32 * 32 (1024) 10 days1,723 1.39
64 * 32 (2048) 10 days `Signal received: SIGFPE - Floating-point exception' around timestep 657

As the proper Backward-Euler runs seem to be working, I don't think I'm going to worry about the problem at 64 * 32.

Backward-Euler and NTZS=15

Mohit says, `Regarding value of NZTS, see Andy’s experiment DKWW with UKCA N144 jobs at different versions. I think he had to increase it to 30 to make some configurations work. However, note that this was GA5.0/ vn8.5 or early vn8.6, so there may be some fixes added later that improved the stability of UKCA.'

NTZS is the parameter selected from UMUI at

  • Model Selection
    • Atmosphere
      • Scientific Parameters and Sections
        • Section 34: UKCA Chemistry and Aerosols
          • MODE
            • Number of substeps for Nucleation/Sedimentation: THIS IS NZTS

These runs are different configurations of facep, which is N216L85 with Backward-Euler offline oxidants.

Variations for 256PEs
NMPPE*NMPPN (total) Model run lengthTotal time (s) Speed (model years per day)
8 * 32 (256, evening) 10 days4,523 0.531
8 * 32 (256, during day) 10 days4,539 0.529
16 * 16 (256) 10 days4,415 0.544
32 * 8 (256, during day) 10 days4,464 0.538
32 * 8 (256, evening) 10 days4,469 0.538

This suggests that 32 * 8 is better than 8 * 32, but disagrees with the UM documentation in section `Inputs of interest for Parallel Computers' (thanks to Mohit for link).

Increasing PEs
NMPPE*NMPPN (total) Model run lengthTotal time (s) Speed (model years per day)
16 * 16 (256) 10 days4,415 0.544
32 * 16 (512) 10 days2,526 0.950
32 * 32 (1,024) 10 days1,650 & 1,654 1.45
48 * 32 (1,536) 10 days1,383 1.74
56 * 32 (1,792) 10 days1,338 1.79
64 * 32 (2,048) 10 days1,284 1.87
64 * 36 (2,304) 10 days1,241 1.93
64 * 40 (2,560) 10 days1,307 1.84
64 * 48 (3,072) 10 days1,301 1.84
64 * 64 (4,096) 10 days1,647 1.46

Should I be worried about lots of the following warnings (I think this is just for the large PE cases)? 

?  Warning Message: Extreme values of ND, MDT, or MD found


N216 without Dr Hook

I've created facex which is a copy of facep, but without Dr Hook, for these runs.

NMPPE*NMPPN (total) Model run lengthTotal time (s) Speed (model years per day)
16 * 16 (256) 10 days4,222 0.568
16 * 32 (512) 10 days2,255 1.06
32 * 32 (1,024) 10 days1,482 1.62
64 * 32 (2,048) 10 days1,135 2.11
64 * 36 (2,304) 10 days1,124 2.14
64 * 64 (4,096) 10 days1,251 1.92

N96 with Dr Hook

These runs are different configurations of facef, which is N96L85 with Backward-Euler offline oxidants

NMPPE*NMPPN (total) Model run lengthTotal time (s) Speed (model years per day)
16 * 8 (128) 1 month4,417 1.63
16 * 16 (256) 1 month2,539 2.84
32 * 16 (512) 1 month1,699 4.24
30 * 28 (840) 1 month1,483 4.86
36 * 28 (1,008) 1 month1,531 4.70
48 * 28 (1,344) 1 month1,418 5.08

N96 without Dr Hook

These runs are different configurations of facev, which is N96L85 with Backward-Euler offline oxidants

NMPPE*NMPPN (total) Model run lengthTotal time (s) Speed (model years per day)
16 * 8 (128) 1 month4,061 1.77
16 * 16 (256) 1 month2,260 3.19
32 * 16 (512) 1 month1,439 5.00
32 * 28 (896) 1 month1,163 6.19
48 * 28 (1,344) 1 month1,155 6.23

N96 with Dr Hook and with 30 minute time step

For this I've coped facef to facer, and changed the timestep

  • ->Model Selection
    • ->Atmosphere
      • ->Scientific Parameters and Sections
        • Time-Stepping
          • Number of timesteps per `period': CHANGED FROM 72 TO 48
NMPPE*NMPPN (total) Model run lengthTotal time (s) Speed (model years per day)
8 * 16 (128) 1 month3,748 1.92
16 * 16 (256) 1 month2,125 3.39
32 * 16 (512) 1 month1,429 5.04
32 * 28 (896) 1 month1,194 6.03
48 * 28 (1,344) 1 month1,163 6.19

N96 without Dr Hook and with 30 minute time step

I've changed facer, so that it no longer has Dr Hook.

NMPPE*NMPPN (total) Model run lengthTotal time (s) Speed (model years per day)
8 * 16 (128) 1 month3,486 2.07
16 * 16 (256) 1 month1,912 3.77
32 * 16 (512) 1 month1,189 6.06
32 * 28 (896) 1 month984 7.32
48 * 28 (1,344) 1 month995 7.24