Timings for full chemistry

All these runs are on the Power 7.

N216

The runs are different configurations of faceg, which is N216L85 with full stratospheric chemistry. I'm using the elapsed time for the total time. Each timestep seems to be 15 mins (192 timesteps for 2 days).

NMPPE*NMPPN (total)	Model run length	Total time (s)	Speed (model years per day)
8 * 24 (192)	Maximum memory limit of 50,000 MB exceeded
8 * 32 (256) (with UKCA_SYNC+)	2 days	2,622	0.183
8 * 32 (256)	2 days	2,609	0.184
16 * 32 (512)	2 days	1,475	0.325
32 * 32 (1,024)	2 days	1,010	0.475
32 * 32 (1,024)	1 day	503	0.476
32 * 64 (2,048)	2 days	985	0.487

⁺UKCA_SYNC is an additional routine added to measure the imbalance of UKCA_MAIN1, the addition of MPI barrier is likely to slow the code a bit.

Looks like the code will never quite reach 0.5 model years per day. The nitrates are still to be added in (probably another 10 tracers).

PE mapping (256 PEs)

NMPPE*NMPPN (total)	Model run length	Total time (s)	Speed (model years per day)
8 * 32 (about a month earlier than other runs)	2 days	2,609	0.184
8 * 32	2 days	2,615	0.184
16 * 16	2 days	2,660	0.180
16 * 16	2 days	2,650	0.181
32 * 8	2 days	2,707	0.177
32 * 8	2 days	2,689	0.179

N216 without Dr Hook

I've created cakdv which is a copy of faceg, but without Dr Hook, for these runs.

NMPPE*NMPPN (total)	Model run length	Total time (s)	Speed (model years per day)
16 * 16 (256)	5 days	5,896	0.204
16 * 32 (512)	5 days	3,012	0.398
32 * 32 (1,024)	5 days	1,842	0.651
64 * 32 (2,048)	5 days	1,421	0.844
64 * 36 (2,304)	5 days	1,409	0.852
64 * 64 (4,096)	5 days	1,644	0.730

N96 with Dr Hook

The runs are different configurations of facei, which is N96L85 with full stratospheric chemistry. They all have UKCA_SYNC in them. Each timestep seems to be 20 minutes (2,160 timesteps for 1 month).

NMPPE*NMPPN (total)	Model run length	Total time (s)	Speed (model years per day)
8 * 8 (64)	5 days	3,954	0.303
16 * 8 (128)	5 days	2,042	0.588
16 * 12 (192)	1 month	8,231	0.875
16 * 16 (256)	1 month	6,504	1.11
24 * 16 (384)	1 month	4,915	1.46
16 * 32 (512)	Too many processors in the North-South direction ( 32) to support the extended halo size. Try running with 28 processors.
32 * 16 (512)	1 month	4,946+	1.46
		4,295	1.68
24 * 24 (576)	1 month	3,880	1.86
32 * 24 (768)	1 month	3,464	2.08
40 * 24 (960)	1 month	3,128	2.30
48 * 28 (1,344)	1 month	2,928	2.46
64 * 24 (1,536)	Too many processors in the East-West direction ( 64) to support the extended halo size ( 4). Try running with 48 processors.

⁺ run at same time as a 32x16 for N216 offline oxidants (AM 8 Oct 2014), both runs seem surprisingly slow (this one on hpc2f, the other on hpc2e). The same run was done the following morning and much quicker.

N96 without Dr Hook

The same as above but without Dr Hook and using facew

NMPPE*NMPPN (total)	Model run length	Total time (s)	Speed (model years per day)
8 * 16 (128)	10 days	3,499	0.686
16 * 16 (256)	1 month	5,671	1.27
32 * 16 (512)	10 days	1,166	2.06
32 * 16 (512)	1 month	3,447	2.06
32 * 28 (896)	1 month	2,611	2.09
48 * 28 (1,344)	1 month	2,381	3.02

N96 with 30 minute time step

I've copied facei, and removed Dr Hook, to create facey for this.

NMPPE*NMPPN (total)	Model run length	Total time (s)	Speed (model years per day)
8 * 16 (128)	10 days	3,072	0.781
16 * 16 (256)	1 month	4,921	1.46
32 * 16 (512)	1 month	3,058	2.35
32 * 28 (896)	1 month	2,282	3.16
48 * 28 (1,344)	1 month	2,083	3.46