CMIP configurations

Like before, I'm assuming that coupling slows the components down about 10% (this seemed to be roughly true on the IBM).

Estimates from 23 July 2015

We sent an e-mail to Dave Munday on 23 July 2015, where our estimate for UKESM-LO was 4,725 AUs and for UKESM-LO-CC was 1,755 AUs. If we'd correctly switched from node-hours to core-hours, these numbers would have been 32 * 4,725 = 151,200 AUs and 32 * 1,755 = 56,160 AUs.

Our estimate for ORCA1+MEDUSA was 143.4 AUs, so that should have been 143.4 * 32 = 4,589 AUs.

UKESM-LO

UM N96 running full chemistry and coupled to ORCA1 MEDUSA. For this run it makes sense to use XIOS included in NEMO (rather than separate executable).

Testing so far suggests that full chemistry is about the same speed on Cray as IBM, and I have Cray times for ORCA1 MEDUSA from Richard.

1 model year/day

Speed (model years/day) Speed/0.9 (model years/day) Predicted components Cray ARCHER*
Recommended components Total nodes Recommended components Total nodes
fullChemN96 -dt30 medOrca1 fullChemN96 -dt30 (nodes) medOrca1 (nodes) fullChemN96 -dt30 (nodes) medOrca1 (nodes)
1 1.11 184 (8x23) 12 (4x3) 192 (16x12) (6) 32 (8x4) (1) 7 192 (16x12) (8) 24 (6x4) (1) 9
* ARCHER has 24 CPUs per node, the cray has 32.

On Cray

  • one model year is completed in 24 hours
  • one model year needs 7 * 24 = 168 node-hours
  • one model year needs 32 * 168 = 5,376 core-hours

On ARCHER

  • one model year is completed in 24 hours
  • one model year needs 9 * 24 = 216 node-hours
  • one model year needs 24 * 216 = 5,184 core-hours
  • one model year needs 5,184 * 15 = 77,760 AUs
  • if correction factor to go from XC40 to XC30 is needed, one model year needs 1.1 * 77,760 = 85,536 AUs

2 model years/day

Speed (model years/day) Speed/0.9 (model years/day) Predicted components Cray ARCHER*
Recommended components Total nodes Recommended components Total nodes
fullChemN96 -dt30 medOrca1 fullChemN96 -dt30 (nodes) medOrca1 (nodes) fullChemN96 -dt30 (nodes) medOrca1 (nodes)
2 2.22 440 (22x20) 24 (6x4) 448 (28x16) (14) 32 (8x4) (1) 15 456 (24x19) (19) 24 (6x4) (1) 20
* ARCHER has 24 CPUs per node, the cray has 32.

On Cray

  • one model year is completed in 24 / 2 = 12 hours
  • one model year needs 15 * 12 = 180 node-hours
  • one model year needs 32 * 180 = 5,760 core-hours

On ARCHER

  • one model year is completed in 24 / 2 = 12 hours
  • one model year needs 20 * 12 = 240 node-hours
  • one model year needs 24 * 240 = 5,760 core-hours
  • one model year needs 5,760 * 15 = 86,400 AUs
  • if correction factor to go from XC40 to XC30 is needed, one model year needs 1.1 * 86,400 = 95,040 AUs

UKESM-LO-CC

UM N96 running offline oxidants and coupled to ORCA1 MEDUSA. For this run it makes sense to use XIOS included in NEMO (rather than separate executable).

Testing so far, suggest the offline oxidants is 1.5 times slower on Cray compared to IBM, and I have Cray times for ORCA1 MEDUSA from Richard.

2 model years/day

Speed (model years/day) Speed/0.9 (model years/day) Predicted components Cray ARCHER*
Recommended components Total nodes Recommended components Total nodes
offOxN96 -dt30 medOrca1 offOxN96 -dt30 (nodes) medOrca1 (nodes) offOxN96 -dt30 (nodes) medOrca1 (nodes)
2 2.22 (or 3.33 if *1.5) 220 (20x11) 24 (6x4) 224 (16x14) (7) 32 (8x4) (1) 8 240 (16x15) (10) 24 (6x4) (1) 11
* ARCHER has 24 CPUs per node, the cray has 32.

On Cray

  • one model year is completed in 24 / 2 = 12 hours
  • one model year needs 8 * 12 = 96 node-hours
  • one model year needs 32 * 96 = 3,072 core-hours

On ARCHER

  • one model year is completed in 24 / 2 = 12 hours
  • one model year needs 11 * 12 = 132 node-hours
  • one model year needs 24 * 132 = 3,168 core-hours
  • one model year needs 15 * 3,168 = 47,520 AUs
  • if correction factor to go from XC40 to XC30 is needed, one model year needs 1.1 * 47,520 = 52,272 AUs

3 model years/day

Speed (model years/day) Speed/0.9 (model years/day) Predicted components Cray ARCHER*
Recommended components Total nodes Recommended components Total nodes
offOxN96 -dt30 medOrca1 offOxN96 -dt30 (nodes) medOrca1 (nodes) offOxN96 -dt30 (nodes) medOrca1 (nodes)
3 3.33 (or 5.00 if *1.5) 374 (22x17) 36 (6x6) 384 (24x16) (12) 64 (10x6) (2) 14 384 (24x16) (16) 48 (8x6) (2) 18
* ARCHER has 24 CPUs per node, the cray has 32.

On Cray

  • one model year is completed in 24 / 3 = 8 hours
  • one model year needs 14 * 8 = 112 node-hours
  • one model year needs 32 * 112 = 3,584 core-hours

On ARCHER

  • one model year is completed in 24 / 3 = 8 hours
  • one model year needs 18 * 8 = 144 node-hours
  • one model year needs 18 * 144 = 3,456 core-hours
  • one model year needs 15 * 3,456 = 51,840 AUs
  • if correction factor to go from XC40 to XC30 is needed, one model year needs 1.1 * 51,840 = 57,025 AUs

MEDUSA at ORCA1

10 model years/day

Speed (model years/day) Predicted components Cray ARCHER*
Recommended components (nodes) Recommended components (nodes)
10 120 (12x10) 128 (12x10) (4) 120 (12x10) (5)

On Cray

  • one model year is completed in 24 / 10 = 2.4 hours
  • one model year needs 4 * 2.4 = 9.6 node-hours
  • one model year needs 32 * 9.6 = 307.2 core-hours

On ARCHER

  • one model year is completed in 24 / 10 = 2.4 hours
  • one model year needs 5 * 2.4 = 12 node-hours
  • one model year needs 24 * 12 = 288 core-hours
  • one model year needs 15 * 288 = 4,320 AUs
  • if correction factor to go from XC40 to XC30 is needed, one model year needs 1.1 * 4,320 AUs = 4,752 AUs

20 model years/day

Speed (model years/day) Predicted components Cray ARCHER*
Recommended components (nodes) Recommended components (nodes)
20 255 (15x17) 256 (15x17) (8) 264 (15x17) (11)

On Cray

  • one model year is completed in 24 / 20 = 1.2 hours
  • one model year needs 8 * 1.2 = 9.6 node-hours
  • one model year needs 32 * 9.6 = 307.2 core-hours

On ARCHER

  • one model year is completed in 24 / 20 = 1.2 hours
  • one model year needs 11 * 1.2 = 13.2 node-hours
  • one model year needs 24 * 13.2 = 316.8 core-hours
  • one model year needs 15 * 316.8 = 4,752 AUs
  • if correction factor to go from XC40 to XC30 is needed, one model year needs 1.1 * 4,752 = 5,228 AUs

MEDUSA at ORCA025

1 model year/day

Speed (model years/day) Predicted components Cray ARCHER*
Recommended components (nodes) Recommended components (nodes)
1 480 (24x20) 480 (24x20) (15) 480 (24x20) (20)

On Cray

  • one model year is completed in 24 / 1 = 24 hours
  • one model year needs 15 * 24 = 360 node-hours
  • one model year needs 32 * 360 = 11,520 core-hours

On ARCHER

  • one model year is completed in 24 / 1 = 24 hours
  • one model year needs 20 * 24 = 480 node-hours
  • one model year needs 24 * 480 = 11,520 core-hours
  • one model year needs 15 * 11,520 = 172,800 AUs
  • if correction factor to go from XC40 to XC30 is needed, one model year needs 1.1 * 172,800 = 190,080 AUs

2 model years/day

The curve doesn't fit the point near here well, so I'm just going to use Julien's data point: 36x30 (1,080) runs at 1.98 model years/day

Speed (model years/day) Predicted components Cray ARCHER*
Recommended components (nodes) Recommended components (nodes)
2 1,080 (36x30) 1,088 (36x30) (34) 1,080 (36x30) (45)

On Cray

  • one model year is completed in 24 / 2 = 12 hours
  • one model year needs 34 * 12 = 408 node-hours
  • one model year needs 32 * 408 = 13,056 core-hours

On ARCHER

  • one model year is completed in 24 / 2 = 12 hours
  • one model year needs 45 * 12 = 540 node-hours
  • one model year needs 24 * 540 = 12,960 core-hours
  • one model year needs 15 * 12,960 = 194,400 AUs
  • if correction factor to go from XC40 to XC30 is needed, one model year needs 1.1 * 194,400 = 213,840 AUs