MOHCCP kick-off: hybrid model

9 May 2018

Hybrid model at N96-N48-ORCA1 resolution

Motivation, key points and performance of hybrid

  • We have improved the performance of the aerosol and the chemistry (mostly by adding OpenMP) but they are still very expensive. Compared to running just the dynamical core and some cheap extras, aerosol and chemistry slow the runtime by roughly the following factors
    Adding AerosolAdding Aerosol & Chemistry
    UM8.6 on IBM X1.8 times slower X4.5 times slower
    UM11.0 on Cray X1.6 times slower X2.9 times slower
  • The hybrid model runs this expensive aerosol and chemistry in a separate UM at a lower resolution
    • When dynamical core is at N216 (Snr), aerosol & chemistry is at N96 (Jnr)
    • When dynamical core is at N96 (Snr), aerosol & chemistry is at N48 (Jnr)
  • Coupling of executables is all done through OASIS-MCT.
  • The full chemistry is called every hour, and the coupling between Snr and Jnr happens once an hour just after this (more often and Snr and Jnr would have to wait for one another)
  • We try to lock Jnr to Snr by sending most of the dynamical core variables from Snr and overwriting these fields in Jnr every hour.
    • We send all velocities, dry virtual potential temperature, etadot (change in vertical coordinate with time), and exner function (related to pressure).
    • We no longer send dry density, as this seemed to ruin mass conservation
    • Moisture variables come with their own mask, and so we don't try to degrade their resolution and send these.
  • Any aerosol or chemistry field required by Snr is sent from Jnr every hour
  • While Jnr is about 1/2 the resolution of Snr is has the expensive aerosol and chemistry. Consequently it requires a similar number of nodes to Snr to run as fast.
    • Hence for a fixed resource, Snr and Jnr will each have about 1/2 the nodes which a standard ATM would have.
    • In this case the hybrid model is about 40% faster than a GA + StratTrop (i.e. UM with full chemistry)
  • By removing the slow processes to a lower resolutions, the top speed of the hybrid model is much higher (but uses more resources)
    • The top speed of a hybrid model is about twice that of a standard GA + StratTrop.
  • Estimates of top speeds (note that we haven't coupled hybrid to ocean or built UKESM N216-ORCA025 yet - we have built a UKESM-CN N216-ORCA025)
    N96-ORCA1/ N96-N48-ORCA1 N216-ORCA025/ N216-N96-ORCA025
    UKESM 5 model years/day 1.8 model years/day
    UKESM-hybrid 10 model years/day 3.6 model years/day