How UMUI works

My interpretation of how UMUI works. All its direct output is written in ~/umui_jobs/<run id>. It

• Creates the namelists, such as CNTALL, CNTLATH, CNTLGEN, INITHIS, IOSCNTL, RECONA, SHARED, SIZES and STASHC.
• Creates the input data, such as PRESM_A
• Creates an additional assortment of files, such as
  • INITFILEENV: list of `export' statements
  • PPCNTL: "ARCHSYS='false'"
  • JOBSHEET: I think this is just for information
  • SCRIPT: later integrated into umuisubmit_compile, umuisubmit_rcf and umuisubmit_run)
  • SUBMIT: creates umuisubmit_compile, umuisubmit_rcf and umuisubmit_run
  • COMP_SWITCHES: sets the EXTR_<model component> and COMP_<model component> variables to `true' or `false'.
• Creates the extraction files
  • USR_FILE_OVRDS, USR_MACH_OVRDS and USER_PATHS_OVRS: the user overrides
  • FCM_UMATMOS_CFG, FCM_UMRECON_CFG and FCM_UMSCRIPTS_CFG.
  • FCM_BLD_COMMAND
  • EXTR_SCR
• Create the submission script, UMSUBMIT.
• It creates and runs MAIN_SCR, which
  • Set various variables
  • Determine if $RUN_COMPILE should be true (this is likely)
  • If $RUN_COMPILE is true, it runs EXTR_SCR which
    • Set various variables including
      • export UM_CONTAINER=$UM_SVN_BIND/container.cfg@vn8.6
    • extracts the base repository for each <model component> if EXTR_<model component> is set to `true' in COMP_SWITCHES (all the EXTR_<model component> false in my current run)
    • extracts the requested sub-models for each <model component> if COMP_<model component> is set to `true' in COMP_SWITCHES (COMP_UMSCRIPTS, COMP_UMATMOS and COMP_UMRECON are true in my current run). It uses the contents of USR_FILE_OVRDS, USR_MACH_OVRDS and USER_PATHS_OVRS here, if these files exist. If COMP_<model component> is true, it
      • export UM_JOB_CFG=.../FCM_<model component>_CFG
      • fcm extract -v 1 $UM_CONTAINER 1> $UM_OUTDIR/ext.out 2>&1, which
        • uses the FCM_<model component>_CFG file to build the <model component> code, including any branches, and the executables below /data/local/<user>/<run id>.
        • NOT SURE WHERE FCM_BLD_COMMAND comes into this? It
          • it uses fcm build -v 2 -j 6
          • it copies files to /data/cr/ukesm/<user>/<run id>
  • It runs UMSUBMIT, with arguments like

    UMSUBMIT -h <host> -m 3 <run id> stage_1_submit  
    

    where -m 3 indicates the load leveler queue and stage_1_submit is a run script yet to be written. UMSUBMIT then
    • Set various variables, including
      • submitid=`date +%j%H%M%S`
    • Replaces `:::submitid:::' string in SUBMIT with $submitid
    • Copies the contents of ~/umui_jobs/<run id> to /home/cr/ukesm/<user>/umui_runs/<run id>-$submitid, which is on the HPC accounts.
    • It makes SUBMIT on HPC account executable and runs it, which
      • set variables including
        • $SUBMITID, $RUNID and $JOBDIR
        • Destination for standard output files
      • Sets $RUN_MODEL (likely to be true)
      • Set if new reconfiguration executable is required, RCF_NEW_EXEC (probably true).
      • Set various the number of PEs, time limits etc.
      • Creates umuisubmit_compile, umuisubmit_rcf and umuisubmit_run (umuisubmit_var, and umuisubmit_*_header files are created along the way, but are deleted after the main umuisubmit_* files are created).
      • Creates stage_1_submit and stage_2_submit (which submit the umuisubmit_compile, umuisubmit_rcf and umuisubmit_run scripts).
      • Creates the archive scripts, archi_submit, archi_script and archive.todo
    • It remotely uses llsubmit to submit stage_1_submit, which
      • runs umuisubmit_compile, which builds the code and executables
      • Uses llsubmit to submit stage_2_submit, which
        • runs umuisubmit_rcf, which
          • sets information for the LoadLeveler.
          • sets various variables, including
            • export VN=8.6
            • export RUN_ATMOS=false; export RCF_ATMOS=true; export RCF_NEW_EXEC=false, so that UMScr_TopLevel will run the reconfiguration executable
          • runs several scripts from /projects/um1/vn8.6/ibm/scripts, including
            • UMScr_TopLevel, which
              • gets the namelists and several other of the the files generated by UMUI.
              • creates the pe_output directory
              • runs the qsmaster script from same directory, which determines that qsrecon script should be run, and
                • qsrecon runs the reconfiguration run
              • run submitchk from the same directory
        • uses llsubmit to run umuisubmit_run, this is similar to umuisumbit_rcf, except that
          • export RUN_ATMOS=true and export RCF_ATMOS=false.
          • qsmaster runs the qsatmos script, which run the standard run

I'm not sure how the archiving works, but looks like it's done with qsarchive.py.