===================================================================== Porting & testing notes for Vn7.8 HadGEM3-A GA3.0 package from Met O. N96 L85 ===================================================================== Original Met O job akqga: http://collab.metoffice.gov.uk/twiki/bin/view/Project/CAPTIVATE/HadGEM3Evolution Installation instructions are given here: http://collab.metoffice.gov.uk/twiki/bin/view/Support/HG3AGA30instructions Puma files are stored here: /home/umui/hadgem3/vn7.8/HG3AGA30 HECToR files are stored here: /work/n02/n02/hum/vn7.8/HG3AGA30 ===================================================================== 5 January 2012 Timing runs for 10 days on HECToR Phase 3: Pes Nodes Decomp (EWxNS) Time (s) --- ----- -------------- -------- 64 2 8x8 3102.065 128 4 8x16 1721.519 ===================================================================== 9 February 2011 Porting: 1. Import branches. - Unpack branch package: tar -zxvf HG3AGA30_patches.tgz - Check they don't already exist on puma (i.e. imported by another user). In UM trac browser search for branch names (will automatically be in commit message when created). Then work out equivalent puma revision numbers to Met O revisions given in branch. These already exist under fcm:um_br/dev/malcolm/ VN7.8_incrCLO_ukmo/ Met O r 28763 -> puma r 5538 VN7.8_JULES_fixes_ukmo/ r 28826 -> r 5546 VN7.8_reinstate_ISCCP_ukmo/ r 28761 -> r 5535 Need to create new branch: fcm:um_br/dev/annette/VN7.8_climateRestartIOError_ukmo And apply patch: Met O r 29090 -> puma r 5671 2. Upload basis file - Unpack local package: tar -zxvf HG3AGA30_local.tgz - Create new job: xftva - Import basis file 3. Sort input files. - Copy local files to directory on puma: /home/umui/hadgem3/vn7.8/HG3AGA30/ - Unpack ancillary files: tar -zxvf HG3AGA30_ancils.tgz - Copy files to directory on HECToR: /work/n02/n02/hum/vn7.8/HG3AGA30 4. Edit job - General details and job submission method Add username, tic-code, email address, hostname and QSUB submission details. - Script Inserts and Modifications Remove IBM environment variables - Directory environment variables Point to HECToR files. Set job directory to be $WORK/um/$RUNID - Hand-edits Point to puma files. - FCM Extract directories and Output levels Location of puma extract directory and HECToR job directory. - FCM Options for Atmosphere Puma FCM settings for 7.8. Put in equivalent branches and revisions for puma. Add in 7.8 ncas branches. - Compile Option for Atmosphere and Reconfiguration Switch to compile model executable and reconf. - User STASH-master files Point to puma files. Notes: * Initially 10 day test run - Looks OK I think * Run on 6x8 = 48 cores * Crashes with following STASH items so turn off: - (1,235) TDAY30yr, DIAG, UPD, package R - (2,207) TDAYM, DIAG, UPD, package R * Compile with 'high' optimisations. * Recuce output: - Lower FCM output to 1 - Move print output from operational to normal - Delete all pe files on successful completion. Timing results: All 10 day runs with dump after 10 days (6x8) Times from timer maximum elapsed wallclock time. Options Time (s) high optimisations & std output 3894.082 high optimisations & reduced output 3917.086 Timing runs & bit-comparison tests: 3d runs with 1d dumps: Times from Maximum elapsed wallclock time in UM timer. Pes Decomp Time (s) B/c 24 4x6 2344.568 y 48 6x8 1299.019 y 72 8x9 924.853 y 96 8x12 736.648 y Testing Reconfiguration OK Automatic-resubmission & b/c OK Archiving OK