New Research Satellite Observation Class - with humidity observation option

Ross Bannister, July 2003, December 2003, August 2004

For humidity data - fit to RH or q observations

Last updated 17th August 2004

Code intended for general use, but only invoked when using the new observations
Review level: 5.3

This document consists of the following sections:

Overview
Changes to use of the code
Modifications to existing code
New code
Details of tests and results

Overview of development

The "ResSat" observation type is to replace the obsolescent "Sat120" observation type. The new type will allow assimilation of vertical profiles of temperature, ozone, and humidity measurements. Such profiles will normally be obtained from satellite retrievals (the Sat120 type could assimmilate only temperature).

There are two main modes of the code (determined by the variable ResSatMode).

Point value mode. This treats the profile as a set of point values, so that the observation operator performs vertical interpolation of the model cx values to the observation location. This option has ResSatMode=1.
Layer averaved mode. This treats each value in the profile as a representation of the average value of that quantity in a vertical layer. The bounds of the layer is specified alongside the profile values and averaging is performed with respect to a 'top-hat' function. This option has ResSatMode=2.

There are two humidity options. Humidity observations arrive from the VarObs file in the form of vertical profiles of relative humidity (RH). Before OPS, the observations were in the form of profiles of specific humididity (q), but, due to the way the OPS works, they must be converted into RH. In this version of the code, Var has two possibilities of dealing with this (determined by the variable ResSatHumidityMode).

Fit to RH. This is the original mode. The fit is made from the Cx+ column to RH. This is non-linear and inefficient. This option has ResSatHumidityMode=1.
Fit to q. As the retrieval was in the form of q, this option can legitimately convert the RH, T to q values (recovering the original values). The fit is then made to q, which is linear and more efficient than the first option. This option has ResSatHumidityMode=2.

More information on the scientific details can be found as a DARC technical report via "Layer Averaging Operators for 3d-Var". Changes to the code have been made, in the first instance, to the Var. Stable 20/1.2 version of the code.

Changes to use of the code

The code will (or at least should) run in its original mode (ie by fitting to RH observations rather than q) if it is run inside a Var job previously used for the old ResSat executable - but with one exception. The namelist flag variable UseResSatRHs should be substituted with UseResSatHumidities. This change has been made for clear use of the code as the flag no longer applies only to RH, but could apply to q (depending upon the state of the variable ResSatHumidityMode). Setting-up a new variable ResSatHumidityMode will allow switching between RH and q as described above. N.B. if no ResSatHumidityMode is set-up, then the default value of ResSatHumidityMode = 1 is used.

Modifications to existing code

The code changes are highlighted in the following table, listed by module. The links show the code with the changes. For purposes of the review, changes are highlighted with !**Ross**, but this comment will be deleted for before lodging. Later changes for the new humidity option (see above) are highlighted with !**Rossq**.

VarMod_ObsInfo (1.84)	.
VarMod_ObsInfo.f90	Added ozone in the Obheader_type (line 185) Added q in the Obheader_type (line 186) Added ozone in the Ob_type (line 246) Added q in the Ob_type (line 246) Added ozone in the ModelOBheader_type (line 293) Added q in the ModelOBheader_type (line 294) Added ozone in the ModelOB_type (line 314) Added q in the ModelOB_type (line 315) Incremented NumObFiles to 21 (line 409) Added "ResSat" to filename list (lines 428-432) Note that the NumObFiles variable has been incremented by more than 1 to correspond to the entries in GenMod_ObsInfo (see below).
Var_YAllocate.f90	Added code for ozone profile (lines 298-311) Added code for q profile (lines 313-326)
Var_YDeAllocate.f90	Added code for ozone profile (lines 185-192) Added code for q profile (lines 194-201)
Var_YPrint.f90	Added code for ozone profile (lines 115-117, 216-222) Added code for q profile (lines 119-121, 224-230)
Var_OBPrint.f90	Added code for ozone profile (lines 128-130, 352-359) Added code for q profile (lines 132-134, 361-368)
Var_ObDeallocate.f90	Added code for ozone profile (lines 162-163) Added code for q profile (lines 165-166)
Var_YSubtract.f90	Added code for ozone profile (lines 89-92) Added code for q profile (lines 94-97)
Var_YScale.f90	Added code for ozone profile (lines 84-85) Added code for q profile (lines 87-88)
Var_YRandom.f90	Added code for ozone profile (lines 66, 134-138) Added code for q profile (lines 65, 140-144)
Var_YDot.f90	Added code for ozone profile (lines 90-92) Added code for q profile (lines 94-96)
Var_YCopy.f90	Added code for ozone profile (lines 369-396) Added code for q profile (lines 398-425)
Var_YAdd.f90	Added code for ozone profile (lines 86-89) Added code for q profile (lines 91-94)

GenMod_ObsInfo (1.29)	.
GenMod_ObsInfo.f90 (Copy of Roger Brugge's code)	Included ObsGroupResSat=21 (line 413) Incremented NumObsGroups=21 (line 414) Other EnviSat changes required for OPS only (lines 508-510)

VarMod_ObsControl (1.111)	.
Var_Obs2PenAndGrad.f90	USE variable ObsGroupResSat from GenMod_ObsInfo (line 77)(see above) INCLUDE Var_ResSatPenAndGrad routine (line 102) (see below) Call to Var_ResSatPenAndGrad routine (in new module VarMod_ResSat (lines 242-247) (see below)
Var_InitObsControl.f90	USE variable ObsGroupResSat from GenMod_ObsInfo (line 110) (see above) USE variable ResSatStats from VarMod_ObsOptions (line 141) (see below) INCLUDE Var_SetupResSatStats routine (line 208) (see below) Make sure that cx % theta doesn't get changed to cx % T for ResSat type (lines 339-349) (old and new dynamics) Include call to Var_SetupResSatStats (lines 524-529) (see below)

VarMod_ObsIO (1.128)	.
Var_ObAndCxRead.f90	USE variable UseResSat from VarMod_ObsOptions (line 132) (see below) USE variable ObsGroupResSat from GenMod_ObsInfo (line 147) (see above) Check that the ResSat file is required (line 272) Check that ResSat has correct obsgroup number (21) (lines 434-438) Special note - DARC only. This routine requires non-standard modifications to fix anomalous ATOVS data. This non-standard code will not be lodged.
Var_ObRead.f90	USE variable ObsGroupResSat from GenMod_ObsInfo (line 131) (see above) USE variable VarField_ozone from OpsMod_Constants (line 208) (see below) Declare new integer variable PntObOzone (line 277) In loop "AllVrbls", increment the Ivar variable to loop to 45 times and add ozone to the loop (lines 378, 468-469) In loop "UairVrbl1", add ozone (lines 661-662) Label ResSat ozone, T, RH observations as representing pressure layers (line 695) Store integer offsets to location of ozone profiles (line 780) Make sure that the pressure data for the ResSat obs type is labelled as pressure layers - as for Sat120 (line 876) Spotted and corrected unrelated error in existing comment ("!out" to "!in") (line 902) Added ozone profile branch to CASE statement (lines 1220-1225) Special note of caution - need to check (line 511 in the above "Ob % Header % ObsGroup = UMhdr % IntC(IC_ObsGroup))" does contain the number ObsGroupResSat. If not either update in OPS or do the quick-fix as lines 490-498 of DARC's previous version. Special note of caution - need to check that MaxNumObVar is large enough. This is set in line 527 of this version of the code, and comes from the UM header. There is a 'quick-fix' in lines 516-519 of DARC's previous version. Special note of caution - need to check that Ob%header%SatID%Status, Ob%header%SolarZenith%Status and Ob%header%Surface%Status have StatusAbsent for the ObsGroupResSat. Failure to do this will result in problems elsewhere (example bug fix in lines 1561-1572 of DARC's previous version.) Special note of caution - need to check that pressure values are read-in from somewhere. Action - remove WRITE statements before lodging Special note - DARC only. This routine requires non-standard additions for reading profile dependent pressures
Var_CxRead.f90	Special note of caution - need to check (line 262 of this file) that Cx%Header%ObsGroup contains the number ObsGroupResSat. If not either update in OPS or do the quick-fix as lines 268-276 of DARC's previous version. Action - remove WRITE statements before lodging

VarMod_ObsOptions (1.80)	.
VarMod_ObsOptions.f90	Added LOGICAL variable "ResSatStats" (line 118) Added LOGICAL variable "OzoneStats" (line 128) Added LOGICAL variable "qStats" (line 129) Added LOGICAL variable "UseResSat" (line 163) Added LOGICAL variables "UseResSatTs", "UseResSatHumidities" and "UseResSatOzones" (lines 187-190) Added INTEGER variable "ResSatMode" (lines 220-223) (flag for switching between simple interpolation (=1), layer averaging (=2) - others planned later). Added INTEGER variable "ResSatHumidityMode" (lines 224-225) (flag for controlling the way that humidity observations are treated, either fit to RH (=1) or convert and fit to q (=2)) Added REAL variables "ResSatTWt", "ResSatRHWt", "ResSatOzoneWt" and "ResSatqWt" (lines 274-277) Added the above variables to the namelist (lines 286, 288, 298, 303, 331, 332, 333)
Var_ReadObservationNL.f90	USE variable "ResSatStats" (line 117) USE variables "UseResStat", "UseResStatTs", "UseResSatHumidities" and "UseResSatOzones" from VarMod_ObsOptions (see above) (line 124) Added "ResSatStats" to CASE statement (line 195) Added code to check ResSat flags (lines 287-294) Added "UseResSat" to list (line 300) Added line for ResSat (line 359) Added line for ResSat (line 377)

OpsMod_Constants (1.28)	.
OpsMod_Constants.f90 (Copy of Roger Brugge's code)	Add "VarField_ozone" and "VarField_plevelsa" (lines 169-170)

GenMod_Utilities (1.19)	.
Gen_SpotTemperature.f90	Correct bugs which may cause problems in the future "999." to "999" (line 129) and "0." to "0" (line 141)

New code

The new code is highlighted in the following table, listed by module. The links show the code with descriptions.

VarMod_ResSat (1.2)	.
VarMod_ResSat.f90	Equivalent of module used elsewhere set-up for the ResSat obs. type (actually adapted from Nesdis)
Var_SetupResSatStats.f90	Equivalent of routine used elsewhere set-up for the ResSat obs. type (actually adapted from Sonde)
Var_ResSatPenAndGrad.f90	Equivalent of routine used elsewhere to call routines to calculate model equivalents of ResSat obs, compute the adjoint with respect to the model obs, the penalty function contribution, and to calculate the adjoint state with respect to model columns. Action - remove WRITE statements before lodging
Var_ResSat2PenAndGrad.f90	Equivalent of routine used elsewhere to calculate ResSat penalty and gradient with respect to model observations (actually adapted from Nesdis)
Var_InitExtraResSatqObs.f90	Set-up observation q structures for conversion of RH (and T, p) to q. Conversion performed in Var_CalcExtraResSatqObs (see below).
Var_CalcExtraResSatqObs.f90	Convert observation RH (and T, p) to q. Arrays are set-up and allocated in Var_InitExtraResSatqObs (see above).
Var_ResSatOperator.f90	Equivalent of routine used elsewhere to calculate model equivalents ob observations (actually adapted from Nesdis)
Var_ResSatOperator_Adj.f90	Equivalent of routine used elsewhere to calculate adjoints with respect to model columns (actually adapted from Nesdis)
Var_ResSatOperator_Test.f90	Equivalent to routines used elsewhere to perform adjoint test for layer averaging operators
Var_LayerAv.f90	Code to perform layer averaging of Cx+ columns to give model observations
Var_LayerAv_Adj.f90	Code to perform adjoint of layer averaging routine
Var_RHLayerAv.f90	Code to perform layer averaging of Cx+ columns of relative humidity to give model observations
Var_RHLayerAv_Adj.f90	Code to perform adjoint of layer averaging of Cx+ columns of relative humidity
Var_InterpOperator.f90	Adaption for use with ResSat from Var_ATOVSOperator (calls routines to perform vertical interpolation)
Var_InterpOperator_Adj.f90	Adaption for use with ResSat from Var_ATOVSOperator_Adj (calls routines to perform adjoint of vertical interpolation)
Var_InterpOperator_Test.f90	Equivalent to routines used elsewhere to perform adjoint test for interpolation operators
Var_VertInterpResSatObs.f90	Adapted from Var_VertInterpObs, allowing for obs level pressure data which is not contained in the Ob data structure
Var_VertInterpResSatObs_Adj.f90	Adapted from Var_VertInterpObs_Adj, allowing for obs level pressure data which is not contained in the Ob data structure

Code tests

(In preparation)
Tests (to be) performed are detailed in this table with links to results

Run	Increments	Stats	stdout	ResSat diagnostic output
Run 1: Run the standard code with all available obs, except ResSat obs	O3: L1, L22, L35, L40 T: L1, L22, L35, L40 RH: L1, L22, L35, L40	Stats	stdout	N/A
Run 2: Run modified code with all available obs, except ResSat obs	O3: L1, L22, L35, L40 T: L1, L22, L35, L40 RH: L1, L22, L35, L40	Stats	stdout	N/A
Run 3: Run modified code with all available obs, plus ResSat O3, T, RH obs, but with ResSat obs switched off	O3: L1, L22, L35, L40 T: L1, L22, L35, L40 RH: L1, L22, L35, L40	Stats	stdout	N/A
Run 4: Run modified code with all available obs, plus ResSat O3, T, RH obs, but with only ResSat O3 switched on	O3: L1, L22, L35, L40 T: L1, L22, L35, L40 RH: L1, L22, L35, L40	Stats	stdout	Diagnostic
Run 5: Run modified code with all available obs, plus ResSat O3 obs	O3: L1, L22, L35, L40 T: L1, L22, L35, L40 RH: L1, L22, L35, L40	Stats	stdout	Diagnostic
Run 6: Run modified code with all available obs, plus ResSat O3, T, RH obs, but with only ResSat T switched on	O3: L1, L22, L35, L40 T: L1, L22, L35, L40 RH: L1, L22, L35, L40	Stats	stdout	Diagnostic
Run 7: Run modified code with all available obs, plus ResSat T obs	O3: L1, L22, L35, L40 T: L1, L22, L35, L40 RH: L1, L22, L35, L40	Stats	stdout	Diagnostic
Run 8: Run modified code with all available obs, plus ResSat O3, T, RH obs, but with only ResSat RH switched on	O3: L1, L22, L35, L40 T: L1, L22, L35, L40 RH: L1, L22, L35, L40	Stats	stdout	Diagnostic
Run 9: Run modified code with all available obs, plus ResSat RH obs	O3: L1, L22, L35, L40 T: L1, L22, L35, L40 RH: L1, L22, L35, L40	Stats	stdout	Diagnostic
Run 10: Run modified code with all available obs, plus ResSat O3, T, RH obs with all three switched on	O3: L1, L22, L35, L40 T: L1, L22, L35, L40 RH: L1, L22, L35, L40	Stats	stdout	Diagnostic