Comparing Coriolis data with Jim Cumming's Argo data

Time profiles for 2010

Possible juld_qc values for 2010

coriolis_juld_qc	-999 (11,358), 0 (6,720), 1 (97,047) and 4 (1,448)
coriolis_jim_juld_qc	-999 (12,247), 0 (6,800), 1 (95,980) and 4 (1,546)

Relationships

• 8,756 times when coriolis_juld_qc = -999 and coriolis_jim_juld_qc = -999
  • 2,602 times when coriolis_juld_qc = -999 and coriolis_jim_juld_qc != -999. coriolis_jim_juld_qc is
    • 0 - 125 times
    • 1 - 2,416 times
    • 4 - 61 times
  • 3,491 times when coriolis_juld_qc != -999 and coriolis_jim_juld_qc = -999. coriolis_juld_qc is
    • 0 - 71 times
    • 1 - 3,406 times
    • 4 - 14 times
• coriolis_juld_qc = 0 <=> coriolis_jim_juld_qc = 0 (6,649 times)
• coriolis_juld_qc = 1 <=> coriolis_jim_juld_qc = 1 (93,563 times)
• coriolis_juld_qc = 4 <=> coriolis_jim_juld_qc = 4 (1,433 times)
• coriolis_juld_qc = -999 <=> coriolis_pos_qc = -999 (no exceptions)
• coriolis_jim_juld_qc = -999 <=> coriolis_jim_pos_qc = -999 (no exceptions)

Position profiles for 2010

Possible pos_qc values for 2010

coriolis_pos_qc	-999 (11,358), 0 (43), 1 (105,021), 3 (69), 4 (63), 5 (7) and 8 (12)
coriolis_jim_pos_qc	-999 (12,247), 0 (43), 1 (104,127), 3 (73), 4 (61), 5 (6) and 8 (16)

Relationships

• 8,756 times when coriolis_pos_qc = -999 and coriolis_jim_pos_qc = -999
• coriolis_pos_qc = 0 <=> coriolis_jim_pos_qc = 0 (no exceptions)
• coriolis_pos_qc = 1 <=> coriolis_jim_pos_qc = 1 (101,546 times)
• coriolis_pos_qc = 3 <=> coriolis_jim_pos_qc = 3 (69 times)
• coriolis_pos_qc = 4 <=> coriolis_jim_pos_qc = 4 (47 times)
• coriolis_pos_qc = 5 <=> coriolis_jim_pos_qc = 5 (6 times)
• coriolis_pos_qc = 8 <=> coriolis_jim_pos_qc = 8 (11 times)

Temperature profiles for 2010

Possible tmp_qc values.

coriolis_tmp_qc value	-999 (11,365), 0/A (94,013), 1/B (7,246), 2/C (526), 3/D (171), 4/E (90), 5/F (3,162)
bom_jim_tmp_qc	-999 (12,247), 10 (93,276), 11 (7,555), 12 (554), 13 (173), 14 (90), 15 (2,678)

Relationships

• 8,758 times when coriolis_tmp_qc = -999 and coriolis_jim_tmp_qc = -999
• coriolis_tmp_qc = 0/A <=> coriolis_jim_tmp_qc = 10 (91,152)
• coriolis_tmp_qc = 1/B <=> coriolis_jim_tmp_qc = 11 (7,055)
• coriolis_tmp_qc = 2/C <=> coriolis_jim_tmp_qc = 12 (519)
• coriolis_tmp_qc = 3/D <=> coriolis_jim_tmp_qc = 13 (165)
• coriolis_tmp_qc = 4/E <=> coriolis_jim_tmp_qc = 14 (88)
• coriolis_tmp_qc = 5/F <=> coriolis_jim_tmp_qc = 15 (2,613)

Summary

Generally when coriolis_pos_qc != -999, coriolis_jim_pos_qc = coriolis_pos_qc + 10.

Locating the missing data

Judging by the matches for the -999 numbers, there seems to be a lot of data which Jim has which we don't and vice versa.

When we have data that Jim doesn't

When coriolis_juld_qc != -999, but coriolis_jim_juld_qc = -999. The first entries are:

 argoid  |     juld     
---------+--------------
 3900236 | 22002.504792    (record 8 is found in PF file)
 3900254 | 22003.089468    (record 8 is found in PF file)
 3900258 | 22061.302512    (record 13 is found only in TE file)
 3900261 | 21997.278333    (record 7 is found only in TE file)
 3900262 | 22060.508808
 3900271 | 22001.957025
 3900273 | 22001.974016
 3900273 | 22011.445602
 3900292 | 22218.665035
 3900292 | 22228.191343
 3900292 |  22237.66875
 3900292 | 22247.184282
 3900292 | 22256.685637
 3900292 | 22266.195648
 3900292 | 22275.725949
 3900334 | 22002.233669
 3900341 |  22001.83169
 3900349 | 21974.594433
 3900350 | 21974.603854
 3900402 |  22001.34309
 3900411 | 22003.171296
 3900424 | 22002.227685
 3900424 |  22059.87037
 3900425 | 22032.057569
 3900440 |  22001.51684
 3900449 | 22003.071296
 3900451 | 22003.091285
 3900453 | 22003.101181
 3900456 | 22003.026181
 3900457 | 21974.407454
 3900457 | 22003.265845
 3900464 | 22001.799676
 3900484 |    22001.515
 3900486 | 22001.513414
 3900495 | 22001.513079
 3900496 | 22001.501377
 3900509 | 22159.171134
 3900510 | 22137.198194    (record 21 is found only in TE file)
 3900514 | 22064.255914    (record 14 is found only in TE file)
 3900529 | 22003.127778    (record 8 is found in PF file)
 3900537 |  22210.32485
 3900537 | 22220.035787
 3900537 | 22230.316354
 3900537 |  22250.30794

Some of Jim's missing data could be explained by his code only reading the CO_*_PR_PF_P.nc files and not the CO_*_PR_TE_P.nc files, which can fill the gaps in the CO_*_PR_PF_P.nc files, such as 3900510 (irec=21) and 3900514 (irec=14).

When Jim has data but that we don't

When coriolis_juld_qc = -999, but coriolis_jim_juld_qc != -999. The first entries are:

 argoid  |     juld     
---------+--------------
 3900510 | 21974.531713    
 3900512 | 21974.975891
 3900521 |  22230.43169
 3900522 | 22264.416053
 3900523 | 21928.400718
 3900523 | 22088.442361
 3900523 | 22233.408646
 3900523 | 22248.408727
 3900523 | 22253.397141
 3900523 | 22273.407465
 3900531 | 22250.122917
 3900537 | 21930.014525
 3900568 | 22034.527986
 3900592 | 21969.476435
 3900625 | 21972.867211
 3900626 | 21974.534803
 3900640 | 21970.208611
 3900646 | 21970.075671
 3900662 | 21974.662361
 3900667 | 21974.677141
 3900682 | 21974.466678
 3900691 |   21974.4914
 3900705 | 21969.484988
 3900727 | 22057.350463
 3900727 | 22067.349745
 3900728 | 22058.396759
 3900728 | 22068.375394
 3900729 | 22062.352824
 3900729 | 22072.373056
 3900732 | 21974.546898
 3900733 | 21974.528877
 3900734 | 21974.564583
 3900748 | 21969.481713
 3900751 | 21969.479225
 3900752 | 21977.546366
 3900752 | 21987.545174
 3900753 | 21975.559178
 3900753 | 21985.557176
 3900754 | 21979.563877
 3900755 | 21988.568333
 3900756 | 21975.559086
 3900756 | 21985.555637
 3900757 | 21980.558125
 3900760 | 22067.465428

I can't work where Jim get this extra data from?

Temperature levels for 2010 for 4900* argo data

Possible tmp_qc_lvl values for 4900* argo data.

coriolis_tmp_qc_lvl	-999 (3,449,744), 0 (118,804), 1 (156,954) and 4 (1,450)
coriolis_jim_tmp_qc_lvl	-999 (3,097,008), 1 (361,603) and 4 (3,578)

This doesn't look like a healthy comparison.

ETS score for temperature profiles for 2010

Taking 0 to be good and 5 to be bad

		coriolis
		Reject	Accept	Total
d_mode	Reject	2508 (2.667%)	2634 (2.801%)	5142 (5.468%)
	Accept	515 (0.548%)	88380 (93.984%)	88895 (94.532%)
	Total	3023 (3.215%)	91014 (96.785%)	94037 (100%)

ETS: 0.426589272082501
bias: 1.70095931194178