Skip to main content

Table 7 Global leaveout analysis for additional Category 2 entries—including only challenges where the correct answer was not in any training set

From: Critical Assessment of Small Molecule Identification 2016: automated methods

  Allen Brouard Dührkop Ruttkies Vaniya Verdegem
  CFM_orig CFM_re-train CSI_IOKR_A CSI_IOKR_AR* CSI_IOKR_U CSI:FID_orig CSI:FID_leaveout* MetFrag* MetFrag+CFM* MS−FINDER MAGMa+ MAGMa*
Top 1 Neg. 6 6 6 6 4 0 0 4 10 7 4 3
Top 1 Pos. 4 9 9 10 7 9 13 1 3 3 2 2
Top 1 10 15 15 16 11 9 13 5 13 10 6 5
Top 3 23 24 29 26 27 17 23 16 27 25 16 14
Top 10 46 40 45 46 40 25 32 39 47 38 35 35
Mean rank 52.57 64.05 106.5 97.84 99.92 52.81 41.48 68.38 37.16 28.7 76.75 100.4
Med. rank 10 12.5 8 10 12 7 3 14.5 8 7.5 23.5 20.5
Mean RRP 0.863 0.872 0.849 0.856 0.837 0.891 0.91 0.863 0.878 0.738 0.832 0.811
Med. RRP 0.966 0.961 0.963 0.967 0.956 0.981 0.993 0.942 0.972 0.806 0.924 0.902
Gold 18 21 23 26 19 11 17 7 18 18 10 9
F1 score 628 654 735 691 632 403 557 484 707 594 462 434
Medal Sc. 79 94 105 98 91 59 87 50 95 85 46 46
  1. n = 43 (negative) and n = 44 (positive)
  2. The best value for selected statistics is marked in bold