Table 16 Summary of ChER's performance under the CHEMDNER track setting (set 1), under similar experimental settings as state-of-the-art methods (sets 2-4), and when applied to various corpora (sets 5-9).
Data | Pre-processing | Cust. | Post-processing | Micro-averages | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
Training | Test | Splitter | Tokeniser | Feats. | Abbr. | Comp. | P | R | F1 | |
1 | CHEMDNER | CHEMDNER | LingPipe | GENIA | ✗ | ✗ | ✗ | 88.87 | 70.95 | 78.91 |
training & dev. | test | Cafetiere | OSCAR4 | ✓ | ✓ | ✓ | 92.76 | 81.30 | 86.65 | |
2 | SciBorg (CM):3-fold CV | LingPipe | GENIA | ✗ | ✗ | ✗ | 80.44 | 55.16 | 65.45 | |
Cafetiere | OSCAR4 | ✓ | ✓ | ✓ | 85.96 | 74.22 | 79.66 | |||
3 | SCAI-IUPAC | SCAI-100 | LingPipe | GENIA | ✗ | ✗ | ✗ | 84.78 | 66.87 | 74.77 |
training | (IUPAC) | Cafetiere | GENIA | ✓ | ✓ | ✓ | 86.70 | 67.50 | 75.90 | |
4 | NaCTeM Metabolites:10-fold CV | LingPipe | GENIA | ✗ | ✗ | ✗ | 81.72 | 64.49 | 72.09 | |
Cafetiere | OSCAR4 | ✓ | ✓ | ✓ | 81.42 | 79.66 | 80.53 | |||
5 | CHEMDNER | SCAI-100 | LingPipe | GENIA | ✗ | ✗ | ✗ | 72.56 | 66.00 | 69.13 |
training & dev. | (All) | Cafetiere | OSCAR4 | ✓ | ✓ | ✓ | 77.85 | 78.69 | 78.27 | |
6 | CHEMDNER | Patents | LingPipe | GENIA | ✗ | ✗ | ✗ | 72.66 | 52.97 | 61.27 |
training & dev. | Cafetiere | OSCAR4 | ✓ | ✓ | ✓ | 73.43 | 57.91 | 64.75 | ||
7 | CHEMDNER | DDI | LingPipe | GENIA | ✗ | ✗ | ✗ | 76.52 | 75.00 | 75.75 |
training & dev. | test | Cafetiere | OSCAR4 | ✓ | • | ✓ | 75.88 | 92.05 | 83.18 | |
8 | CHEMDNER | PK | LingPipe | GENIA | ✗ | ✗ | ✗ | 79.29 | 84.66 | 81.89 |
training & dev. | Cafetiere | GENIA | ✓ | ✓ | ✓ | 79.83 | 88.34 | 83.87 | ||
9 | CHEMDNER | NaCTeM | LingPipe | GENIA | ✗ | ✗ | ✗ | 63.57 | 71.63 | 67.36 |
training & dev. | Metabolites | Cafetiere | OSCAR4 | ✓ | ✓ | ✓ | 65.08 | 83.29 | 73.07 | |