Table 1 Symbol tokens included in the tokenizer.
From: CHEMDNER system with mixed conditional random fields and multi-scale word clustering
|
~ • ! @ # $ % ^ & * - = _ + ˉ ( ) [ ] { } ; ' : " , . / < > ×> < ≤ ≥ ↑ ↓ ← → • ′ ° ~ ≈ ? Δ ÷ ≠ | ‘ ’ “ ” §£ € \ 0 1 2 3 4 5 6 7 8 |