Table 2 Performance comparison of generators using the MOSES Benchmark
From: Probabilistic generative transformer language models for generative design of molecules
GMT | MOSES reference models | ||||||||
|---|---|---|---|---|---|---|---|---|---|
GMT- SMILES | GMT-PE- SMILES | GMT- SELFIES | GCT -SGDR | VAE | AAE | char RNN | |||
Validity | \(\uparrow\) | 0.8587 | 0.8288 | 1.000 | 0.9916 | 0.9767± 0.0012 | 0.9368± 0.0341 | 0.9748± 0.0264 | |
Unique@1k | \(\uparrow\) | 1.0000 | 1.0000 | 1.0000 | 0.998 | 1.0±0.0 | 1.0±0.0 | 1.0±0.0 | |
Unique@10k | \(\uparrow\) | 0.9998 | 0.9995 | 1.0000 | 0.9797 | 0.9984± 0.0005 | 0.9973± 0.002 | 0.9994 ± 0.0003 | |
IntDiv | \(\uparrow\) | 0.8569 | 0.8558 | 0.8701 | 0.8458 | 0.8558± 0.0004 | 0.8557± 0.0031 | 0.8562 ± 0.0005 | |
Filters | \(\uparrow\) | 0.9766 | 0.9797 | 0.7961 | 0.9982 | 0.6949± 0.0069 | 0.9960± 0.0006 | 0.9943 ± 0.0034 | |
Novelty | \(\uparrow\) | 0.9531 | 0.8829 | 0.9683 | 0.6756 | 0.6949± 0.0069 | 0.7931± 0.0285 | 0.8419 ± 0.0509 | |
Test | 0.5381 | 0.5778 | 0.4673 | 0.6513 | 0.6257± 0.0005 | 0.6081± 0.0043 | 0.6015 ± 0.0206 | ||
SNN | \(\uparrow\) | TestSF | 0.5143 | 0.5460 | 0.4485 | 0.5990 | 0.5783± 0.0008 | 0.5677± 0.0045 | 0.5649 ± 0.0142 |
Test | 0.7294 | 0.1986 | 3.7750 | 0.7980 | 0.0990± 0.0125 | 0.5555± 0.2033 | 0.0732 ± 0.0247 | ||
FCD | \(\downarrow\) | TestSF | 1.2607 | 0.7595 | 4.5698 | 0.9949 | 0.5670± 0.0338 | 1.0572± 0.2375 | 0.5204 ± 0.0379 |
Test | 0.9879 | 0.9982 | 0.9869 | 0.9922 | 0.9994± 0.0001 | 0.9910± 0.0051 | 0.9998 ± 0.0002 | ||
Frag | \(\uparrow\) | TestSF | 0.9850 | 0.9958 | 0.9831 | 0.8562 | 0.9984± 0.0003 | 0.9905± 0.0039 | 0.9983 ± 0.0003 |
Test | 0.8661 | 0.9125 | 0.8431 | 0.8562 | 0.9386± 0.0021 | 0.9022± 0.0375 | 0.9242 ± 0.0058 | ||
Scaf | \(\uparrow\) | TestSF | 0.1650 | 0.1087 | 0.1096 | 0.0551 | 0.0588± 0.0095 | 0.0789± 0.009 | 0.1101 ± 0.0081 |