Table 3 Comparison of prediction and uncertainty quantification performance on out-of-sample splits of Buchwald-Hartwig dataset
From: Uncertainty-aware prediction of chemical reaction yields with graph neural networks
Out-of-sample split | Measure | YieldBERT | YieldBERT-DA | Proposed (\(\lambda = 0.1\)) |
|---|---|---|---|---|
Test 1 | MAE (%p) | 7.351 ± 0.099 | \(\mathbf{7.015} \pm \mathbf{0.758}\) | 8.082 ± 0.827 |
RMSE (%p) | \(\mathbf{11.441} \pm \mathbf{0.342}\) | 11.761 ± 1.398 | 13.746 ± 1.175 | |
R\(^2\) | \(\mathbf{0.824} \pm \mathbf{0.010}\) | 0.811 ± 0.047 | 0.744 ± 0.042 | |
Spearman \(\rho\) | – | 0.380 ± 0.065 | \(\mathbf{0.454} \pm \mathbf{0.046}\) | |
Test 2 | MAE (%p) | 7.266 ± 0.724 | 6.588 ± 0.328 | \(\mathbf{6.300} \pm \mathbf{0.647}\) |
RMSE (%p) | 11.144 ± 1.267 | 9.886 ± 0.741 | \(\mathbf{9.476} \pm \mathbf{1.027}\) | |
R\(^2\) | 0.829 ± 0.037 | 0.866 ± 0.020 | \(\mathbf{0.876} \pm \mathbf{0.026}\) | |
Spearman \(\rho\) | – | \(\mathbf{0.494} \pm \mathbf{0.044}\) | 0.397 ± 0.043 | |
Test 3 | MAE (%p) | 9.129 ± 0.745 | 11.052 ± 0.950 | \(\mathbf{8.986} \pm \mathbf{0.314}\) |
RMSE (%p) | \(\mathbf{14.276} \pm \mathbf{0.820}\) | 18.041 ± 1.395 | 14.939 ± 0.622 | |
R\(^2\) | \(\mathbf{0.741} \pm \mathbf{0.030}\) | 0.585 ± 0.067 | 0.717 ± 0.024 | |
Spearman \(\rho\) | – | 0.406 ± 0.065 | \(\mathbf{0.423} \pm \mathbf{0.031}\) | |
Test 4 | MAE (%p) | 13.671 ± 1.067 | 18.422 ± 0.620 | \(\mathbf{13.190} \pm \mathbf{0.754}\) |
RMSE (%p) | 19.679 ± 1.397 | 24.279 ± 0.494 | \(\mathbf{18.774} \pm \mathbf{0.566}\) | |
R\(^2\) | 0.444 ± 0.077 | 0.157 ± 0.034 | \(\mathbf{0.496} \pm \mathbf{0.031}\) | |
Spearman \(\rho\) | – | 0.366 ± 0.100 | \(\mathbf{0.461} \pm \mathbf{0.040}\) |