Skip to main content

Table 3 Comparison of prediction and uncertainty quantification performance on out-of-sample splits of Buchwald-Hartwig dataset

From: Uncertainty-aware prediction of chemical reaction yields with graph neural networks

Out-of-sample split

Measure

YieldBERT

YieldBERT-DA

Proposed (\(\lambda = 0.1\))

Test 1

MAE (%p)

7.351 ± 0.099

\(\mathbf{7.015} \pm \mathbf{0.758}\)

8.082 ± 0.827

RMSE (%p)

\(\mathbf{11.441} \pm \mathbf{0.342}\)

11.761 ± 1.398

13.746 ± 1.175

R\(^2\)

\(\mathbf{0.824} \pm \mathbf{0.010}\)

0.811 ± 0.047

0.744 ± 0.042

Spearman \(\rho\)

0.380 ± 0.065

\(\mathbf{0.454} \pm \mathbf{0.046}\)

Test 2

MAE (%p)

7.266 ± 0.724

6.588 ± 0.328

\(\mathbf{6.300} \pm \mathbf{0.647}\)

RMSE (%p)

11.144 ± 1.267

9.886 ± 0.741

\(\mathbf{9.476} \pm \mathbf{1.027}\)

R\(^2\)

0.829 ± 0.037

0.866 ± 0.020

\(\mathbf{0.876} \pm \mathbf{0.026}\)

Spearman \(\rho\)

\(\mathbf{0.494} \pm \mathbf{0.044}\)

0.397 ± 0.043

Test 3

MAE (%p)

9.129 ± 0.745

11.052 ± 0.950

\(\mathbf{8.986} \pm \mathbf{0.314}\)

RMSE (%p)

\(\mathbf{14.276} \pm \mathbf{0.820}\)

18.041 ± 1.395

14.939 ± 0.622

R\(^2\)

\(\mathbf{0.741} \pm \mathbf{0.030}\)

0.585 ± 0.067

0.717 ± 0.024

Spearman \(\rho\)

0.406 ± 0.065

\(\mathbf{0.423} \pm \mathbf{0.031}\)

Test 4

MAE (%p)

13.671 ± 1.067

18.422 ± 0.620

\(\mathbf{13.190} \pm \mathbf{0.754}\)

RMSE (%p)

19.679 ± 1.397

24.279 ± 0.494

\(\mathbf{18.774} \pm \mathbf{0.566}\)

R\(^2\)

0.444 ± 0.077

0.157 ± 0.034

\(\mathbf{0.496} \pm \mathbf{0.031}\)

Spearman \(\rho\)

0.366 ± 0.100

\(\mathbf{0.461} \pm \mathbf{0.040}\)