Fig. 1

Overview of model design and BERT-based classifier architecture. a The figure illustrates the step-by-step process of the model design pipeline. It involves a two-phase approach starting with pretraining, followed by fine-tuning on downstream tasks. b The overall architecture of the BERT classifier. The Bert model with multiple stacks of self-attention layers, each of which comprises several attention heads. The model was utilized for a chemical reaction classification task through a classifier head. Additionally, the encoding of the [CLS] token serves as the reaction learnable fingerprint