The Multitask Effect in Molecular Deep Learning

Multitask neural network models share parameters in order to make predictions jointly over multiple tasks in a dataset. Previous investigations of multitask neural networks have shown improved accuracy in comparison with singletask neural network models, which is known as the multitask effect. In prior work on molecular property prediction with multitask neural networks, it is difficult to determine whether the improvements to the accuracy of predictions are due to the model hyperparameter settings or from the multitask effect. In this work, the multitask effect in graph neural networks for molecular property predictions tasks was investigated by carefully separating the improvements in the model performance due to hyperparameter settings. Bayesian hyperparameter optimization was used to optimize the singletask and multitask machine learning models separately. When training multitask networks with molecular property prediction benchmark datasets, it was found that the multitask effect can be positive or negative. We decided to take a closer look at how multitask learning is impacted by self-supervised pretraining from learning representations and show that the positive and negative multitask effects can be reduced as long as the pretraining dataset is large enough.