Transformer-based Molecular Classification Models for Blood Brain Barrier Penetration Prediction
Project Overview
The blood-brain barrier is a specialized structure in the central nervous system that separates the circulating blood from the brain tissue. The blood-brain barrier is essential for maintaining the brain's microenvironment and protecting it from potentially harmful substances in the blood.
However, it presents a significant challenge for drug delivery to the brain, as many drugs cannot penetrate this barrier. The degree of penetration depends on various factors, including the size, charge, and lipophilicity of the molecule. Some drugs are designed to have properties that allow them to penetrate the blood-brain barrier, while others require modifications or special delivery methods to achieve this.
In this project, we will develop a classifier that will predict whether a molecule can be used for blood-brain barrier medicine or not.
What were the key results of your research project?
We evaluated a range of approaches, including classical machine learning (ML) models, transformer-based approaches and large language models for their ability to predict blood-brain barrier permeability across three datasets (DeepChem BBBP, PubChem, MaskAtom).
Classical ML methods (Logical regression, randomforest) paired with transformer-based feature embeddings outperformed more complex models. Performance varied dataset to dataset which highlights the importance of having a large, varied and well-balanced set of molecular compositions when training models for BBBP.
GitHub repository: https://github.com/HamzahP81/bbbp-transformer
How do you feel you have benefited from completing this internship and has it made you consider future career paths?
I've gained valuable hands-on experience in running and experimenting with state-of-the-art machine learning models. In addition to the technical skills gained, I've had the opportunity to collaborate with PhD students, offering an opportunity to engage in a research-oriented, team-based environment. This collaborative setup has not only enhanced my understanding of applied machine learning workflows, but also helped me develop essential skills in communication, project coordination, and peer learning within a research setting. I'm excited to leverage this experience in more research-focused roles in my career.
Download presentation slides: