Data Pre-processing and Generation for Real-World Neural Architecture Search Applications
Project Overview
Investigating the use of generative AI models to create synthetic data of weather conditions that pre-existing datasets don't represent fully. Methods included diffusion methods, CycleGAN and text-to-image generation. Found sufficient data to train these models and use for reconstructing the original images. Testing the images viability using ResNet models that were independently trained to test if these methods were useful to scale up to a larger extent.
What were the key results of your research project?
- ResNet18 weather classifier: achieved 86.5% test accuracy on the unseen BDD100K clear/rainy test set. This gave a quantitative way of evaluating the synthetic images.
- CycleGAN produced the strongest quantitative results, 90.5% were classified as the intended weather. For the clear to rainy set, mean rain score increased from 0.1815 to 0.8663, an increase of 0.6847.
- InstructPix2Pix successfully changed weather while preserving existing scenes reasonably well, 69% of the test set was classified as rainy, with the mean rain score increasing from 0.1410 to 0.6648 (0.5238 mean increase). However, it struggled when asked to make much larger changes like changing location.
- Text-to-image generation created entirely new road scenes, 70% were classified as rainy, with a mean rain score of 0.6446. It was considerably faster at roughly 16 seconds per image, but urban infrastructure could become distorted.
How do you feel you have benefitted from completing this internship and has it made you consider future career paths?
I feel like completing this internship has opened my eyes to the opportunities available in data science and machine learning. Before completing the programme I was mainly focussed on finance roles. The 8 week project has been extremely informative and very enjoyable for me and I would like to pursue a career where I could have a similar experience. I feel as though my coding skills have become much more streamlined and my confidence in my research methods is much higher now than it has ever been.