Research Project: ArteFact - Contextualising paintings with the art history corpus
Project Overview
I developed the app ArteFact which is a tool for searching through the art history corpus using images. For any painting, a user can see which sentences an AI vision language model most associates with that painting. The app’s features include: image cropping; saliency map generation; immediate viewing on sentences in the context from where they were extracted; instant downloading of the original work; and citation information to properly acknowledge the author of any sentence served.
What were the key results of your research project?
CLIP is able to successfully associate sentences about paintings with the paintings themselves. The key result is actually functional: the ArteFact app itself, available at samwaugh/ArteFact on Hugging Face.
How do you feel you have benefitted from completing this internship and has it made you consider future career paths?
This internship has improved my confidence writing software tools for research applications. It has given me lots of inspiration about how much software can be produced with the help of AI in a short amount of time.
GitHub Repository: https://github.com/sammwaughh/ArteFact