What do Large Language Models actually know?
Why did you apply for this internship?
I’ve always loved the idea of consulting in the tech space and wanted to gain some hands-on experience. While searching for summer internships, I saw plenty of opportunities in software engineering or consulting, but none that combined both and also incorporated my love for learning.
When I came across the listing for this internship, it felt like the perfect fit. The chance to learn software engineering practices, gain practical experience across a wide breadth of topics, and consult while interacting with people from all walks of life really appealed to me and motivated me to apply.
What did you hope to gain in completing this project?
I hoped to finish the internship with a deeper understanding of software engineering, meaningful connections, and stronger technical and soft skills.
I recognise the interdisciplinary nature of N8 CIR, and by completing this project, I knew it would push me out of my comfort zone, allowing me to learn through practice while building on my knowledge through interactions with a wide range of individuals.
Project Overview
This research project explores the idea of explainability in AI, looking into what makes a large language model (LLM) choose its outcomes, using Explainable AI (XAI) concepts. More specifically, through the analysis of text simplification and comparing tools: Captum AI and Circuit-Tracer libraries.
What were the key results of your research project?
When identifying areas of divergence, it was clear that this stemmed mainly from content word disagreement and, less importantly, punctuation. Thus, it provides invaluable information on the distinction between influence and importance. We can see that there is no significant correlation between per-token attribution and per-token confidence probability in relation to the resulting tokens. With this, we can infer that in one situation, the context surrounding the final answer from the model can be impacted by different tokens in comparison to a single token's overall importance in the final answer.
GitHub repository: https://github.com/jenellebankas/HPC_repo
How do you feel you have benefited from completing this internship and has it made you consider future career paths?
It has made me more confident to explore areas of interest that I do not know much about, and has allowed me to be more confident in my programming abilities. Ultimately, this internship has provided another insight into a career that I had not heard of beforehand, providing another potential path for the future.
Download presentation slides: