Evaluating the Performance of Large Language Models (LLMs): a Comparative Analysis of LLAMA and BERT in News Article Summarization and Sentiment Analysis of Social Media Posts.
Φόρτωση...
Ημερομηνία
2025-01-17
Συγγραφείς
Gorokhov, Sergei
Τίτλος Εφημερίδας
Περιοδικό ISSN
Τίτλος τόμου
Εκδότης
Παραπομπή
Παραπομπή
Άδεια Creative Commons
Εκτός εάν σημειώνεται διαφορετικά, η άδεια αυτού του αντικειμένου περιγράφεται ως Attribution 4.0 International
Περίληψη
Περίληψη
ABSTRACT
This dissertation was written as a part of the MSc in Data Science at the International Hellenic University. The research study titled “A Comparative Analysis of LLaMA and BERT in News Article Summarisation and Sentiment Analysis of Social Media Posts” seeks to evaluate and compare the effectiveness of LLaMA 2 7B and BERT base in two different applications of NLP. The two primary goals were to establish how well they performed for various types of textual data and offer guidelines on the best ways to harness them in practical applications. The study utilised two datasets: The dataset used for summarisation is a news articles dataset and the second set is the TweetEval dataset for sentiment analysis. Hence, the results of both models were assessed on accuracy, resource uses, and time required for training. The outcomes established showed that BERT scored better on both tasks than LLaMA with a validation accuracy of 94 % for news summarisation and 64 % for sentiment analysis in contrast to LLaMA which scored only 79% and 54% respectively. Lacking LLaMA’s additional
complexity did not penalize BERT for not attaining the same level of performance as the original tasks. The analysis pointed out the need to select which model to use for each task, while it emerged that BERT was more appropriate for classifying tasks that involve contextual information. Recommendations for BERT are to adapt BERT models to fit into your specific industry or niche and for LLaMA, it is advised for the more intricate, generative tasks.
I would like to express my gratitude to my supervisor for their valuable insights and guidance throughout the project. I want to say special thanks to my collaborators and colleagues for their constructive feedback and support, which enriched this work making it easier to complete.