Comparative Performance Analysis of Large Language Models in Generative Business Intelligence: Insights from Llama3 and BambooLLM

Author(s)	Bhavesh Jaisinghani, Saurabh Aggarwal
Country	United States
Abstract	This study addresses the critical gap in Large Language Model (LLM) evaluation for business intelligence by conducting a rigorous comparative analysis of Llama3-70b-8192 and BambooLLM across five key data analysis tasks. Utilizing the AdventureWorks Cycle dataset, we developed a comprehensive evaluation framework measuring task efficiency, weighted accuracy, and misinterpretation rates. Results demonstrate that Llama3-70b-8192 outperforms BambooLLM with a 40% lower misinterpretation rate and 25% higher task efficiency across structured and interpretive business intelligence challenges. This study highlights the potential for optimizing fine-tuning strategies for task items that combine structured and interpretive elements, offering valuable insights for optimizing fine-tuning strategies and informing future research directions in LLM evaluation for business intelligence applications.
Keywords	Large Language Models, Business Intelligence, Generative AI, Data Analysis, Data Analytics, Model Performance Evaluation, Llama3, BambooLLM, Predictive Analytics, Machine Learning, Artificial Intelligence in Business, AI
Field	Computer > Artificial Intelligence / Simulation / Virtual Reality
Published In	Volume 6, Issue 6, November-December 2024
Published On	2024-12-28
DOI	https://doi.org/10.36948/ijfmr.2024.v06i06.33967
Short DOI	https://doi.org/g82ghd

About IJFMR Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Reviewer Referral Program Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us		+91-9687-828-838	editor@ijfmr.com

International Journal For Multidisciplinary Research