International Journal For Multidisciplinary Research

E-ISSN: 2582-2160     Impact Factor: 9.24

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

Call for Paper Volume 6 Issue 4 July-August 2024 Submit your research before last 3 days of August to publish your research paper in the issue of July-August.

Pattern Finding In Log Data Using Hive on Hadoop

Author(s) Swapna Sahu
Abstract Web log file, in the computing context, is the log file which get routinely generated and maintained by a web server. Analysing web server access logs will give information regarding user’s behavior. Log files generate data which contain valuable information from the user which get stored in the web server. Server logs act as a guest sign-in sheet. Log files give information about the pages which had a heavy traffic and least. What sites refer visitors to your site? What pages that your visitors view? Because of the tremendous usage of web, the web log files are growing at faster rate and the size is becoming huge. Processing this explosive growth of log files using relational database technology has been facing a bottle neck. To analyse such large datasets we need parallel processing system and reliable data storage mechanism, Big data uses the Hadoop where massive quantity of information is processed using cluster of commodity hardware. In this paper we present the Hadoop framework for storing and processing large log files and also analysing through hive, Hive is used in pre-processing of voluminous of log files and help us to find out the statics present in website and which help in our learning too.We can also perform optimization on hive query and we also compare the performance of both the analytical tools on analysing log files.
Keywords Hadoop, data mining, log file analysis, behaviour mining, web mining
Field Computer > Data / Information
Published In Volume 1, Issue 2, September-October 2019
Published On 2019-10-01
Cite This Pattern Finding In Log Data Using Hive on Hadoop - Swapna Sahu - IJFMR Volume 1, Issue 2, September-October 2019.

Share this