Discovering Web Server Logs Patterns Using Clustering and Association Rules Mining

Abstract

Recording server log data files are nowadays a commonplace practice. The server log data files capture useful information during the interaction of users with the online site, as well as the interaction among users during online sessions. The discovered patterns are used for improving web site organization and behavior. Clustering is a discovery process in data mining, which groups set of data items, in such a way that maximizes the similarity within clusters and minimizes the similarity between two different clusters. Various forms of clustering are used in a wide range of applications. In this paper, the server log files of the Website “ www.sust.edu” is considered for overall study and analysis. This paper presents the discovering patterns of Web Usage Mining (WUM) using Clustering and Association Rule from web log data. In the first stage, the data pre-processing phase was performed, and in the second stage k-means and density-based clustering algorithms are used for clustering the log file into groups. Then these clusters are plotted on a plane using WEKA tool, where different clusters are distinguished by distinct colors and distinct symbols. Performance and accuracy of the clustering algorithms are presented and compared. The results of the experiment show that clustering with feature selection gives better Performance. In the third stage apriori algorithm is used to discover relationship among data. Eliminating redundant rules and clustering decreased the size of the generated rule set to obtain Interestingness rules.