Framework for Analysing Diverse Patterns on Web and Social Media Through Data Analytics

Abstract

The advancement of internet technology and the dynamic nature of World Wide Web attract large number of users for publishing and retrieving information. But due to the heterogeneous and huge quantity of data, most of the information on the web is uninteresting to the users. Thus developing effective algorithm for retrieving the relevant information without accessing the complete data; at the outset, it has become an important concern among the Web mining research communities. Though most researchers focused their research work in this area, still their focus is only on retrieving similar patterns by leaving dissimilar patterns which are likely to contain the outlying data. This work concentrates on mining web content outliers which extracts the dissimilar web document taken from a group of documents of the same domain. Moreover, mining web content outliers helps in promoting business activities and improving the quality of the search results. In this work, a novel mathematical approach based on proportionate method is developed for retrieving relevant web document through outlier detection technique. The removal of outlaid documents improves the quality of search results catering to the user needs. Experimental results proved that this method gives better results in terms of accuracy, recall and specificity than the existing approach. Web usage outlier mining is dedicated to find usage patterns which differ significantly from the rest of the web documents taken from the web server log files. Shifting through the unstructured and ever growing web data for outliers is more challenging than finding outliers in numeric datasets. The existing web mining algorithms concentrate on finding similar patterns leaving dissimilar patterns that are likely to contain outlying data such as exceptions, noise, irrelevant and rare patterns. Analysis on these diverse patterns on the web plays a crucial role in identifying competitors in business, detecting frauds in banking activities, detecting network intrusion, and spam filtering