A hybrid approach to web page categorization

Abstract

In the recent past, the World Wide Web has been witnessing an explosive growth and search engines are the most popular way of finding information on it. In most cases, the user is flooded with thousands of web pages in response to his or her search query and many users hardly go past the first few web pages. It is really debatable as to how useful or meaningful it is for any search engine to return so many web pages in response to a user query. In spite of the sophisticated page ranking algorithms employed by the search engines, the pages the user actually needs may actually get lost in the huge amount of information returned. Since most users of the web are not experts, grouping of the web pages into meaningful categories helps them to navigate quickly by reducing the search space. Web page classification and clustering are the two tasks which have been traditionally carried out by human beings who are experts in the domain. But in this electronic age, with the explosion in the amount of information available on the net, it is becoming increasingly difficult for human experts to classify or cluster all the documents available on the World Wide Web. Hence, it is increasingly evident that automatic techniques be used instead of human experts to carry out the tasks of web document classification and clustering, as part of the activity of categorizing them. Web page categorization is the main focus of this thesis. It is strongly believed and felt that the experience of a person using a web search engine is enhanced multifold if the search results are nicely categorized as against the case where the results are displayed as a flat list.