In order to avoid unwanted information during the search indexes, website owners can instruct spiders never to crawl specified data files or directories with the standard robots.txt file in the basis directory from the area. Furthermore, a page can be explicitly excluded from a internet search engine's databases through the use of a meta tag precis