Crawlers (or bots) are used to collect info obtainable on the internet. By using website navigation menus, and studying inner and external hyperlinks, the bots begin to understand the context of a page. Of course, the words, images, and different information on pages also assist search engines like