Monday, October 26, 2009

How Google index any website

There is big question that how Google index any website. Here I am just trying to explain about Google indexing. A web crawler or spider plays an important role for indexing of any website. A web crawler reads keywords, description, hyperlinks, title and whole content of website as well. According to the information provided by spider Google stores the relevant information in its database about that website and decides the ranking and position of that website in Search engine. And search engine optimization helps you to attain this ranking in Google for your website.



Google has more than 200 web crawlers scanning the web each day. Same crawler visits so many websites each day and also different crawler may visit the same website. There are so many crawlers work for website. The crawler which crawl the URL of website collects the all information related to website. And it checks the following information related to URL of website.
  • New URLs
  • Comparison of old URL with new one
  • Old URL with old date stamp
  • Old URLs with new date stamps
  • 301 redirection of URL
  • 302 redirection of URL
  • 404 error URLs
The main and important indexing job is done by deep crawlers. Deep crawlers pick information of related to content of website, HTML, images, content & flash etc. Deep crawlers need to crawl millions of website each month. Once the Deep Crawlers index the content, it goes into their originating IDCs. Content is then processed, sorted and synchronized to the rest of the IDCs. A few years back, when the data size was manageable, this data synchronization used to happen once a month, lasting for 5 days, called Google Dance. Nowadays, the data synchronization happens constantly, which some people call Ever flux. An important thing that URL should be static not dynamic, because Google takes long time crawl dynamic URLs.


You can see from the image given below the major part of all searches depends on Google. Only Google is competing all other search engines.
What to do to make website Google friendly

Help crawlers by creating good interlinking, keywords, content and site map on your website. Delete all blocks from your website so Google can easily crawl your website.

Get lots of good incoming links to your pages from other websites to improve the importance of your web pages. There is no special need to submit your website to search engines. Links to your website on other websites are sufficient.

No comments:

Post a Comment