How to Speed Up Google Crawl Rate
by Frank Jovine on 05/05/2009 in Google
There’s an option in Google Webmaster tools that allows you to increase the crawl speed to improve page indexing in Google. The default option is “Let Google determine my crawl rate.”
The two example screen shots show the default value and the value I set by selecting the “Set custom crawl rate.”
Default value
How to set custom crawl rate
I increased the crawl rate by 200%. This will speed up the indexing substantially for new links that are on the main page of my blog. When you are done with your custom setting, click on save.
You must have a Google account in order to use Google Webmaster Tools.
Extreme John
May 5th, 2009
Excellent information as always guys, I will be heading over to give this a shot now. Thank you. Added to Delicious
Frank J
May 5th, 2009
John,
Glad we could help!
Mike M
May 5th, 2009
Doesn’t work for some sites, on my account is says “Google has determined the crawl rate for this site and it cannot be changed”.
Frank J
May 5th, 2009
Mike,
I haven’t seen that, it may be because it’s a sub domain, but not sure. I will check further.
Internet Security Blog
May 8th, 2009
Hi Frank,
In my experience, changing the Google Crawl Rate does not guarantee you an increased crawling by Google. It is just a suggestion to Google. Google takes its own decision on when to crawl your site depending upon the importance and the frequency of update of your site.
I have my self experienced it on my blog several times.
Silki
Frank J
May 9th, 2009
It has worked for a few of my websites, but mostly for sites, blogs in this case that publish 2 to 3 new blogs per day.