2012-01-20, 05:08 PM
|
#7
|
|
Certified Nice Person
Join Date: Oct 2003
Location: Dirty Undies, NY
Posts: 11,268
|
Our lord, Google, actually slowed their crawl rate in order to aid those of us who were protesting SOPA.
http://searchenginewatch.com/article...rotesting-SOPA
Quote:
GoogleBot Slowed... Well, to a Crawl
The Next Web cites web performance and security company CloudFlare in showing a 60.7 percent decrease in Google's crawl rate on January 17. This is an inevitable and natural result of its spying an increase in 503 headers, the recommended cue to GoogleBot that the temporary blackout content should not be indexed. Baidu also slowed to a lesser extent, by 11.23 percent.
|
Quote:
|
Bing, on the other hand, crawled 2.35 percent faster. TNW's Brad McCarty noted, "The unintended consequence here is that today Google perhaps has results that are not as 'fresh' as Bing. However, it's also possible that Bing's increased crawl rate has left it with a few more results that are indicative of the blackout, instead of what a site's content actually is."
|
__________________
Click here to purchase a bridge I'm selling.
|
|
|
Threaded Mode