Googlebot is getting quicker

twthumb · 2005-08-11, 03:32 PM

I just was watching my logs for maybe something interesting and saw google coming by. I always thought google wouldn't flood a server with request:

Quote:

66.249.66.211 - - [11/Aug/2005:21:22:48 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:22:53 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:22:55 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:22:56 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:03 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:06 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:07 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:12 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:14 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:18 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:21 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:23 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:25 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:42 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:44 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:46 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:50 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:52 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:53 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:56 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:58 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:00 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:05 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:06 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:15 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:17 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:18 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:18 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:18 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:18 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:27 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:28 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:33 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:36 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:37 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:39 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:42 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:44 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:46 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:49 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:51 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:52 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:55 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:57 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:59 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:25:08 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:25:10 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:25:11 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:25:16 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

Of couse you can request pages a lot faster, but this is kind of strange of google, sometimes more than 4 request a second.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Switch to Hybrid Mode Threaded Mode	Rate This Thread: