Greenguy's Board WebcamWiz CRAZY $5,000 Reward Bonuses
WebcamWiz CRAZY $5,000 Reward Bonuses WebcamWiz CRAZY $5,000 Reward Bonuses WebcamWiz CRAZY $5,000 Reward Bonuses WebcamWiz CRAZY $5,000 Reward Bonuses WebcamWiz CRAZY $5,000 Reward Bonuses

Go Back   Greenguy's Board > Search Engines
Reload this Page Googlebot is getting quicker
Register FAQ Calendar Today's Posts

Reply
 
Thread Tools Search this Thread Rate Thread Display Modes
Old 2005-08-11, 03:32 PM   #1
twthumb
A woman is like beer. They look good, they smell good, and you'd step over your own mother just to get one!
 
Join Date: Aug 2005
Posts: 54
Googlebot is getting quicker
I just was watching my logs for maybe something interesting and saw google coming by. I always thought google wouldn't flood a server with request:

Quote:
66.249.66.211 - - [11/Aug/2005:21:22:48 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:22:53 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:22:55 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:22:56 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:03 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:06 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:07 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:12 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:14 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:18 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:21 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:23 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:25 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:42 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:44 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:46 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:50 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:52 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:53 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:56 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:23:58 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:00 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:05 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:06 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:15 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:17 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:18 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:18 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:18 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:18 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:27 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:28 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:33 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:36 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:37 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:39 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:42 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:44 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:46 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:49 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:51 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:52 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:55 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:57 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:24:59 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:25:08 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:25:10 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:25:11 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.66.211 - - [11/Aug/2005:21:25:16 +0200] "GET xxx "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
Of couse you can request pages a lot faster, but this is kind of strange of google, sometimes more than 4 request a second.
__________________

- Build your own blog empire
twthumb is offline   Reply With Quote
Old 2005-08-11, 09:51 PM   #2
cj
Heh Heh Heh! Lisa! Vampires are make believe, just like elves and gremlins and eskimos!
 
cj's Avatar
 
Join Date: Jun 2005
Location: Tucson, Az
Posts: 70
Send a message via ICQ to cj Send a message via AIM to cj
i noticed that google spiders my sites, 3 times aday, to yahoo's 320.... And ya'll heard of google's new site map submit... Make a site map and then submit it to google, got my pages in the search engine, but they suck, only showing the URL and title or description... So another words... don't submit a site map...

Yahoo is kinda using the same idea, by saving a text file to your server called "urllist.txt", you can use a site spider, and spider your site your self. save it to the text file then submit your text file to yahoo....

when yahoo adds the url, they just don't like the url, but url, title and description...

another thing i noticed about google.com, they don't like tags.. like default.asp?id=1000

you can get your pages in with tags... but it takes alot longer... fuck it sucks...

i think google is going down hill and yahoo is going to fuck them up :o)
cj is offline   Reply With Quote
Old 2005-08-11, 10:15 PM   #3
Erick G
Look maah I moved out and got my own space:)
 
Join Date: Jun 2005
Posts: 555
Send a message via ICQ to Erick G
Google is buzy spending bazillions of dollars from IPO
__________________

Exclusive Feeds
Erick G is offline   Reply With Quote
Old 2005-08-12, 01:28 PM   #4
twthumb
A woman is like beer. They look good, they smell good, and you'd step over your own mother just to get one!
 
Join Date: Aug 2005
Posts: 54
Hmm, i never made a sitemap, i might take a look at it, thanks for the tip!

I also thought that google didn't like query strings (?id=x&do=that) but for some reason he seems to like my query strings, i can't find any reason why. For example the above request all hit a page which has a url with a query string. Google is a strange thing
__________________

- Build your own blog empire
twthumb is offline   Reply With Quote
Old 2005-08-13, 07:09 PM   #5
cj
Heh Heh Heh! Lisa! Vampires are make believe, just like elves and gremlins and eskimos!
 
cj's Avatar
 
Join Date: Jun 2005
Location: Tucson, Az
Posts: 70
Send a message via ICQ to cj Send a message via AIM to cj
google hates query strings... But after while they seem to add them... but it takes months & months... Don't submit a site map!!!

This is what your pages will look like, Exmaple:
http://www.google.com/search?hl=en&q...jspornsite.com

No title or description just url address...

Use Yahoo's urllist.txt, its up to you, let me know if you have better luck than my self.
cj is offline   Reply With Quote
Old 2005-08-13, 07:10 PM   #6
cj
Heh Heh Heh! Lisa! Vampires are make believe, just like elves and gremlins and eskimos!
 
cj's Avatar
 
Join Date: Jun 2005
Location: Tucson, Az
Posts: 70
Send a message via ICQ to cj Send a message via AIM to cj
my site was up 6-22-05 and i already got 2,700 pages in yahoo.com. use yahoo, fuck google...

Don't rely on g, it takes months from them to add pages, or maybe they just hate my site...
cj is offline   Reply With Quote
Old 2005-08-15, 05:34 PM   #7
twthumb
A woman is like beer. They look good, they smell good, and you'd step over your own mother just to get one!
 
Join Date: Aug 2005
Posts: 54
Quote:
Originally Posted by cj
google hates query strings... But after while they seem to add them... but it takes months & months... Don't submit a site map!!!

This is what your pages will look like, Exmaple:
http://www.google.com/search?hl=en&q...jspornsite.com
Ooh, that sucks big time. You convinced me to never make a site map for a new site.
I hope google picks up your site soon.
__________________

- Build your own blog empire
twthumb is offline   Reply With Quote
Old 2005-08-16, 06:21 AM   #8
cj
Heh Heh Heh! Lisa! Vampires are make believe, just like elves and gremlins and eskimos!
 
cj's Avatar
 
Join Date: Jun 2005
Location: Tucson, Az
Posts: 70
Send a message via ICQ to cj Send a message via AIM to cj
thanks alot, but i don't think so... site maps fuck you... so pass the word...

oh yeah... wanna exchange links? :o)

cj@deluxeelectronics.com
cj is offline   Reply With Quote
Old 2005-08-29, 06:06 AM   #9
Ace-imlive
Heh Heh Heh! Lisa! Vampires are make believe, just like elves and gremlins and eskimos!
 
Join Date: Jun 2005
Location: Back In School
Posts: 78
Quote:
Originally Posted by cj
thanks alot, but i don't think so... site maps fuck you... so pass the word...

oh yeah... wanna exchange links? :o)

cj@deluxeelectronics.com

google is slower

but just create a HTML simple sitemap

versus the xml they ask for or the .gz

then the bot grabs it no problem
Ace-imlive is offline   Reply With Quote
Old 2005-08-29, 10:48 AM   #10
Halfdeck
You can now put whatever you want in this space :)
 
Halfdeck's Avatar
 
Join Date: Oct 2004
Location: New Haven, CT
Posts: 985
Send a message via ICQ to Halfdeck
The reason why google doesn't like cj's page is because the html code has alot of errors, not because he submitted a site map.
__________________
Success is going from failure to failure without a loss of enthusiasm.
Halfdeck is offline   Reply With Quote
Old 2005-08-29, 12:30 PM   #11
Surfn
If you don’t take a chance the Angels won’t dance
 
Surfn's Avatar
 
Join Date: Aug 2003
Location: Earth on occasion
Posts: 8,812
Send a message via ICQ to Surfn
Quote:
Originally Posted by Halfdeck
The reason why google doesn't like cj's page is because the html code has alot of errors, not because he submitted a site map.
LOL true enough
__________________

Surfn's Links Are you a partner?

Surfn is offline   Reply With Quote
Old 2005-08-31, 03:41 PM   #12
troy
I can spearhead the whole begging thing
 
troy's Avatar
 
Join Date: Apr 2003
Location: nederlands
Posts: 414
Send a message via ICQ to troy
Quote:
i think google is going down hill and yahoo is going to fuck them up :o)
I hope so
__________________
Cumshot Specialist
Great Porn
troy is offline   Reply With Quote
Old 2005-08-31, 11:45 PM   #13
SH
If something goes wrong at the plant, blame the guy who can't speak English
 
Join Date: Jun 2004
Location: Netherlands
Posts: 37
yahoo is only good for finding spam, think msn search will be the next google in few years when its build in new windows version
SH is offline   Reply With Quote
Old 2005-09-01, 01:37 AM   #14
Surfn
If you don’t take a chance the Angels won’t dance
 
Surfn's Avatar
 
Join Date: Aug 2003
Location: Earth on occasion
Posts: 8,812
Send a message via ICQ to Surfn
If you are having a hard time with google or yahoo listing (getting your page results in the top 20) today you will have a hard time with any new SE that comes along. This isn't rocket science, close maybe. All the documentation is public on what each SE needs to list your sites.

It seems that people read less and less and whine more and more
__________________

Surfn's Links Are you a partner?

Surfn is offline   Reply With Quote
Old 2005-09-02, 04:39 AM   #15
sean
Rock stars ... is there anything they don't know?
 
Join Date: Jan 2005
Posts: 19
Quote:
Originally Posted by SH
yahoo is only good for finding spam, think msn search will be the next google in few years when its build in new windows version
True.I'm already getting more and more hits from MSN.
__________________
Hardlinks for sale on PR4 blogs.ICQ:304973189 or luckyteens at gmail.com
sean is offline   Reply With Quote
Reply

« Previous Thread | Next Thread »

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -4. The time now is 08:28 PM.

Contact Us - Greenguy's Board - Archive - Top

Mark Read
Powered by vBulletin® Version 3.8.1
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
© Greenguy Marketing Inc