Robots/Spiders visitors ??

th3g4me · 2004-03-08, 01:04 AM

Googlebot (Google)
LinkWalker
Unknown robot (identified by 'robot')

Whats that ?

Fonz · 2004-03-08, 03:47 AM

that are spiders from search engines. They visit your site and analyse it. This way they determine how you should appear on their search engine.

Linkster · 2004-03-08, 06:22 AM

Some are doing the SE work - like the Googlebot one - some are looking for email addresses, and harvesting links for auto-generating emails to you or others(definitely something to look at if you are getting alot of the bad ones is to ban them using robots.txt) - I'll post a list of the ones I know in a bit

2004-03-08, 01:04 AM	#1
th3g4me Trying is the first step towards failure Join Date: Feb 2004 Posts: 121	Robots/Spiders visitors ?? Googlebot (Google) LinkWalker Unknown robot (identified by 'robot') Whats that ?

2004-03-08, 03:47 AM	#2
Fonz Former pr0n slinger. Join Date: Aug 2003 Location: Antwerp, Belgium Posts: 7,929	that are spiders from search engines. They visit your site and analyse it. This way they determine how you should appear on their search engine. __________________ See how I abuse little trees on my Shumi no Bonsai Blog

2004-03-08, 06:22 AM	#3
Linkster NO! Im not a female - but being a dragon, I do eat them. Join Date: Mar 2003 Location: Sex Delta Posts: 5,084	Some are doing the SE work - like the Googlebot one - some are looking for email addresses, and harvesting links for auto-generating emails to you or others(definitely something to look at if you are getting alot of the bad ones is to ban them using robots.txt) - I'll post a list of the ones I know in a bit __________________ Pussy Chompers Porn Links NSCash