SE Submission

ClickBuster · 2004-12-20, 07:18 PM

Do you submit to SEs or you wait for the robots and crawl your site? If so do you use robots meta tags? I read that it's better not to submit to Google, because that affects ranking - better be found, than beeing told where you're (ie if GB founds you thru a ref link, it's better than pointing the site for crawling).

Also, can somebody tell me how would you identify a bot?

I had several times found that strange referer URL in my logs:

xxxx:+++++++++...

Can somebody tell me what this is?

T Pat · 2004-12-20, 07:52 PM

I don't, if you build a free site and submit it to some of the bigger LL's you'll get crawled in a day or two or three

If you have a hub you want crawled just use one of the links your allowed on your free site for your hub and it will get crawled too.

AWStates is real easy to read and I think most hosting companies are includeing it in all their hosting packages now. That's all you need to track the bots
looks like this:

Robots/Spiders visitors (Top 10) - Full list - Last visit
12 different robots Hits Bandwidth Last visit
Road Runner: The ImageScape Robot 1906 15.66 MB 18 Dec 2004 - 06:11
Googlebot (Google) 695 11.51 MB 19 Dec 2004 - 19:59
Inktomi Slurp 621 9.19 MB 20 Dec 2004 - 04:57
Alexa (IA Archiver) 208 3.79 MB 17 Dec 2004 - 00:11
larbin 28 433.68 KB 09 Dec 2004 - 12:41
Unknown robot (identified by 'crawl') 7 47.70 KB 18 Dec 2004 - 03:11
Walhello appie 3 35.90 KB 20 Dec 2004 - 05:56
Jeeves 2 39.06 KB 16 Dec 2004 - 05:33
SurveyBot 2 37.54 KB 13 Dec 2004 - 12:34
Unknown robot (identified by 'robot') 2 39.06 KB 18 Dec 2004 - 05:46
Others 2 39.07 KB

DangerDave · 2004-12-20, 08:10 PM

Quote:

Originally posted by ClickBuster
I had several times found that strange referer URL in my logs:

xxxx:+++++++++...

Can somebody tell me what this is?

This is a fake referer "spoofed" by the users security software.. eg: Norton etc...

DD

2004-12-20, 07:18 PM	#1
ClickBuster I'm normally not a praying man, but if you're up there, please save me Superman! Join Date: Dec 2004 Location: Bulgaria Posts: 476	SE Submission Do you submit to SEs or you wait for the robots and crawl your site? If so do you use robots meta tags? I read that it's better not to submit to Google, because that affects ranking - better be found, than beeing told where you're (ie if GB founds you thru a ref link, it's better than pointing the site for crawling). Also, can somebody tell me how would you identify a bot? I had several times found that strange referer URL in my logs: xxxx:+++++++++... Can somebody tell me what this is? __________________ The tendency is to push it as far as you can -- Fear and Loathing In Las Vegas

2004-12-20, 07:52 PM	#2
T Pat You can now put whatever you want in this space :) Join Date: Aug 2003 Location: Paridise Posts: 3,244	I don't, if you build a free site and submit it to some of the bigger LL's you'll get crawled in a day or two or three If you have a hub you want crawled just use one of the links your allowed on your free site for your hub and it will get crawled too. AWStates is real easy to read and I think most hosting companies are includeing it in all their hosting packages now. That's all you need to track the bots looks like this: Robots/Spiders visitors (Top 10) - Full list - Last visit 12 different robots Hits Bandwidth Last visit Road Runner: The ImageScape Robot 1906 15.66 MB 18 Dec 2004 - 06:11 Googlebot (Google) 695 11.51 MB 19 Dec 2004 - 19:59 Inktomi Slurp 621 9.19 MB 20 Dec 2004 - 04:57 Alexa (IA Archiver) 208 3.79 MB 17 Dec 2004 - 00:11 larbin 28 433.68 KB 09 Dec 2004 - 12:41 Unknown robot (identified by 'crawl') 7 47.70 KB 18 Dec 2004 - 03:11 Walhello appie 3 35.90 KB 20 Dec 2004 - 05:56 Jeeves 2 39.06 KB 16 Dec 2004 - 05:33 SurveyBot 2 37.54 KB 13 Dec 2004 - 12:34 Unknown robot (identified by 'robot') 2 39.06 KB 18 Dec 2004 - 05:46 Others 2 39.07 KB __________________ How To Keep An Asshole In Suspense I'll Tell You Later