robots.txt do you need to have one?

Klove · 2004-09-29, 03:04 PM

do you need to have one, or can you survive without one?

DangerDave · 2004-09-29, 04:27 PM

Klove you don't NEED to have one.. but if you have /dirs/ you want robots to stay out of... then yes... it's worth having one.

DD

Klove · 2004-09-29, 04:27 PM

Thank you, I want it to sniff my entire domain! ;P

2004-09-29, 03:04 PM	#1
Klove Well you know boys, a nuclear reactor is a lot like women. You just have to read the manual and press the right button Join Date: Sep 2004 Location: Montreal, Quebec Posts: 156	robots.txt do you need to have one? do you need to have one, or can you survive without one? __________________ Klove ---------------------------- <a href="http://www.smutdaily.net/webmasters/">Submit To SmutDaily.Net</a>

2004-09-29, 04:27 PM	#2
DangerDave Bonged Join Date: Mar 2003 Location: BrisVegas, AUSTRALIA Posts: 4,882	Klove you don't NEED to have one.. but if you have /dirs/ you want robots to stay out of... then yes... it's worth having one. DD __________________ Old Dollars >>>> Now with over 90 Hosted Free Sites <<<< DangerDave.com.au - Adult Links to Free Porn

2004-09-29, 04:27 PM	#3
Klove Well you know boys, a nuclear reactor is a lot like women. You just have to read the manual and press the right button Join Date: Sep 2004 Location: Montreal, Quebec Posts: 156	Thank you, I want it to sniff my entire domain! ;P __________________ Klove ---------------------------- <a href="http://www.smutdaily.net/webmasters/">Submit To SmutDaily.Net</a>