Greenguy's Board - View Single Post

Maj. Stress · 2009-07-09, 05:38 PM

The robots.txt should go in the root of the domain. It's a good idea to use one even if you want the entire domain indexed. Some overlooked uses for robots.txt are not indexing anything in your cgi-bin folder or 2257 pages.

2009-07-09, 05:38 PM	#6
Maj. Stress Progress rarely comes in buckets, it normally comes in teaspoons Join Date: Jun 2005 Location: Dark Side Of Naboo Posts: 1,289	The robots.txt should go in the root of the domain. It's a good idea to use one even if you want the entire domain indexed. Some overlooked uses for robots.txt are not indexing anything in your cgi-bin folder or 2257 pages.