Greenguy's Board - View Single Post

Maj. Stress · 2009-01-30, 05:05 AM

Don't worry about google crawling your site. They have their own agenda.
Robots.txt instructs bots which areas of your website to crawl and not crawl. A google search should give you plenty of info to get started.

2009-01-30, 05:05 AM	#2
Maj. Stress Progress rarely comes in buckets, it normally comes in teaspoons Join Date: Jun 2005 Location: Dark Side Of Naboo Posts: 1,289	Don't worry about google crawling your site. They have their own agenda. Robots.txt instructs bots which areas of your website to crawl and not crawl. A google search should give you plenty of info to get started.