Thread: robot.txt
View Single Post
Old 2009-07-13, 01:36 AM   #2
Maj. Stress
Progress rarely comes in buckets, it normally comes in teaspoons
 
Maj. Stress's Avatar
 
Join Date: Jun 2005
Location: Dark Side Of Naboo
Posts: 1,289
Quote:
Originally Posted by faxxaff View Post
A lot of robots including Googlebot will do nothing but ignore a robots.txt file.
Do you have any factual data to back that comment up?
Maj. Stress is offline   Reply With Quote