 |
|
|
|
|
|
|
|
|
2007-02-07, 09:07 PM
|
#1 |
|
They have the Internet on computers, now?
Join Date: Apr 2005
Posts: 148
|
robot.txt file
Can someone explain the value of robot.txt. Is it important?
I have created a robot.txt file and uploaded it into the root of my domain but when I do a search for it using one of the many tools available I get an error message: robots.txt file does not appear to be valid and the text points to my 404 page. What am I doing wrong? Should I be concerned? |
|
|
|
|
Threaded Mode