 |
|
|
|
|
|
|
|
|
|
|
2007-02-08, 03:12 PM
|
#1 |
|
I want to set the record straight - I thought the cop was a prostitute
Join Date: May 2004
Location: Serbia & Montenegro
Posts: 292
 |
you should double check if your robots.txt file is properly uploaded in domain root (that you probably done already)
Do you have any redirections in .htaccess file? According to the documentation, blank robots.txt file is equal to allowing all bots to spider everything on your site. |
|
|
|
2007-02-09, 09:57 AM
|
#2 |
|
They have the Internet on computers, now?
Join Date: Apr 2005
Posts: 148
|
It is in my root directory. Yes I have a re-direct in my .htaccess file to a 404 page. Do I need to add something so the bots can read my blank robots.txt file?
|
|
|
|
 |
|
|
Hybrid Mode
