|
I doubt there can be a hidden robots.txt, unless there's a period before it - but then it wouldn't be robots.txt, it would be .robots.txt
Even if there is no robots.txt, a spider will still check for it. Does it show a 404 in your logs?
__________________
To alcohol! The cause of, and solution to, all of life’s problems
|