Robots.txt

[upallnite]

Originally Posted by raveo

Disallow: /private/ does this mean my members area ?
Would that fix the "broken" link that comes up when it tried to crawl the password protected area?

raveo
as DJilla said you need to change the example to your own file/folder names and setup.

Your members area is /members/
so you would use
Disallow: /members/

The folders you don't have as in the example you need to remove from the robots.txt file

Also need to remove this part End Format Example from the bottom.

Having a wacked robots.txt is worse then not having one at all. And it doesn't have anything to do with broken links. You just need to do a hunt and fix!