|
Some people do that to avoid duplicate page problems. People who build doorways are getting smart to the fact that google doesn't like them (making me feel better about asking to always be on an index.html page and not on index9898978.html). Obviously a robots.txt that blocks SE's from spidering the page limits your likely long term traffic, and that does change the balance between incoming and outgoing traffic potentials.
I can understand why they might do it, and I can understand you removing them from your list.
Alex
|