robots txt to prevent crawling of freesites

[ponygirl]

Originally Posted by picXX

Hi.

I am the owner of both soccerwank.com , sexcarrot.com and pornogata.com wich have the same robots.txt files and all the domains.

When i first started out building galleries and freesites, i was told to create a robots.txt file like that to prevent google from crawling thousands of duplicate galleryfiles and hundred of duplicated freesitefiles.

I am linking to the freesites on my mainsite, but offcourse that would not benefit all the LL i am submitting to.

I willl change the robots.txt files on all my domain asap, and prevent from crawling only the galleryfolders instead of the freesite directory folders.

I am sorry i have braught up the issue, cuz i was really not aware of it, i just followed some friends good advice.

I rather thought it was something like that.

Originally Posted by picXX

And please dont see my as a possible cheater.. i have never even once tried to cheat fellow webmasters with intention.

That's why I wanted to see what this was all about...I've never seen anything in your sites suggesting you cheat in any way, and I don't think you are a cheater either picXX. Thanks for coming by and clearing it up