Another download program to add to htaccess

[Surfn]

Originally posted by topsmutlinks
A friend of mine is telling me that putting my domains in there will only prevent hotlinking and not block the software that downloads the entire site in one day. Is this true?

Sorry to keep asking stupid questions.

That's usually done with spiders. Do you know how to write text to tell spiders to follow or not follow the page/site?

This will keep them out. BUT it will keep SE spiders out as well. Keep that in mind:

<meta name="copyright" content=" This web page is copyrighted and the property of the author. Copying and distribution is forbidden without prior permission ">


<meta name="robots" content=" NOINDEX,NOFOLLOW ">