Quote:
Originally Posted by ivko999
If your site doesn't contain original content, you should exclude it from crawling (by sending us email or simply blocking the bot)
|
I think it's the other way around. If you host orginal content - you should block the bot.
Should we assume that your bot will ignore robots.txt, so we must use .htaccess to completely block you from showing our pages in your site's frames?
Code:
RewriteCond %{HTTP_REFERER} eonsex\.com [NC]
RewriteRule .* - [F]