|
|
|
|
|
|
|
|
|
|
|
#1 | |
|
Internet! Is that thing still around?
Join Date: Nov 2006
Posts: 6
|
Quote:
If your site doesn't contain original content, you should exclude it from crawling (by sending us email or simply blocking the bot) If you do host original content, then it is your best interest to be crawled. |
|
|
|
|
|
|
#2 | |
|
Certified Nice Person
|
Quote:
![]() Should we assume that your bot will ignore robots.txt, so we must use .htaccess to completely block you from showing our pages in your site's frames? Code:
RewriteCond %{HTTP_REFERER} eonsex\.com [NC]
RewriteRule .* - [F]
Last edited by Useless; 2006-12-23 at 02:53 PM.. |
|
|
|
|
|
|
#3 | |
|
Don't get discouraged; it's usually the last key that opens the lock...
|
Quote:
![]() |
|
|
|
|
![]() |
|
|