|
|
|
|
|
|
|
|
|
|
|
#1 |
|
Internet! Is that thing still around?
Join Date: Nov 2006
Posts: 6
|
Eonpal crawling
hello,
we (eonpal) crawl sites for adult content. more info can be found here: http://www.eonpal.com However, If you want to be excluded from it, please send us list of domains and we'll block our crawler from going there. or simply just block traffic coming with http referrer *.eonpal.com (this is what you referred as referral spam) good or bad? your content might appear in search engine so I guess is good. if it's not, send us list of domains, we'll exclude them immediately. I'll be around post to answer your questions, if any cheers, |
|
|
|
|
|
#2 |
|
Certified Nice Person
|
What do you mean by that exactly? Are you looking to list our URLs on your engine, even though your ads aren't on our pages, or are you ripping the photo content from our pages to use on your almost 2 million galleries?
If you are listing our URLs - why? Are you overriding our affiliate codes with your own? Explain to us why we want you to crawl our sites. |
|
|
|
|
|
#3 | |
|
Internet! Is that thing still around?
Join Date: Nov 2006
Posts: 6
|
Quote:
If your site doesn't contain original content, you should exclude it from crawling (by sending us email or simply blocking the bot) If you do host original content, then it is your best interest to be crawled. |
|
|
|
|
|
|
#4 | |
|
Certified Nice Person
|
Quote:
![]() Should we assume that your bot will ignore robots.txt, so we must use .htaccess to completely block you from showing our pages in your site's frames? Code:
RewriteCond %{HTTP_REFERER} eonsex\.com [NC]
RewriteRule .* - [F]
Last edited by Useless; 2006-12-23 at 02:53 PM.. |
|
|
|
|
|
|
#5 | |
|
Don't get discouraged; it's usually the last key that opens the lock...
|
Quote:
![]() |
|
|
|
|
![]() |
|
|