Greenguy's Board

Greenguy's Board (http://www.greenguysboard.com/board/index.php)
-   General Business Knowledge (http://www.greenguysboard.com/board/forumdisplay.php?f=10)
-   -   Programs: Lock up your servers against Fusker (http://www.greenguysboard.com/board/showthread.php?t=18000)

cd34 2005-03-28 04:32 PM

So, with that, we have:

RewriteEngine on
# optionally allow empty referrers, remove to disallow empty referrers
RewriteCond %{HTTP_REFERER} !^$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(.*@)?([a-z0-9-]+\.)*yourdomain.com(:[0-9]+)?(/.*)?$ [NC]
RewriteRule .*\.(asf|mpg|mpeg|wmv|avi|rm|gif|jpeg|jpg)$ - [NC,F,L]

That basic rule should work in 99% of the situations. I've regression tested it with all of the test-set that I created earlier. I'll run with this on some real world testing on some clients.

Thank you Raymor for helping us come up with a fairly well protected drop-in rule that we can perhaps slap in a faq somewhere for webmasters to somewhat protect themselves. I know the .wmv is somewhat useless, but, it will stop some of the siterippers that do send referrers.

Then maybe we can get the submission pages to link back to a FAQ page at greenguysboard.com and educate the freesite/tgp submitters a bit.

Tommy 2005-03-28 04:37 PM

can you lay that htaccess out like i did (the whole thing)
so us laymen only have to copy and paste

frankthetank 2005-03-28 05:05 PM

Quote:

Originally Posted by cd34
So, with that, we have:

RewriteEngine on
# optionally allow empty referrers, remove to disallow empty referrers
RewriteCond %{HTTP_REFERER} !^$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(.*@)?([a-z0-9-]+\.)*yourdomain.com(:[0-9]+)?(/.*)?$ [NC]
RewriteRule .*\.(asf|mpg|mpeg|wmv|avi|rm|gif|jpeg|jpg)$ - [NC,F,L]

That basic rule should work in 99% of the situations. I've regression tested it with all of the test-set that I created earlier. I'll run with this on some real world testing on some clients.

Thank you Raymor for helping us come up with a fairly well protected drop-in rule that we can perhaps slap in a faq somewhere for webmasters to somewhat protect themselves. I know the .wmv is somewhat useless, but, it will stop some of the siterippers that do send referrers.

Then maybe we can get the submission pages to link back to a FAQ page at greenguysboard.com and educate the freesite/tgp submitters a bit.


I read the thread carefully and just looked at my stats. So I´m a victim of fusker, too. I tried out cd34 code and it works. My pics are gone from fusker and they can be seen from my domain.

Thank´s all of you... |thumb

swedguy 2005-03-28 05:18 PM

Quote:

Originally Posted by cd34
RewriteEngine on
# optionally allow empty referrers, remove to disallow empty referrers
RewriteCond %{HTTP_REFERER} !^$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(.*@)?([a-z0-9-]+\.)*yourdomain.com(:[0-9]+)?(/.*)?$ [NC]
RewriteRule .*\.(asf|mpg|mpeg|wmv|avi|rm|gif|jpeg|jpg)$ - [NC,F,L]

It's looking good. I would add a line for the IP too.

RewriteCond %{HTTP_REFERER} !^http://(.*@)?1\.2\.3\.4(:[0-9]+)?(/.*)?$

or

RewriteCond %{HTTP_REFERER} !^http://(.*@)?(([a-z0-9-]+\.)*yourdomain.com|1\.2\.3\.4)(:[0-9]+)?(/.*)?$ [NC]

I just ran a quick test and the latter is slightly faster, but a lot more messy if people should just copy/paste it and replace domain and IP in it.

Ms Naughty 2005-03-28 07:32 PM

OK I'm like Tommy, I really don't understand the code of htaccess...
And I've got a growing collection of different htaccess files saved but now I'm not sure which is the best one to use.

I also found this in my collection, labelled: "Htaccess for site strippers"
So I thought I'd throw it in to further confuse the situation LOL

RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Iria.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Stripper.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Copier.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Crawler.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Snagger.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Reaper.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Grabber.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Sucker.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Downloader.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Siphon.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Collector.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mag-Net.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Pockey.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*DA.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Snake.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BackWeb.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*gotit.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Vacuum.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SmartDownload.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Pump.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Ninja.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JOC.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*likse.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Memo.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*leech.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mirror.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Recorder.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Likse.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*attach.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Magnet.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bandit.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Ants.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Buddy.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Whacker.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*DISCo\Pump.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Drip.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EirGrabber.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*EyeNetIE.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FlashGet.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Gets.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go!Zilla.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Go-Ahead-Got-It.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Grafula.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*IBrowse.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*InterGET.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Internet\Ninja.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JetCar.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JustView.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MIDown\tool.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mister\PiX.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NearSite.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetSpider.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline\Explorer.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*PageGrabber.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Papa\Foto.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Pockey.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ReGet.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Slurp.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SpaceBison.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SuperHTTP.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebFetch.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebReaper.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebWhacker.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Web\Image\Collector.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Web\Sucker.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Webster.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*eCatch.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*ia_archiver.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*lftp.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*tAkeOut.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FileHound.*$

Chop Smith 2005-03-28 08:00 PM

Is someone going to put all of this together for us Htaccess code challenged folks?

amber438 2005-03-28 08:53 PM

Quote:

Originally Posted by cd34
RewriteCond %{HTTP_REFERER} !^$ [NC]

This line..for some reason I removed it from my htaccess files. I can not remember why. What does it do?

I share phpadsnew with all my domain ergo I have to list them all to get the ads to show.
I've also added what grandma scrotum has listed so my htaccess is a mile long..
so..how can we shorten it alos getting the most protection possible?

cd34 2005-03-28 09:48 PM

This rule allows empty referrers:

RewriteCond %{HTTP_REFERER} !^$ [NC]

GrandmaScrotum, phew, that ruleset has been floating around the net for a long time, and, its got many common errors. First, there is a space missing before the P, so, it would never match this. The non-anchored match at the front and the floating match to end of string aren't needed either.

RewriteCond %{HTTP_USER_AGENT} ^.*DISCo\Pump.*$ [OR]

Would be better as:

RewriteCond %{HTTP_USER_AGENT} ^DISCo\ Pump [OR]

and I sort of changed these by putting:

RewriteCond %{HTTP_USER_AGENT} ^Mozilla
RewriteRule /* - [L]

In front of your list of rules.

However, I really question whether those rules really do much anymore, since many surfers pretend to be Mozilla anyway.

But, if you were going to do it:

RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^Mozilla
RewriteRule /* - [L]

RewriteCond %{HTTP_USER_AGENT} ^Teleport [OR]
RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR]
RewriteCond %{HTTP_USER_AGENT} ^Aculinx [OR]
RewriteCond %{HTTP_USER_AGENT} ^Ants [OR]
RewriteCond %{HTTP_USER_AGENT} ^attach [OR]
RewriteCond %{HTTP_USER_AGENT} ^Backstreet [OR]
RewriteCond %{HTTP_USER_AGENT} ^BackWeb [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bandit [OR]
RewriteCond %{HTTP_USER_AGENT} ^Buddy [OR]
RewriteCond %{HTTP_USER_AGENT} ^Burner [OR]
RewriteCond %{HTTP_USER_AGENT} ^Caitoo [OR]
RewriteCond %{HTTP_USER_AGENT} ^Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^Commander [OR]
RewriteCond %{HTTP_USER_AGENT} ^Copier [OR]
RewriteCond %{HTTP_USER_AGENT} ^Crawler [OR]
RewriteCond %{HTTP_USER_AGENT} ^Curl [OR]
RewriteCond %{HTTP_USER_AGENT} ^DA [OR]
RewriteCond %{HTTP_USER_AGENT} ^Devil [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo\ Pump [OR]
RewriteCond %{HTTP_USER_AGENT} ^Down2Web [OR]
RewriteCond %{HTTP_USER_AGENT} ^Downloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^DownloadIt [OR]
RewriteCond %{HTTP_USER_AGENT} ^Drip [OR]
RewriteCond %{HTTP_USER_AGENT} ^Eater [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
RewriteCond %{HTTP_USER_AGENT} ^Eclipt [OR]
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^Enterprise [OR]
RewriteCond %{HTTP_USER_AGENT} ^Express [OR]
RewriteCond %{HTTP_USER_AGENT} ^Extractor [OR]
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]
RewriteCond %{HTTP_USER_AGENT} ^FairAd\ Client [OR]
RewriteCond %{HTTP_USER_AGENT} ^FileHound [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashSite [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlipBrowser [OR]
RewriteCond %{HTTP_USER_AGENT} ^Get [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]
RewriteCond %{HTTP_USER_AGENT} ^gotit [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grab [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]
RewriteCond %{HTTP_USER_AGENT} ^Greed [OR]
RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
RewriteCond %{HTTP_USER_AGENT} ^Hoover [OR]
RewriteCond %{HTTP_USER_AGENT} ^HtGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^HTTrack [OR]
RewriteCond %{HTTP_USER_AGENT} ^IBrowse [OR]
RewriteCond %{HTTP_USER_AGENT} ^iFox98 [OR]
RewriteCond %{HTTP_USER_AGENT} ^IGJpg$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]
RewriteCond %{HTTP_USER_AGENT} ^InternetLinkAgent [OR]
RewriteCond %{HTTP_USER_AGENT} ^IPhoto [OR]
RewriteCond %{HTTP_USER_AGENT} ^iwantmy [OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]
RewriteCond %{HTTP_USER_AGENT} ^JOC [OR]
RewriteCond %{HTTP_USER_AGENT} ^JustView [OR]
RewriteCond %{HTTP_USER_AGENT} ^Keepoint [OR]
RewriteCond %{HTTP_USER_AGENT} ^leech [OR]
RewriteCond %{HTTP_USER_AGENT} ^lftp [OR]
RewriteCond %{HTTP_USER_AGENT} ^likse [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Magnet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mag-Net [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^Memo [OR]
RewriteCond %{HTTP_USER_AGENT} ^MemoWeb [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mirror [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^MSProxy [OR]
RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetDrag [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline [OR]
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR]
RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
RewriteCond %{HTTP_USER_AGENT} ^PerManSurfer [OR]
RewriteCond %{HTTP_USER_AGENT} ^PlantyNet_WebRo [OR]
RewriteCond %{HTTP_USER_AGENT} ^Pockey [OR]
RewriteCond %{HTTP_USER_AGENT} ^Pump [OR]
RewriteCond %{HTTP_USER_AGENT} ^Reaper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Recorder [OR]
RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Retriever [OR]
RewriteCond %{HTTP_USER_AGENT} ^SilentSurf [OR]
RewriteCond %{HTTP_USER_AGENT} ^Siphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^Snagger [OR]
RewriteCond %{HTTP_USER_AGENT} ^Snake [OR]
RewriteCond %{HTTP_USER_AGENT} ^Snarf [OR]
RewriteCond %{HTTP_USER_AGENT} ^Snatcher [OR]
RewriteCond %{HTTP_USER_AGENT} ^SpaceBison [OR]
RewriteCond %{HTTP_USER_AGENT} ^Stripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]
RewriteCond %{HTTP_USER_AGENT} ^Vacuum [OR]
RewriteCond %{HTTP_USER_AGENT} ^Vampire [OR]
RewriteCond %{HTTP_USER_AGENT} ^wantmy [OR]
RewriteCond %{HTTP_USER_AGENT} ^Weazel [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web [OR]
RewriteCond %{HTTP_USER_AGENT} ^WFARC [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Whacker [OR]
RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWCopy [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWoffle
# whatever rule you want here.... block every request with the following rule
RewriteRule .* - [F]

Ms Naughty 2005-03-28 10:03 PM

By the way, if that huge "site stripper" htaccess is flawed, please feel free to point that out. As I said, I have no real idea what I'm doing here and I just copied that list from somewhere.
Edit: Woops, looks like I should refresh before I post LOL. Thanks for the info :D

cd34 2005-03-29 02:10 AM

slight leak in the rule, adjusted with yourdomain\.com (thanks Swedguy)

RewriteEngine on
# leave this line in allow empty referrers, remove to disallow empty referrers
RewriteCond %{HTTP_REFERER} !^$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(.*@)?([a-z0-9-]+\.)*yourdomain\.com(:[0-9]+)?(/.*)?$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(.*@)?1\.2\.3\.4(:[0-9]+)?(/.*)?$
RewriteRule .*\.(asf|mpg|mpeg|wmv|avi|rm|gif|jpeg|jpg|zip)$ - [NC,F,L]

In the last hour, its caught 14683 hotlink requests on one client's machine. Mostly fusker/usefulidiot, but, a few others that were surprising. hotmail, msngroups, a bunch of blogs, a few forums. Overall, it seems to be doing well -- negligable impact on the traffic on the machine though.

Altheon 2005-03-29 03:11 AM

I just tried to fusker my site and it failed so I know this code works:

Code:

Options +FollowSymlinks
RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?mydomain.com(/)?.*$    [NC]
RewriteRule .*\.(gif|jpg|jpeg|bmp|fla|mov|avi|wmv)$ - [F,NC]

If you all see a problem with it please let me know.

Additionally if you want to protect your directory structure from being viewed pop this little bit of code in there:

Code:

Options -Indexes
Then you will not have to put an index.html in each directory.

-A

Wenchy 2005-03-29 05:23 PM

For the .htaccess challenged, a summation if you please...
 
Okay, so can someone answer a couple of questions for me, please?

First, I'm working under the assumption that this code...

RewriteEngine on
# leave this line in allow empty referrers, remove to disallow empty referrers
RewriteCond %{HTTP_REFERER} !^$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(.*@)?([a-z0-9-]+\.)*yourdomain\.com(:[0-9]+)?(/.*)?$ [NC]
RewriteCond %{HTTP_REFERER} !^http://(.*@)?1\.2\.3\.4(:[0-9]+)?(/.*)?$
RewriteRule .*\.(asf|mpg|mpeg|wmv|avi|rm|gif|jpeg|jpg|zip)$ - [NC,F,L]

...is the recommended .htaccess for anti-fuskering my domains? I can/should repeat this line...

RewriteCond %{HTTP_REFERER} !^http://(.*@)?([a-z0-9-]+\.)*yourdomain\.com(:[0-9]+)?(/.*)?$ [NC]

... for each of my domains that I want included.

Second, the code that grandmascrotum put up there a couple of posts... I heard somewhere that that huge string can actually put a load on your server each time a page, any page, is loaded. Any truth to that rumor? And, assuming the rumor is false, is there any clear evidence that using that "collection" in my .htaccess actually does any good these days?

Thanks |thumb

RawAlex 2005-03-29 05:37 PM

Wenchy, I wouldn't put the long list except possibly inside a member's area, or an area that has high quantities of content, or possibly on key personal link areas (where many of your own sites are listed). Otherwise it's a ton of work for very little return, IMHO.

As for the "blocking fusker", the code you listed blocks ALL attempts to reach your images by a third party. If you want to allow things like google or yahoo to index your images, you need to let them in.

The alternate route is to specifically block all access from certain domains. You can do this on a more global level (either at the root level of your webserver, or in the webserver config). This could become a long list, so it really is up to you how you decide to handle it.

Either way, making your sites unfriendly to Fusker is an important move.

Alex
Alex

Wenchy 2005-03-29 05:53 PM

Thanks, Alex; sounds like excellent advice.

I'd prefer to allow google, yahoo, etc., and logic dictates I do that by using...

RewriteCond %{HTTP_REFERER} !^http://(.*@)?([a-z0-9-]+\.)*yourdomain\.com(:[0-9]+)?(/.*)?$ [NC]

... and replacing "yourdomain" with the appropriate SE domain.

Back in the day I was the victim of a hotlinker (japanese, I think) and it cost me a fortune in BW. I'm on a mad quest to prevent a repeat of those events whenever and by what ever means possible. I prefer to only have my nightmares when I'm sleeping |shocking|

Appreciate the assistance! |cool|

raymor 2005-03-29 07:24 PM

That long anti-ripper .htaccess is bad/wrong
in at least a couple ways. All of those rules will
impact performance. Even with all of those rules,
though, it's not nearly complete, so it won't block
more than half of the rippers.
The first rule of security is to disallow everything
that isn't specifically allowed.
That .htaccess violates that rule, leading to
the two problems I mentioned.
Rather, it would be better to list the 3 or 4 user agents
that are allowed and disallow everything else.
You'd allow IE, the Gecko browsers (Mozilla, Firefox and Safari are all Gecko and
thus would probably use just one rule),
Opera and perhaps you'd come up with a couple more.
Anything besides IE, Firefox, Safari, Mozilla, and Opera would be redirected.
Of course you may wish to also allow the main SE spiders.

This also has the inherent flaw that you're
assuming one thing based on another thing,
and in fact based on what the user tells you.
The major rippers will let the user set the User-agent
however they want, so just because it
SAYS it's IE doesn't mean that it is.
In fact several rippers are IE based and will therefore report as IE.
On the other hand some people using IE, Mozilla, or Firefox set their user-agent
to something else, such as "None of Your Business Version 0".
But in fact it's not the software name that you're
concerned with, it's a particular BEHAVIOR of the software.
So why not blocked based on that behavior?
That's what Strongbox does. Strongbox blocks
anyone who goes ripping your site, blindly following every single link.
On the other hand it does not block any browser where
the user actually clicks on the links.
THAT is what you really want to block, so that's
what Strongbox looks at, rather than the reported
name of the software.

airdick 2005-03-29 10:40 PM

Quote:

Originally Posted by RawAlex

As for the "blocking fusker", the code you listed blocks ALL attempts to reach your images by a third party. If you want to allow things like google or yahoo to index your images, you need to let them in.


Alex

I have always denied hotlinking and never added any special rules to allow google or yahoo, but I have images that are indexed in images.google.com and images.yahoo.com. My thumbnails that appear on those sites are not hotlinked -they are hosted on google & yahoo's servers and each thumb links to the page that that it was taken from, much like a thumbnail tgp.

kalle7 2005-03-31 08:10 AM

Just checked and found some of my stuff in fusker. Updated htaccess file uploaded and no more fusker! Nice |thumb

frankthetank 2005-03-31 12:46 PM

That´s what you get when you "fusker" the url "fusker.leww.com":

That URL has been banned for illegal content, on the owners request, or because it is a members only page!

maybe the guy now has realized that he is a thief... :D :D :D

jmf000 2005-04-01 10:00 AM

I just wonder would the condition
RewriteCond %{HTTP_REFERER} !^http://([-a-z0-9A-Z]+\.)*yourdomain\.com(/|$|:[0-9])
work faster than
RewriteCond %{HTTP_REFERER} !^http://(([a-z0-9-]+\.)+)?yourdomain.com(:[0-9]+)?(/.*)?$ [NC]
since it is unlikely subdomains contain uppercases.
I hope it will provide adequate security.
And by the way as I recall Domains are case sensitive (potentially).


And one more clue : I would not place .htaccess with such antihotlinking technics in the root of domain. Instead I would place it in the subdirectory(ies) under which "real heavy" content resides. So leachers still could see your hosted banners linked to sponsors but not actual images/video.

Thanks for your time.

Verbal 2005-04-01 11:41 AM

Quote:

Originally Posted by raymor
Rather, it would be better to list the 3 or 4 user agents that are allowed and disallow everything else.

This is some sound advice right here, instead of maintaining a huge blocked list. |thumb

jmf000 2005-04-02 04:08 AM

IMHO it is not important to pay attention to user agents at all.
1)Absolutely everyone who uses "teleporting" soft is able and does impersonate their kind of "browser" as "IE" or alike.
2)Surfers from leeching sources are using usual browser soft like IE or Opera and sequently not differ in this way from others.
3)Many new SEs started today, so you either have to include them all in your allowed browsers list or will loose their traffic.
4)Either black list or allowable list is larger is a kind of question.
I think it is waste of time to pay attention on user browser. Though you can exclude "TELEPORT" and very few other "defaults".

Thanks.


All times are GMT -4. The time now is 04:02 PM.

Powered by vBulletin® Version 3.8.1
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
© Greenguy Marketing Inc