htaccess help...yep, I searched first :)

[MrYum]

Been messing with this all afternoon...driving me nuts...admittedly a short trip...lol.

Noticed in server logs that I've been fuskered...fuckers

Searched the board...read a LOT. Looked at Cleo's notes...thanks Cleo! Used the htaccess generator link on your page to generate the code.

The code seems to work...UNLESS I add the null referer line to allow browsers/firewalls that remove referrer header info.

This is the line in question:

RewriteCond %{HTTP_REFERER} !^$

Without that line...it works. Add the line...and the htaccess checkers show the test image.

Also worth noting...the code doesn't seem to stop fusker either way

I know there's some very tech savvy folks on this board...any assistance would be hugely appreciated.

Also, what's the consensus on adding an allow line so google can spider/cache images?

[Cleo]

You are right about the no referral issue.

Add this to your htacess

order allow,deny
deny from lewww.com
allow from all

[swedguy]

Have in mind that IF you have a redirect at the end of the htaccess, that page will load instead.

An example:

They have fuskered your domain.com/01.jpg image that is 30k. You put the htaccess in there and redirect them all to your LL that is 150k (HTML + images). So what will happen is that the request for the 01.jpg redirects and loads your LL instead. They will not see it since it's "trapped" in a image, it will only be seen as a broken image.

What I said above hopefully makes sense after you read it couple of times, I just typed it up real fast without fixing bad sentences

So instead of this line in the htaccess

RewriteRule (gif|jpg|png|swf|mpg|avi|GIF|JPG|PNG|SWF|MPG|AVI)$ http://www.yourdomain.com/bad.html [R,L]

use this:

RewriteRule (gif|jpg|png|swf|mpg|avi|GIF|JPG|PNG|SWF|MPG|AVI)$ - [F]


That way nothing will load and they will get a forbidden error = no bandwidth used.



Btw. post what your htaccess looks like

[Porn Meister]

"the htaccess checkers"? Then they must not be sending a referring url is all

I'm not sure if it's 100% safe to assume that since fukser shows up in your logs that it's sending referring information, but it's probably 99% (I'd guess).

So chances are that even if an htaccess checker is allowed in, then you're ok.

Try testing it by adding the htaccess checkers domain to your htaccess, then check it, hehe

I've looked at the fusker site and although I saw them linking to a gallery or two, they didnt hurt me at all so I didnt bother to block it yet.

Maybe when I do, I'll let them link to html, but for jpg/gif I'll have it load up an image advertising my root url(s), hehe

[Porn Meister]

"That way nothing will load and they will get a forbidden error = no bandwidth used."

Unless you've specified an errordocument for 401 errors too (which I do for instance)

[cd34]

order allow,deny
deny from lewww.com
allow from all

nooooooooo

this won't stop hotlinking, and will force every request below this to require reverse dns lookup - and depending on your server and upstream DNS, this could cause quite a performance issue.

What you can do is something like:

RewriteEngine on
RewriteCond %{HTTP_REFERER} usefulidiots [OR]
RewriteCond %{HTTP_REFERER} fusker [OR]
RewriteCond %{HTTP_REFERER} webrats
RewriteRule .* - [F]

That will block the 3 PITA sites.

Alternatively, the reverse would be to allow only the valid domains to hotlink (i.e. your site)

Regrettably, with the blank referrer, most people can bypass the check. Usually during a hotlink attack, I disallow blank referrers until the storm blows over.

In fact, we even wrote a tux-specific rule called lockdown that does that.

[Cleo]

I've had them load up my LL server so bad that it slowed down to a crawl.

The only thing that has worked for me is just to completely block the domain as mod_rewrite didn't seem to work.

EDIT
cd34 posted at the same time as me.

[Porn Meister]

And dont make a mistake I did once and put an [OR] on the last url to block like in cd34's example (notice the last url has no [OR]? Thats important)! LOL

infinite mod-rewrite loop. wheee

[Cleo]

So can I just replace my allow/deny with your example?


ErrorDocument 404 http://erotic-exotic-porn.com/
ErrorDocument 403 http://erotic-exotic-porn.com/

RewriteEngine on
RewriteCond %{HTTP_REFERER} lewww [OR]
RewriteCond %{HTTP_REFERER} fusker
RewriteRule .* - [F]

DirectoryIndex index.shtml index.php index.html index.htm

RewriteEngine on
RewriteOptions inherit
RewriteCond %{HTTP_REFERER} !^$

RewriteCond %{HTTP_REFERER} !^http://www.cleoshornylinks.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://cleoshornylinks.com*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://cleoshornylinks.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.cleoshornylinks.com*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://cleoshornylinks.com:80/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.cleoshornylinks.com:80/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://cleoshornylinks.com:80.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.cleoshornylinks.com:80.*$ [NC]

RewriteRule (gif|jpg|png|swf|mpg|avi|GIF|JPG|PNG|SWF|MPG|AVI)$ http://cleo-says-fuck-you.com/ [R,L]




Having "RewriteEngine on" in there twice doesn't look right to me. Should I kill the second one?

[cd34]

ErrorDocument 404 http://erotic-exotic-porn.com/
ErrorDocument 403 http://erotic-exotic-porn.com/

#don't really need this if you are doing 'positive' checking below
# however, if you did do this, it would short circuit things, making the deny happen much more quickly, at which point you should do [F,L]
#RewriteEngine on
#RewriteCond %{HTTP_REFERER} lewww [OR]
#RewriteCond %{HTTP_REFERER} fusker
#RewriteRule .* - [F,L]

DirectoryIndex index.shtml index.php index.html index.htm

# the regexp engine performs better with only a left anchor rather than a left & right anchor with a floating match

RewriteEngine on
RewriteOptions inherit
RewriteCond %{HTTP_REFERER} !^$

RewriteCond %{HTTP_REFERER} !^http://www.cleoshornylinks.com [NC]
RewriteCond %{HTTP_REFERER} !^http://cleoshornylinks.com [NC]
RewriteCond %{HTTP_REFERER} !^http://cleoshornylinks.com [NC]
RewriteCond %{HTTP_REFERER} !^http://www.cleoshornylinks.com [NC]
# since there is a left anchor above, you can really remove these since the are already matched above
#RewriteCond %{HTTP_REFERER} !^http://cleoshornylinks.com:80/.*$ [NC]
#RewriteCond %{HTTP_REFERER} !^http://www.cleoshornylinks.com:80/.*$ [NC]
#RewriteCond %{HTTP_REFERER} !^http://cleoshornylinks.com:80.*$ [NC]
#RewriteCond %{HTTP_REFERER} !^http://www.cleoshornylinks.com:80.*$ [NC]

RewriteRule (gif|jpg|png|swf|mpg|avi|GIF|JPG|PNG|SWF|MPG|AVI)$ http://cleo-says-fuck-you.com/ [R,L]

That's what I would suggest -- it removes a few checks which streamlines the request processor. Saving a few cpu cycles per request doesn't sound like much, but, when the thundering herd comes in, sometimes even saving a few cpu cycles here and there can allow your machine to perform a little better.

I have a P3/800 doing 1.3 million pageviews a day at 12% cpu utilization. A client has a P2.4 doing 300k (doing the same effective task) and averages 40% cpu utilization.

Personally, I like the optimizations.

[GeorgeTH]

HELP - I'm so bad when it comes to understanding this technical gobbledygook!

Originally posted by cd34
[# the regexp engine performs better with only a left anchor rather than a left & right anchor with a floating match

SAMPLE, part of my current htaccess:

RewriteCond %{HTTP_REFERER} ^http://forum.mokkels.nl/.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://forum2.mokkels.nl/.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://forum3.mokkels.nl/.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http(s)?://fusker.lewww.com/.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?fusker.lewww.com/.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?bbs.peachy18.com/.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?images.devilfinder.com/.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?devilfinder.com/.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?grin-reaper.com/.*$ [NC]

Did your comment mean that I should NOT have lines like
RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?grin-reaper.com/.*$,
but rather 2 lines like
RewriteCond %{HTTP_REFERER} ^http://grin-reaper.com/.*$
and
RewriteCond %{HTTP_REFERER} ^http://www.grin-reaper.com/.*$

???

[Cleo]

For some odd reason I do seem to need it with the port in there.

For testing I took the lines out with the port number.
Clear your cache and see the broken banner
http://cleoslinksforhornysurfers.com:80/
Looks like I need to put the lines with the port number back in. Because of all the domains that I own my htaccess is 600+ lines so it would have been really nice if I could have taken these lines out.

Fusker somehow gets around my htaccess. I'm guessing that they block referral information taking advantage of the fact that I allow typeins to see an image.
RewriteCond %{HTTP_REFERER} !^$
So I'm not really sure why this does block them but it does seem to.
order allow,deny
deny from lewww.com
allow from all

This makes me wonder if this will actually work.
RewriteEngine on
RewriteCond %{HTTP_REFERER} lewww [OR]
RewriteCond %{HTTP_REFERER} fusker
RewriteRule .* - [F,L]

Guess I need to test it.

[cd34]

you know, that is very strange -- I checked one version of apache, and the :80 doesn't work, another version and it does. I don't really see any changes to the mod_rewrite code between them, but, the request processor seems to change what it sends based on the way the header is received.

It is interesting, I guess the :80 is required based on how the request is sent through.

From the testing I have done, those (www\.) matches are slower than two anchored matches, however some floating matches, like

RewriteCond %{HTTP_REFERER} mokkels.nl [NC,OR]

can actually be faster. Because it is not anchored and there are no metas, most regexp libraries translate it to strstr

I'm also not a fan of the /.*$ at the end For some reason, those appear to evaluate slower.

[Cleo]

Spending a bunch of years as a Rent-a-Mac-Geek allowed me to see my sites on thousands of computers all over the place. (PC & Mac)

Being that I worked in the creative fields most of them were fine with porn so they would show me their sites and I would show them mine.

I haven't really played around with the endings but I do seem to remember that I went to the /.*$ at the end after some of my stuff wasn't loading properly in some URLs.

[cd34]

about 1.5 years ago, mod_rewrite was rewritten to use a different regexp engine -- at that time, they fixed a number of issues.

I think I discovered the cause of the :80 issue, I don't understand why some browsers have an issue, but, I'm doing some captures.

According to the specs, that behavior is incorrect.

[MrYum]

Excellent info everyone...thanks!

Some of this is actually starting to sink in...I think...lol.

Yep Swedguy...makes complete sense. Got that from one of the searches I did prior to posting. Even went so far as to create a quick banner saying the images are hotlinked and go to http://www.mydomain.com/ to see the images.

Perhaps true PM, but both of the online utilities I was using specifically mentioned the no referer issue. One would think they would pass referer info along in their tests to verify the htaccess is working. But, perhaps not...

Okay, so based on what I'm reading here...how's this look?

RewriteEngine on
RewriteCond %{HTTP_REFERER} usefulidiots [OR]
RewriteCond %{HTTP_REFERER} fusker [OR]
RewriteCond %{HTTP_REFERER} webrats
RewriteRule .* - [F,L]

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://www.mydomain.com [NC]
RewriteCond %{HTTP_REFERER} !^http://mydomain.com [NC]
RewriteCond %{HTTP_REFERER} !^http://www.anotherdomain.com [NC]
RewriteCond %{HTTP_REFERER} !^http://anotherdomain.com [NC]
RewriteCond %{HTTP_REFERER} !^http://www.anotherdomain2.com [NC]
RewriteCond %{HTTP_REFERER} !^http://anotherdomain2.com [NC]
RewriteCond %{HTTP_REFERER} !^http://www.anotherdomain3.com [NC]
RewriteCond %{HTTP_REFERER} !^http://anotherdomain3.com [NC]
RewriteCond %{HTTP_REFERER} !^http://www.google.com [NC]
RewriteCond %{HTTP_REFERER} !^http://google.com [NC]
RewriteRule ^.+\.(gif|jpg|png|swf|mpg|avi)$ http://www.mydomain.com/img/hotlinked.gif [NC,F,L]
ErrorDocument 404 http://www.mydomain.com/
ErrorDocument 403 http://www.mydomain.com/

Is it correct to have both a deny and an allow in the same htaccess?

Would just load this bad boy up and try it out...but really would prefer not to crash my server...lol!

Thanks again everyone...really appreciate the assist