Ultimate HtAccess file?
 

Howdy all,

It has recently come to my attention that while there are plenty of hints and tips about securing your site with htaccess scripting, there really isn't one place you can go and find the be-all-end-all screaming best of the best htaccess file.

If there is any interest here, I will modify ours and work up a generic template that can be used to turn |pcsucks| into |bananna|. What do you say?

I could use some input on what would make it more useful to you and people you would refer to it.

Mr. H.

I am all ears Mr Hackula. :D

MrHackula you have my attention |rasta|

Sorry, can I just go off at a tangent here.....

.htaccess for redirection (RewriteCond %{HTTP_REFERER}) goes into a loop when???

When redirecting to the same domain?

eg:

www.site.com/index.html ---(redirect)--> www.site.com/otherpage.html

or does this prevent it?

www.site.com/index.html ---(redirect)--> www.site.com/other/index.html

TIA coz I cannot remember. :)

Always interested in learning more about htaccess. :)

yep!! :) do it! |goodnight|

Urb depends on how its set up? If you are trying to "redirect" them to the same DIR.. then it aint gonna work.. so the second one should work...

Hello Mrs H... |waves|

DD

I am working it up as we speak. As for the issue you present here... The key is to remember that .htaccess is primarily designed to protect directory contents. Files in the same directory are difficult (not impossible) to protect using .htaccess.

The second example works but ONLY if you have another .htaccess file in the "/other/" directory. That .htaccess file should contain the line:

RewriteEngine off

...for maximum compatibility with scripts and such.


Hope that helps,

Mr. H.

Howdy Dave! |shake|

Mrs. H sends a big ol' hug your way. :)

Good to see you again. Missed you since the LM incident when I had to walk away from the biz for awhile. All better now. :)

Take care,

Mr. H.


|shake|

Howdy All,

Please look over my list and see if there are any user agents you would like to see banned that I don't already have listed. Post whatever you would like added and info on what it is if it isn't obvious.

Thank you,

Mr. H.

P.S. I realize the list is a bit redundant. All that will be fixed before final release.


RewriteCond %{HTTP_USER_AGENT} ^.*Aculinx.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Ants.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*attach.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Backstreet.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BackWeb.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Backweb.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bandit.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BlackWidow.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Buddy.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Burner.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Caitoo.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Collector.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Commander.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Copier.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Crawler.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Curl.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Devil.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Down2Web.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Downloader.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*DownloadIt.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Eater.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Eclipt.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Enterprise.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Express.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Extractor.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FileHound.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FlashSite.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FlipBrowser.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetBot.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetSmart.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GetWeb.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*gotit.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Grab.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Grabber.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Greed.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Hoover.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HtGet.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*iFox98.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*IPhoto.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JOC.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Keepoint.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*leech.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Likse.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*likse.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Likse.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mag-Net.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Magnet.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Memo.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*MemoWeb.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mirror.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*NetDrag.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Pump.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Reaper.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Recorder.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Retriever.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Siphon.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Snagger.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Snake.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Snarf.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Snatcher.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Stripper.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Sucker.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Vacuum.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Vampire.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Weazel.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Webexe.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebVCR.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Whacker.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^[Ww]eb[Bb]andit [OR]
RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^CherryPicker [OR]
RewriteCond %{HTTP_USER_AGENT} ^CherryPickerElite [OR]
RewriteCond %{HTTP_USER_AGENT} ^CherryPickerSE [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^Crescent [OR]
RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DA\ 4.0.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^DA\ 5.0.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^DA\ 5.3.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo\ Pump.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
RewriteCond %{HTTP_USER_AGENT} ^Drip.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Gets.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^GornKer [OR]
RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grafula.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
RewriteCond %{HTTP_USER_AGENT} ^ia_archive [OR]
RewriteCond %{HTTP_USER_AGENT} ^ia_archiver.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^IBrowse.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^InterGET.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Irvine [OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar.* [OR]

RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^JustView.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^lftp.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^Microsoft.URL [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*NEWT [OR]
RewriteCond %{HTTP_USER_AGENT} ^MSFrontPage [OR]
RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
RewriteCond %{HTTP_USER_AGENT} ^NearSite.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^NICErsPRO [OR]
RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]
RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
RewriteCond %{HTTP_USER_AGENT} ^Pockey.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^ReGet.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^SearchExpress [OR]
RewriteCond %{HTTP_USER_AGENT} ^Siphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]
RewriteCond %{HTTP_USER_AGENT} ^Slurp.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^SpaceBison.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^Telesoft [OR]
RewriteCond %{HTTP_USER_AGENT} ^TV33_Mercator [OR]
RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector.*[OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebAuto.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebBandit [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier.* [OR]

RewriteCond %{HTTP_USER_AGENT} ^WebEMailExtrac.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebFetch.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger\ 1.20b.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^Webster.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus [OR]
RewriteCond %{HTTP_USER_AGENT} ^ZyBorg [OR]
RewriteCond %{HTTP_USER_AGENT} dloader(NaverRobot) [OR]
RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]

Thank you DangerDave and Mr. Hackula :)

Never have I seen a list of user agents this extensive before. :)

Thanks!!! Would you mind sharing a bit of hotlink protection in it as well? I'm fairly new to this. |roses| Thanks.

There's good .htaccess anti-hotlink code here http://www.cleosxxxlinks.com/adult-l...ster-help.html

Just place it in your images folder, so you can keep your domain root clear for other .htaccess stuff.

Thanks a TON urb! I really appreciate all the help from you guys. Hoorah for lower BW bills! |afro|

LUV

Hello PeakPorn,

Looks like URB already pointed you in the right direction, but I wanted to clear something up: That post wasn't a complete htaccess file. All I am doing there is asking if anyone else knows of user agents that I don't already have listed.

A complete file will be forthcoming in the next week.

Thanks,

Mr. H.

|shake| Right, I understood. Also "no warranties, written or implied" etc. etc. etc. :) Thanks for the info though, great stuff.

LUV

MrHackula - great stuff - got here looking for a way to script unwanted domains to be redirected to another domain or page of choice.
An example would be - if I have a tgp gallery posted that gets listed on worldsex.com, and lets say www.jp-sex.com links to that gallery and I didnt submit it to them and really dont want their surfers eating my bandwidth without being able to sell them something, I'd like to send them to another page that maybe includes some console and dialer chains :) they aren't hotlinking images just linking to a valid url that gets posted on one of the major TGPs. I just can't figure a way to do this without affecting the other TGPs I do want to have access?

Question about the anti-hotlinking...if the surfer has firewall software running that won't pass the referer, will he be pushed to your rewriterule page?

I use the referer info for a shared members content area & have had problems with this. Since they're already members, it's easy enough to send them to a page with instructions for changing their security settings. But for submitting TGP galleries, that's another story.

Thanks!

Hello Linkster,

Thanks for the kind words, but I haven't put the file together yet... :)

Let's see if I can figure out a way to post this here without the board tweaking it:

RewriteEngine on
RewriteOptions inherit
RewriteCond %{HTTP_REFERER} ^.*invisionfree.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^.*chatropolis.*$ [NC]
RewriteRule /* http://www.someothershit.com/ [R,L]

Just add the TGPs you dislike to the above code. Copy the fist line, the one with the [OR] on the end, and make as many entries as you like.

I suggest that you use a wildcard similar to what I have here. If you are being hijacked by tgp.russianscumsucker.com and you add this line:

... what are the odds that you will catch something you didn't want to nuke? ;)

Don't forget to trim any spaces off the end of the lines, as the board likes to add them.

Hope this addresses what you were hoping to achieve.

Let me know,

Mr. H.

Howdy KCat,

By saying who can NOT go some place you don't effect anyone but the people you specifically say. In the above example blank referrers go right on through. If you are doing the standard "Allow people from my domain and my IP, deny all others thing" you can add this line:

RewriteCond %{HTTP_REFERER} !^$

... this SHOULD allow people without any referer to pass.

I do NOT recommend this, as scripts and search engines will waltz right in if this is the only protection you are using.

Hope this is helpful,

Mr. H.

I want to allow most scripts since the TGPs will often send out their automated scripts before listing a gallery (checking recips, links out etc.). SEs are fine too because I keep my recip galleries on a throwaway NOINDEX,NOFOLLOW domain anyways.

But the reaper programs like Webreaper & the hotlinkers I'd like to boot. |pissed|

MrHackula - thanks :) just one other question
If I use this htaccess on the root of the domain will it work to protect all galleries on that domain, or should I do it on a directory basis?

BTW - this is the little file that Im going to use (warning - dont follow the url on the rewrite :)

RewriteEngine on
RewriteOptions inherit
RewriteCond %{HTTP_REFERER} ^.*jp-sex.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^.*chatropolis.*$ [NC]
RewriteRule /* http://www.porno-free-sex.com/ [R,L]

Look ok?

Howdy Linkster,

Looks good from here. :)

That htaccess file will have control over all directories below it in the tree. In other words all it's subdirectories will be covered. The exception is having another htaccess file somewhere that has controls which run contrary to this one. If you don't already have a buch of miscellaneous htaccess files running aroud you should be just fine to put it in your root.

Glad I could help,

Mr. H.