Protecting the video - EXAMPLE

[RawAlex]

plateman, 100 different IPs is pretty normal for a single hotlink source. Looking at IP addresses to determine if someone is a hotlinker isn't really a good way to do things. You could end up shutting down a major ISP because 3 or 4 people happen to hit the gallery at the same time.

I would be much more interested to see the SOURCE of the links... where are these people all coming from?

I had one the other day from a japanese chatroom. On IPs, I probably got hit by 5000 different IPS in an hour. But the SOURCE of the hits was one place. One minor adjustment to a global htaccess, and they are not getting there anymore.

Most people aren't just randomly typing in your address - they are coming from somewhere. Blocking that referal source is an important step to removing your hotlinking issues.

Alex

[ClickBuster]

Originally Posted by RawAlex

Clickbuster, you are suggesting it could handle 240,000 requests a minute on that size of a server? I don't think so. I think you got about two zeros too many in there...

Plateman, are the majority of your hotlinks from one or two sources?

Alex

Dude, are you hostile or what! You should read what I wrote again - it says 4000 simultaneus connections. That means that there're 4000 users downloading the movie in the same moment. Never said a thing about 4000 requests - not the same thing. And 240k requests in a minute will most probably kill many of the services there - it's like a DoS And don't get on my like this Alex, I just proposed a custom, home made solution for people that try to protect themselves from hotlinking - if you have something against sharing - let me know about it

Andrew

[RawAlex]

Clickbuster, I never have anything about sharing - what I have is something against false hope. I am not getting on you, I am poking at the idea and saying that I don't think it will work like you are suggesting.

PHP isn't made for, isn't robust enough, and doesn't have the thruput to truly handle what you are suggesting. PHP is an interpreted language, not a compiled one, so it runs basically in a container, which has much higher overhead than cgi. Yes, it can be done the way you suggest, but I seriously doubt the scale you are talking about. Basically you will be anywhere from double to quad hitting the central bus (depending on if PHP is using disk cache during the operation), and you will be limited specifically by the amount of bus time available. Remember, processing files in this manner means READ (bus) PROCESS (bus) possibly CACHE (bus X 2) output to NIC card (bus) - plus all the underlying actual file handling going on with the OS. You could have run the same file across the bus 10 - 20 times before it gets sent out.

Certain banner rotator programs use PHP to pass the banner image, and that alone is enough to grind many servers to the point of being useless. Using PHP to pass 1 meg or more per process for 4000 processes is just not very likely. I think you need to figure out how long it would take for each movie to be delivered from a server with a 10mps line (which probably passes closer to 8mps)? (or Sparky might chime in here with the calc...) My rough guess is this: 8000kbs/ 4000 users is 2kbs, which means 500 seconds per 1 meg file - and that assumes your server ain't doing anything else.

Remember, it's not the person, but the ideas - I think you are a smart guy looking for a better solution - but you need think the implications through before proposing this sort of solution.

Keep trying - your on sort of the right track, just using the wrong tools.

Alex

[cd34]

I looked at the script at the beginning of the thread -- it looks to me like you are using a cookie to prevent hotlinking of .wmv files. Preventing direct access to the file through .htaccess, requiring the file to be sent through php. One potential issue here is that if it takes more than 30 seconds to download the file, its possible that you'll exceed the max_execution_time default in php.ini and the connection would be closed prematurely. Also, since it is a php file, if your host has mod_gzip or mod_deflate configured based on file extension rather than mimetype, you'll corrupt data.

I'd have to dig through the php sources, but, I believe fpassthru uses the kernel sendfile, which, uses a zerocopy buffer on linux (not on Freebsd) -- that in itself would be almost as efficient as using pure apache. I don't know if readfile does, but, its possible.

The 80% duty cycle on ethernet is a bit high, but, considering a full duplex connection, and no transmission delays, 4000 users would quite easily be 500 seconds to download, but, lets consider that 4000 users is probably atypical and that you wouldn't reach that because it would probably exceed Apache's maxclients prior to that. I think the max I have ever seen maxclients set to has been 2048 on a public webserver and is typically 512 or less.

However, your method does bring one thing to mind. You mention .htaccess to protect your videos with a deny from all. This gives me two ideas, one of which replicates what I think you're trying to do, the other being a bit off the beaten path.

With mod_rewrite, you do have %{HTTP_COOKIE}, so, you really could write a rule that checks for the existence of a cookie & value to make sure they can download, and that would avoid php altogether. You'd set the cookie on your page using javascript or a .php file parsed that sets the cookie. Then, apache's mod_rewrite would check for the cookie and decide to serve. This of course eliminates those people using privacy software that don't accept cookies.

There is another way you could do it using php which is a bit different.

If you serve the gallery, you have the IP address of the surfer (or at least the publically available address that they would be sucking the file down from anyhow). You could put the page in a directory and the videos in a subdirectory. When the page is served, you would take the current surfer's IP and append it to the end of the .htaccess in the videos subdirectory along with some timestamp so you can clean it later. If you aren't using php to serve the page that the surfer sees, and I usually dislike using dynamic pages, you could wrap an img src so that it would write the file.

so you end up creating a .htaccess in your videos directory like:

order deny,allow
deny from all
allow from 1.2.3.4 ; timestamp
allow from 3.2.4.5 ; timestamp
allow from 4.5.6.2 ; timestamp

At this point, again, you have apache processing only your initial pageload, and then apache does the work of sending the files. Then once every few hours you clean out the .htaccess using the timestamps to expire 'old' entries.

You could write a mod_rewrite that uses an external ruleset to eliminate the read-write. You really need to figure out the duty cycle for the script and how many downloads/day you're really talking about to figure out which approach will work best.

If you get a Hun listing, the more processing you must do to determine how to prevent the hotlinking, the more trouble you'll end up with.

[Useless]

Um, I was going to say that, but Sparky got to it first.

|confused|

[ClickBuster]

Alex, fuck the resources! Lets figure this out and we'll see what happens next and if it works we will optimize.

The problems here:
a. This has to be easy to integrate
b. It should allow regular surfers to open the file and to disable access for hotlink redirects
c. If possible, it should open directly in the player the surfer is using

The best thing that cames to my mind is to log visits on every page and check logs when it the times come to send the video. The links should point to mod_rewrite accessed script, that way it will look as a regular URL and none of the LLs would mention it and make a problem out of it as it happened a couple of times with my sites. And I think that's it the simpliest way to explain this. Let me know if you want me to go further

Andrew

[RawAlex]

Andrew, resources are always key to any programming discussion, as you don't have infinite cycles, infinite memory, or infinate network access at your disposal. In simple terms, you can't stuff 10 pounds of shit into a 5 pound bag. You have to be EXTREMELY careful to avoid making your system vunerable by making it work too hard or handle too much.

You also don't want to use something that is remote user dependant. Cookies are being declined more and more by end users, which means you could end up refusing more and more valid surfers.

Have you considered putting your videos in a directory that in NOT accessable from the outside at all (not in the domain, but in a seperate folder somewhere on the machine) using htaccess to redirect all regular requests of a small cgi to check refering page and such? That cgi could then serve the file transparently, and it would never be accessable from the outside (no direct link except through the CGI).

You could probably do the same in PHP, but it would be way less effecient, I suspect.

Alex

[Opti]

Originally Posted by cd34

There is another way you could do it using php which is a bit different.

If you serve the gallery, you have the IP address of the surfer (or at least the publically available address that they would be sucking the file down from anyhow). You could put the page in a directory and the videos in a subdirectory. When the page is served, you would take the current surfer's IP and append it to the end of the .htaccess in the videos subdirectory along with some timestamp so you can clean it later. If you aren't using php to serve the page that the surfer sees, and I usually dislike using dynamic pages, you could wrap an img src so that it would write the file.
..........................

CD or Andrew or Anyone.. does this sound feasible as a solution?

To make sure no one can view the video without having viewed your site first.. When the video file is requested, create a sym link on the fly, probably an md5 hash of their IP and browser agent.

Then just delete all symlinks every hour or day or whatever period to suit the situation.

(that's a simplified description but hopefully you see where I am coming from and know if it's feasible)


PS.. you guys should be careful Under-Estimating young Andrew.. you might be surprised to find out what and who he has been employed to code for in the past. ;-)


*edit:

Originally Posted by Alex

Have you considered putting your videos in a directory that in NOT accessable from the outside at all (not in the domain, but in a seperate folder somewhere on the machine) using htaccess to redirect all regular requests of a small cgi to check refering page and such? That cgi could then serve the file transparently, and it would never be accessable from the outside (no direct link except through the CGI).

oops.. missed this... yeah something like that.. but without the CGI or extra checks and stuff. (which shouldnt be needed imho.. as we only want to make sure they see our website to view ads before seeing the vid as the basic function)

[cd34]

I would probably suggest you do that with the hidden directory -- that way, you're not creating 6 symlinks for each pageload.

Workable, yes.

Another possibility is generating a hash on the fly that would be verified by some filter module -- of course, hosting this on a server where you can't install a module might be a bit difficult. But, if you display the page with some hash in the filename, your filter module could determine if the hash was valid and serve the file. You MIGHT be able to do this in mod_rewrite with some creative regexp's that look for a particular key.

Many many ways to do it. The question is, which way uses the fewest resources and will hold up to real-world exposure.

[Opti]

Ideas everywhere... so who is going to code the "perfect" solution for the rest of us?

If 20 of us that want to use this all throw in 10 bucks each.. $200 should buy a good coder's time to put it together and test it properly.

Give it away but code it to redirect hotlinkers to an a fixed page (or load a video advert) which the webmaster can't alter... unless they register for a "licence key" here at GGandJim.

The people who contribute cash for the coding own a share of the default surfer messages... and the board gets the viral marketing effect for webmaster traffic.

[ClickBuster]

Ah... I dream for the days when Im actually going to pay somebody to code for me...

[airdick]

Originally Posted by Opti

Ideas everywhere... so who is going to code the "perfect" solution for the rest of us?

If 20 of us that want to use this all throw in 10 bucks each.. $200 should buy a good coder's time to put it together and test it properly.

Give it away but code it to redirect hotlinkers to an a fixed page (or load a video advert) which the webmaster can't alter... unless they register for a "licence key" here at GGandJim.

The people who contribute cash for the coding own a share of the default surfer messages... and the board gets the viral marketing effect for webmaster traffic.

This works for me:
http://www.antihotlinking.com/

Installs on Apache in about 5 minutes, doesn't use cookies, no php, doesn't seem to put a big load on the server, although I'll admit that I only push an average of 1.5Mbps with the occasional bursts to around 6Mbps. Run a script to install the software, make a few edits in your apache config, restart apache and you're done. It's a set-and-forget type of deal.

There might be better solutions if you're not limited to following TGP/Linklist rules, but if you're a submitter that needs to protect movies this product is money well spent.

[plateman]

Originally Posted by airdick

This works for me:
http://www.antihotlinking.com/

Installs on Apache in about 5 minutes, doesn't use cookies, no php, doesn't seem to put a big load on the server, although I'll admit that I only push an average of 1.5Mbps with the occasional bursts to around 6Mbps. Run a script to install the software, make a few edits in your apache config, restart apache and you're done. It's a set-and-forget type of deal.

There might be better solutions if you're not limited to following TGP/Linklist rules, but if you're a submitter that needs to protect movies this product is money well spent.

-thats what I use and love it..

[Opti]

Originally Posted by airdick

This works for me:
http://www.antihotlinking.com/

Installs on Apache in about 5 minutes, doesn't use cookies, no php, doesn't seem to put a big load on the server, although I'll admit that I only push an average of 1.5Mbps with the occasional bursts to around 6Mbps. Run a script to install the software, make a few edits in your apache config, restart apache and you're done. It's a set-and-forget type of deal.

There might be better solutions if you're not limited to following TGP/Linklist rules, but if you're a submitter that needs to protect movies this product is money well spent.

Thanks for the info Airdick... $300 per domain makes it sound like my 200 dollar estimate for a coders time may have been low huh

They do sound like they have pretty much exactly what we would want though!


(they pay a 20% commission for refferals http://www.antihotlinking.com/resellers.php .. you signed up for that Airdick?)

[airdick]

Originally Posted by Opti

Thanks for the info Airdick... $300 per domain makes it sound like my 200 dollar estimate for a coders time may have been low huh

They do sound like they have pretty much exactly what we would want though!

(they pay a 20% commission for refferals http://www.antihotlinking.com/resellers.php .. you signed up for that Airdick?)

The price is per-server rather than per-domain.

I'm not signed up for the affiliate program and it looks either resellers charge a higher price ($249) or the reseller page hasn't been updated.

[Cleo]

Just seems like using mpegs is so much easier then all the hoops that you all are going through to use wmv files.

[cd34]

There are advantages to .wmv files that .mpeg files don't have. I think most people prefer the .wmv size versus quality, although, for small files, I haven't seen a huge difference.

However, the particular reason I am thinking of, I would love to have people hotlinking my videos. Heck, I would even host sites that got the videos downloaded.