Server Setup

[cellinis]

Hi

This one is for guys who are at ease with the sysadmin stuff.

I have a machine that basically hosts a single site for mgps. However, under extreme load conditions, the server stops responding. Searching for solutions, I came up with the following:
Use thttpd (or similar small server) to serve videos/images/html) and apache for php based tasks for maximum output.
Now my site is divided into two parts - maindomain that hosts static html pages with movies etc, and a subdomain that hosts a link tracking script using php/mysql.

The idea would be to put maindomain on thttpd server and subdomain on apache server.

Questions:
1. Has anyone tried it? Did it really increase the performance of the machine?
2. Anyone knows of where I can find some tuts on doing this?
3. How much ram should ideally be required for serving 130-180k uniques per day for MGP galleries? I am currently on 2GB; and I thought that it should be enough...

[cellinis]

In case it ain't clear, the site is hosted on a dedicated box and is the only site hosted on this box.

[Cleo]

This is why I always get managed hosting. I can just send an e-mail off to my host that says make it work and so that I can go about getting traffic and making sales.

[cellinis]

Hey Cleo!
I'm on managed hosting too. But I want to get a second opinion on this one plus take a look at other available options myself. Their solution is simple, increase RAM, increase CPU power... but then I remember CD34 (that's his code right?) graph on the P3 handling 100MBPS, and I just can't figure out why my box shouldn't handle the same loads too.
This secondary server has been hogging a lot of my time lately, that is fucking true... with all the other things I must do in tandem...

[Cleo]

Like I posted in another thread, CD34 makes servers do amazing things. I'm still in awe at how fast my stuff is now that I moved to his hosting.

[cellinis]

So Nows a good time for me to ask... what's his company site? I tried looking up on his provile, couldn't find anything.

[Cleo]

Originally Posted by cellinis

So Nows a good time for me to ask... what's his company site? I tried looking up on his provile, couldn't find anything.

You would think he would have it is his sig but since he doesn't I have it in mine.

Click my link list button or use the drop down menu in my sig.

[Useless]

Originally Posted by Cleo

You would think he would have it is his sig but since he doesn't I have it in mine.

Maybe he should change his sig to say "PM Useless Warrior and ask him for my hosting URL" |cool|

[Cleo]

I fixed me sig

But I still love the HPA on my submit page.

[cd34]

Apache can handle 20mb/sec without problem. you don't mention which version of Apache, but, if you are running Apache2, mpm-worker is much faster than mpm-prefork if you are doing just static hosting. Of course, mpm-prefork is required for php to run as a module, but, mpm-worker + fastcgi/php is actually faster than mpm-prefork. You say the machine hung, but, that can mean different things. Did it slow to a crawl and have a load of 80+, or did the machine require a reboot that worked for a while afterwards?

thttpd only runs on one thread, so, if you are using HyperThreading and have a dual cpu, all requests will get routed to one cpu -- if you haven't masked your netcard irq to the same cpu that thttpd happens to run on, you could run into other performance issues. boa is also quite fast, but, again, single threaded. Dan Kegel talks a lot about web performance in his C10K paper. Apache also has a series of modifications called the Accelerating Apache project. Most of these are patches that Apache turned down because they broke functionality in the quest for speed. The patches are old, but, none of the issues addressed in the 'SGI' patches have ever been 'fixed'.

If you're looking for all out speed, and you're talking a usermode process, Fnord with dietlibc is very fast as is Chromium. If you don't mind kernel modifications, Tux is what I would suggest.

However, back to your initial Apache issue.

Apache was built to serve reliably, and then quickly. The default configs of most apache installations are sometimes quite bad. The other thing you need to do is identify WHY apache bogs down. When the machine gets slow, are processes waiting on IO, or is the machine CPU bound? It would be doubtful that any machine nowdays would be bound by interrupts.

ExtendedStatus inserts timestamp checks throughout the process -- you can see the obvious detriment here. That is usually one of the first things to go.
If you use mod_rewrite on heavily used pages, move that to your apache config and turn off Overrides. Check to make sure that AllowOverride all isn't set at the root directory (this can kill performance on busy sites). Make sure you don't have any deny from .domain.com in your .htaccess and that reverse dns resolution is turned off in apache. strace one of the child pids to see what it is doing. Then you can see where the bottleneck is.

Or, PM Useless Warrior and ask him for my hosting URL.

http://www.boa.org/ (I always found it ironic this site was hosted on apache)
http://www.kegel.com/c10k.html
http://aap.sourceforge.net/
http://www.fefe.de/fnord/
http://chromium.com/x15.html
http://www.redhat.com/docs/manuals/t...ual/intro.html

and last but not least

http://colo-cation.com/

[cellinis]

Cd34, that's a lot of informative links First I'll have to go over them and look up the patches you talked about before responding informatively to your post.
In short:
It is 1.3.33 Apache (I didn't upgrade to 2)
It is a single celeron CPU.
I played around with the default configs on Apache a bit, but didn't take it overboard since I am hosting a production site on this server. The server is set to max connections of 312. Anything over it makes it unstable.
I haven't bothered with htaccess on this domain other than hotlink protection for movie files... and I'll turn off the reverse dns solution. ExtendedStatus is in the httpd.conf, right? I'll check it out too.

Plus, with your permission, I'll also forward your post to my current host and do a bit of reading on my own.

[cellinis]

When I said hung, I meant it slowed to a crawl. Upon a reboot, it works fine for about a minute (the time the connections start comming) and then gets bogged down again. On kernal modifications or usermode processes, I don't have any preferences, whatever gets my pages loaded faster with minimal work (I'm lazy lol). Tux seems a tedious installation but let me read up on it.
If my host doesn't come up with a solution by tonight, then I'll have a decision to make.... should I use Cleo's link or should I ask Useless Warrior... or should I just click on colo-cation link you provided

[cd34]

you can pass it on.

Use Cleo's link -- she'll get the referral credit