Please Pull any links to...

[walrus]

XXX Blog Index, XXX Blog Maniac (Porn Tags), Porn Blog Surfer and Pure Porn Fed.

I'm really sorry to be doing this but it seems I have little choice.

I've got 4 mega-blogs that unfortunately continue to melt down servers. It started about 4 years ago and was probably the cause of my major server data base loss of a couple years ago. Each blog has over 10k in posts, both hand written and RSS and get between 4-6k uniques a day, depending on the site. I've been through two different and respected hosts and more hosting plans than one can shake a stick at including dedicated server packages but everytime I get things stable I've got 3 months of peace and then the hell starts all over again. I've been through WP-Cache, super-cache, Xcache, memcached, eAccelerator and just about anything else you can think of that is supposed to help stabilize large WP blogs.

I've spent way to much time with this and everything else I've been trying to do has suffered for it. I'm going to work on stabilizing my remaining blogs, moving things like galleries out of the blog and as separate pages from the blog, build more sales pages and SEO pages and simply work on building back up my traffic network with the balance of that traffic not being to a blogging script and rather than running the traffic through the blog I'll use the blog as a traffic source for my other projects.

Sorry about the inconvenience but I really don't see where I have much of a choice.

[Mr Spock]

Sorry to that Walrus , I think we have exchanged links in the past - will look and pull them today.

Seems like what you are saying is that if one becomes too successful with your blogging you'll become a victim of your own success.

[pc]

Totally Understandable Walrus.
I pulling your links down.

[RedCherry]

I'll pull your links down too. Why do large blogs do this, is it because they are all php and the constant queries melt the server down, or use up all the cache?

I know when I posted up hun galleries on my domains with a blog in the root, just running through the standard .htaccess for every hit due to the permalink rewrite caused my apache to keep crashing. Is it from something like this, or just from blog traffic?

Curious, because I don't want to be in your shoes a year down the road if the blogs we have become large and successful.

[balls_deep]

I'm sorry to hear that walrus, I've pulled your links.

If and when you get things back to speed let me know and we'll link back up.

[Simon]

Damn, that sucks, Walrus.

Seems there should be some kind of script which could make static copies of those 40K post pages so you could keep them all online and monetize them with not much server hit at all.

Otherwise it really sucks even worse.

[Ms Naughty]

Sorry to hear that Walrus, I'll pull your links

[dunc]

Walrus - did you try to import them into an alternative blog/cms - on the textpattern site it states that the amount of articles/posts has no effect on speed (http://textpattern.com/faq/79/how-ma...pattern-handle) and I know it has a WP import feature

[cd34]

walrus, before you shut down -- your sites can be mirrored with static files so that they would just be archived and could run on a machine with very little impact on cpu. Then, you could figure out a new solution without tossing all of that work out the window.

[Tekster]

Originally Posted by cd34

walrus, before you shut down -- your sites can be mirrored with static files so that they would just be archived and could run on a machine with very little impact on cpu. Then, you could figure out a new solution without tossing all of that work out the window.

How do you do that? The mirrored part?

[walrus]

Originally Posted by cd34

walrus, before you shut down -- your sites can be mirrored with static files so that they would just be archived and could run on a machine with very little impact on cpu. Then, you could figure out a new solution without tossing all of that work out the window.

Ok, I've got everything still implace, how would one go about mirroring to static files?

[Useless]

I think a common website ripper would make static copies.

[cd34]

any site ripper would work

if you're comfortable with wget, you could do something like:

wget -r -l 99 http://site.com/

you MIGHT try it without adding -F to the command line which will put in <base href> for relative urls

--limit-rate=300k would limit the fetch to 300k/second... if you do one site at a time, depending on the page composition, that might be enough to keep from killing the server when it spiders everything.

It will put it in a directory structure which you can shuffle around. If a page is not linked to, it won't get spidered here, but, you can supply an input file of urls, so, if you have a sitemap, you could parse that to pull the pages not internally crosslinked. With wordpress, I don't think this is a problem.

[walrus]

I'll give it a shot, but probably not for a couple days. I would love to have this work, the sites might not be dynamic but at least they would be alive.

[Jeremy]

Originally Posted by cd34

walrus, before you shut down -- your sites can be mirrored with static files so that they would just be archived and could run on a machine with very little impact on cpu. Then, you could figure out a new solution without tossing all of that work out the window.

Make sure you give this a shot Walrus. There really is bugger all point in throwing away 4 years of work.

cd34 (Chris) really, really, really knows his onions. (I'd also reccommend his hosting, colo-cation.com, in a heartbeat).

[NY Jester]

Damn, Walrus, hope that there is a solution for you, other than shutting them down.

[bDok]

No clue, but it would be nice if there was something that could turn a wordpress site into a static site. So at least you could keep the site up and running.

[walrus]

Originally Posted by Jeremy

Make sure you give this a shot Walrus. There really is bugger all point in throwing away 4 years of work.

cd34 (Chris) really, really, really knows his onions. (I'd also reccommend his hosting, colo-cation.com, in a heartbeat).

Trust me, the thought of throwing away 4 years of work was not an enticing thought so if a site ripper can do the trick, I sure the fuck am going to give it a real try.

[Tekster]

may want to try this walrus: http://www.httrack.com/

[digifan]

I'll miss your pingbacks...