The last few months since getting my own private hosting have been quite challenging in a number of ways as I have had to adapt my handling of my sites to a number of new contingencies: including weird spikes in server activity. It’s been quite a learning experience, but I’m pretty sure I don’t want to go back to regular shared hosting! I love having my own server on Dreamhost…

——-
For more articles on running a business, making money, cutting debt, or creating your blog, subscribe to the RSS feed or email newsletter. There’s a lot more in the Random Walk to Wealth on InvestorBlogger dot com. Subscribe TODAY!
——-

This graph is taken from December 5th and shows four significant spikes in server traffic in the previous 24 hours, with the spike around 00:00 being the largest and also bringing the server down!

Dreamhost helpdesk have VERY helpful to me over the years, and patiently answered almost every query fully and in detail. The last few times though have been frustrating because the amount of activity has been quite high on occasion (without obvious justification either!), and they were suggesting that perhaps I should raise the allocation of resources (and the cost!) to cover the spikes! Now, I’d already tried this once or twice: having raised my basis from $15.00 to $20.00 to $30.00. While this did indeed improve my server responsiveness, I was still getting server outages every day about once or twice a day, and it just didn’t seem that my server traffic justified that!

What is a poor server to do?

Go on a bug hunt! It’s that simple. I decided to find out what was going on, and see if I couldn’t eliminate all the other possibililties before increasing the expense further…!

Step 1. Find the Crap…

I was amazed how much crap had built up on my servers over the three years I’ve been using Dreamhost hosting… It was actually quite shocking. And it took me days to clear. But first I took stock of the files and data that were on my server: html files, jpegs, mp3s (not many), php, applications, data, backups, gzips, etc… Once I determined which domains and subdomains were only serving files, I could eliminate those pretty easily. Then I went on a hunt for old PHP applications, principally old versions of WordPress, Galleries, etc..

Step 2. Three Choices: Choose NOW!

On finding an application, I was faced with three choices: upgrade the software to the current version; remove the software entirely; or password the entire directory. In most cases, I chose the second option to cut down on the security risk. On one or two occasions, I passworded entire subdomains (with .htaccess and .htpasswd files) as well to provide access to these applications only to the person concerned. In the cases of my primary websites, upgrading became mandatory. In rare occasions, I moved applications to the root directory as a stop gap measure.

From now on, though, I’m going to be much stricter on removing old applications because of the potential security problems that can be hidden by forgotten software.

Step 3. Deleting unnecessary plugins and themes

Since most of my blogs run versions of WordPress 2.2+ and I usually auto-install or auto-update the files (courtesy of Dreamhost’s great service), the process is quite quick and painless. Dreamhost have gotten it down to a fine art, such that upgrading involves backing up the Database, Turning off the plugins (one click), click update, and wait for confirmation. But each and every time I update Dreamhost creates a backup of the original files named domain.old, and throws in EVERY theme they can get their hands on with each install). This leads to a lot of stuff each time. So I usually now delete 95% of the themes, and maintain a lean installation with as few plugins as possible (and none spare). This helps make WP as responsive as possible.

But as you can see I was still getting server spikes, and my options were running out. Everything was eliminated in one way or another, and the server was still coming down. So what next?

Step 4. Is it time to ‘log’ out?

I compared the performance of my main blog with several other blogs by checking the error.logs (I had never checked these before) just to see what was going on. Surprisingly, I found the logs quite detailed (they’re in your FTP folder called ‘logs’… click through to your domain to find the logs and then look for the error logs. They were dated a couple of days back so I had to check through and this is what I saw:

That’s right, line after line of the same error: this read “(12)Cannot allocate memory: couldn’t spawn child process: /dh/cgi-system/php5.cgi” and “(12)Cannot allocate memory: couldn’t create child process: …” To make sure that I wasn’t just reading regular errors, I queried it on Google and got nothing, then I checked the logs of Wordpress Sites that I had just started with no themes or plugins. Sure enough, everything looked fine. Then I remember something a Dreamhost engineer had said:

Just the other day, I was dealing with a fellow who had a wordpress blog who was almost instantly crashing his PS with a single hit, because he had some wonky wordpress plugin which was spawning hundreds of php.cgi processes. Jeff

Well, thanks, Jeff. That has proved to be the best advice I’ve received yet… So I’ve been testing different plugins to find out which one or ones have been causing the problems. The best way to do this is to simply turn off all the plugins to restore the neutral state and then turn them on singly and checking the error log to eliminate those that don’t create the problem. I think I have a culprit already, but I’m now double checking them to make sure that I don’t blame the wrong plugin. Think ‘sticky’…

While the testing goes on, please excuse me for having unstable features on my blog as I try to eliminate the problem. But as you know, this problem has been going on for months; and this represents my best hope to get to the root of the problem.

Have you had these problems? What happened? I am not particularly an expert, but I was surprised at how much I could figure out by myself!