How to Handle Thousands of Users with Your Blog
There have been multiple occasions where I am trying to read an article and the website is down. I have encountered this multiple times where people say something like: “I got around 13,000 hits and server couldn’t handle it”. There are even websites that become popular by making the content of the popular article available. One of these cases is DuggMirror, a service that mirrors Digg articles and makes them available in case the source is unavailable. If you are running a blog and you have your own server, here are a few tips to easily handle at least half a million hits per day.
- Install Linux in your server. Linux runs pretty good and there are a lot of tools to support the server administration.
- You will need a web server. A lot of users install Apache but I strongly suggest to use Lighttpd. Your blog will usually have images, CSS files, and Javascript files. Apache usually in general doesn’t handle static files as well as Lighttpd. Lighttpd is an event-driven web server that minimizes the blocking waiting for IO.
- Your blog will need PHP and MySQL. Make sure you have those running.
- Install eAccelerator - By default, every time a PHP script is accessed, the web server must compile the script then execute the compiled code. This task is not needed if the PHP script doesn’t change. What a eAccelerator does is it optimizes the scripts, compiles them, then caches them in a compiled state.
- Install WordPress - This blogging software a lot or all of the features you need. It also scales well when having a lot of traffic.
- Install the WP-Cache plugin - Your requests will live in a database and every time you get a new visitor, WordPress will have to retrieve this content from the database. If you get 30,000 visitors, your blogging software will have to do the same query that many times. WP-Cache prevents by caching to disk whatever is in your database so that it can eliminate the need to query the database that many times.
- Create a memory file system - WP-Cache caches its files on disk. We know that memory access is much faster than disk access. Creating a memory file system (tmpfs) and putting WP-Cache files in it, will tremendously increase the speed of retrieving cache files.
- If you installed WordPress on something like /var/www/myblog and wanted to have a memory file system of 100MB you can execute the following command to mount a tmpfs file system where the cache directory resides:
mount -osize=100m -t tmpfs tmpfs /var/www/myblog/wp-content/cache
- You can also do this from /etc/fstab with the following line:
tmpfs /var/www/myblog/wp-content/cache tmpfs defaults,size=100m 0 0
- If you installed WordPress on something like /var/www/myblog and wanted to have a memory file system of 100MB you can execute the following command to mount a tmpfs file system where the cache directory resides:
We currently host this blog and ThemBid.com on the same server with this setup and sometimes gets around half a million hits per day without a problem.
July 17th, 2007 at 10:26 pm
This is very useful information for us starting in the web servers world.
Thanks,
July 17th, 2007 at 11:29 pm
Good information, but it will still cost you higher than a Dedicated Server.
Plus you will have to keep all the packages up to date to worry less about script kidies.
July 17th, 2007 at 11:46 pm
In fact you just need strong hosting provider for your blog. Any good provider have all this things
July 18th, 2007 at 12:02 am
I am interested in what sort of server load you have when you get that many requests in a day, any idea?
July 18th, 2007 at 12:13 am
Mediatemple. Case solved.
July 18th, 2007 at 12:47 am
Cool. I want to know, how to do these (or similar) things for blog, based on python and django!
July 18th, 2007 at 1:58 am
I used blogger.com..no problem with thousands visitor..hehe
July 18th, 2007 at 4:30 am
nice tips. although my blog just visited once or twice per day:D
July 18th, 2007 at 5:26 am
Great guide to setting up a server for Wordpress use. Unfortunately, if you’re not in control of the hosting side you can’t do a lot of these things. Still, if I ever have to set up a server now I’ll be set.
July 18th, 2007 at 6:00 am
Helps you understand what best practices are when you shop for a host, and some of this you can do yourself regardless of host. Note that Wikipedia and YouTube both use Lighttpd on some critical servers.
Still, MediaTemple’s clustered server grid and reported great support looks like a one-stop shop solution for uptime reliability (that I hope to be switching to soon!)
July 18th, 2007 at 7:09 am
# Cucirca Says:
July 17th, 2007 at 11:29 pm
Good information, but it will still cost you higher than a Dedicated Server.
Plus you will have to keep all the packages up to date to worry less about script kidies.
Actually, I run a similar setup on an old P3 box. Doesn’t cost me a thing.
July 18th, 2007 at 7:15 am
Actually, I run a similar setup on an old P3 Box. Doesn’t cost me a thing.
July 18th, 2007 at 7:33 am
WP-cache is the best way to handle hardcore traffic, thanks for the rest of the tips
July 18th, 2007 at 8:16 am
2. Lighttpd is great but it doesn’t support the use of .htaccess files so if you’re using them, you’ll need to rewrite them in lighty’s own format. Lighttpd also requires FastCGI PHP binaries which means that some users may need to recompile PHP.
4. eA is OK but I prefer XCache (http://xcache.lighttpd.net) which was created by the same person who made lighttpd.
7. tmpfs will spill over to disk if your server needs the RAM. In a case of high traffic levels that will almost certainly happen from all of the FastCGI PHP processes that are running if you’re using lighty or httpd processes if you run Apache’s httpd.
July 18th, 2007 at 9:03 am
Or, you could use Horizon Blogging Software, which uses way less resources that WordPress and handles the digg effect a lot more nicely. Even though it’s in beta, it’s a nice package. I’m currently using it on my blog. I think the project page is http://www.edgedrive.com, check it out.
July 18th, 2007 at 9:17 am
I was thinking this was going to be a completely lame article, the tips are pretty good.
July 18th, 2007 at 9:44 am
What kind of server are you running? Specs would be nice.
July 18th, 2007 at 10:29 am
I’ve seen a few people respond saying just get a hosting company and they’ll do this for you, but I don’t think I know a single hosting company that has WP-Cache installed or is pulling that cache from active memory. Those two tips alone will deliver impressive gains.
July 18th, 2007 at 4:28 pm
Good info alot of it that i’m currenty implementing myself.
cheers
Paul
July 18th, 2007 at 4:30 pm
_How_ do you have half a million hits per day? I would learn and maybe pay something for this information..thanks in advance and best regards!..Petra
July 18th, 2007 at 8:06 pm
You should also check out this:
http://elliottback.com/wp/archives/2007/04/15/why-my-wordpress-site-is-so-much-faster-than-yours/
It goes into a bit more detail
July 18th, 2007 at 9:23 pm
[…] How to Handle Thousands of Users with Wordpress he suggests using Lighttpd vs. Apache, eAccelerator (for PHP) and WP_Cache for WP (tags: Wordpress Traffic) […]
July 18th, 2007 at 9:41 pm
I cant seem to find a place to confirm this. But when using WP-cache, arent those pageviews bypassed by Google Analytics?
ie if you use WP-cache, and you have x amount of visitors who view your site via cached info, none of those visits registers with your statistic setup….
Or am i wrong?
July 19th, 2007 at 5:52 am
Some of these advices are great, but one of my advice would be to perform stress testing on your server before making it “live”. This would help you tweak everything and maybe make your own optimisation in the code if you can (that’s one of the thing that cannot easily be done with 3rd part software, even if open source)
July 19th, 2007 at 6:57 am
eaccelerator does wonders in speeding php codes. tried it before with squirrelmail php based webmail and wow. what a difference does it makes!
July 19th, 2007 at 2:28 pm
[…] Este artigo foi lançado essa semana em: http://blog.thembid.com/index.php/2007/07/17/how-to-handle-thousands-of-users-with-your-blog/ […]
July 19th, 2007 at 2:32 pm
very good!! thankz!!
i make a portuguese translation here!
http://torradeira.net/como-suportar-milhares-no-de-usuarios-no-seu-blog/
July 19th, 2007 at 11:30 pm
[…] This article describes some of these techniques and I know for sure that a bunch of them work quite well (such as the PHP-accelerator which caches compiled php files). Give it a try! […]
July 21st, 2007 at 2:15 am
Stephen, WP-Cache does not bypass Google Analytics. The Google Analytics code will still be in the cached page, WP-Cache does not remove any code from the page. So all visits will be counted.
August 10th, 2007 at 10:25 am
[…] See How to handle thousands of users with your blog […]
December 9th, 2007 at 12:20 pm
[…] to popular belief, platforms such as WordPress can handle the traffic nicely. Here is a nice write up on how you can prepare your website for the […]
December 27th, 2007 at 6:03 am
Load balancing with multiple front-end servers also do help a lot.