How to Handle Thousands of Users with Your Blog
July 17, 2007 – 10:36 amThere have been multiple occasions where I am trying to read an article and the website is down. I have encountered this multiple times where people say something like: “I got around 13,000 hits and server couldn’t handle it”. There are even websites that become popular by making the content of the popular article available. One of these cases is DuggMirror, a service that mirrors Digg articles and makes them available in case the source is unavailable. If you are running a blog and you have your own server, here are a few tips to easily handle at least half a million hits per day.
- Install Linux in your server. Linux runs pretty good and there are a lot of tools to support the server administration.
- You will need a web server. A lot of users install Apache but I strongly suggest to use Lighttpd. Your blog will usually have images, CSS files, and Javascript files. Apache usually in general doesn’t handle static files as well as Lighttpd. Lighttpd is an event-driven web server that minimizes the blocking waiting for IO.
- Your blog will need PHP and MySQL. Make sure you have those running.
- Install eAccelerator - By default, every time a PHP script is accessed, the web server must compile the script then execute the compiled code. This task is not needed if the PHP script doesn’t change. What a eAccelerator does is it optimizes the scripts, compiles them, then caches them in a compiled state.
- Install WordPress - This blogging software a lot or all of the features you need. It also scales well when having a lot of traffic.
- Install the WP-Cache plugin - Your requests will live in a database and every time you get a new visitor, WordPress will have to retrieve this content from the database. If you get 30,000 visitors, your blogging software will have to do the same query that many times. WP-Cache prevents by caching to disk whatever is in your database so that it can eliminate the need to query the database that many times.
- Create a memory file system - WP-Cache caches its files on disk. We know that memory access is much faster than disk access. Creating a memory file system (tmpfs) and putting WP-Cache files in it, will tremendously increase the speed of retrieving cache files.
- If you installed WordPress on something like /var/www/myblog and wanted to have a memory file system of 100MB you can execute the following command to mount a tmpfs file system where the cache directory resides:
mount -osize=100m -t tmpfs tmpfs /var/www/myblog/wp-content/cache
- You can also do this from /etc/fstab with the following line:
tmpfs /var/www/myblog/wp-content/cache tmpfs defaults,size=100m 0 0
- If you installed WordPress on something like /var/www/myblog and wanted to have a memory file system of 100MB you can execute the following command to mount a tmpfs file system where the cache directory resides:
We currently host this blog and ThemBid.com on the same server with this setup and sometimes gets around half a million hits per day without a problem.
32 Responses to “How to Handle Thousands of Users with Your Blog”
This is very useful information for us starting in the web servers world.
Thanks,
By Yasser on Jul 17, 2007
Good information, but it will still cost you higher than a Dedicated Server.
Plus you will have to keep all the packages up to date to worry less about script kidies.
By Cucirca on Jul 17, 2007
In fact you just need strong hosting provider for your blog. Any good provider have all this things
By hotdog on Jul 17, 2007
I am interested in what sort of server load you have when you get that many requests in a day, any idea?
By clubber lang on Jul 18, 2007
Mediatemple. Case solved.
By Troy on Jul 18, 2007
Cool. I want to know, how to do these (or similar) things for blog, based on python and django!
By Svetlyak on Jul 18, 2007
I used blogger.com..no problem with thousands visitor..hehe
By dxblogger on Jul 18, 2007
nice tips. although my blog just visited once or twice per day:D
By sumodirjo on Jul 18, 2007
Great guide to setting up a server for Wordpress use. Unfortunately, if you’re not in control of the hosting side you can’t do a lot of these things. Still, if I ever have to set up a server now I’ll be set.
By Chris Norton on Jul 18, 2007
Helps you understand what best practices are when you shop for a host, and some of this you can do yourself regardless of host. Note that Wikipedia and YouTube both use Lighttpd on some critical servers.
Still, MediaTemple’s clustered server grid and reported great support looks like a one-stop shop solution for uptime reliability (that I hope to be switching to soon!)
By Dave on Jul 18, 2007
# Cucirca Says:
July 17th, 2007 at 11:29 pm
Good information, but it will still cost you higher than a Dedicated Server.
Plus you will have to keep all the packages up to date to worry less about script kidies.
Actually, I run a similar setup on an old P3 box. Doesn’t cost me a thing.
By AbandonedHero on Jul 18, 2007
Actually, I run a similar setup on an old P3 Box. Doesn’t cost me a thing.
By AbandonedHero on Jul 18, 2007
WP-cache is the best way to handle hardcore traffic, thanks for the rest of the tips
By Deal$ on Jul 18, 2007
2. Lighttpd is great but it doesn’t support the use of .htaccess files so if you’re using them, you’ll need to rewrite them in lighty’s own format. Lighttpd also requires FastCGI PHP binaries which means that some users may need to recompile PHP.
4. eA is OK but I prefer XCache (http://xcache.lighttpd.net) which was created by the same person who made lighttpd.
7. tmpfs will spill over to disk if your server needs the RAM. In a case of high traffic levels that will almost certainly happen from all of the FastCGI PHP processes that are running if you’re using lighty or httpd processes if you run Apache’s httpd.
By Jason Litka on Jul 18, 2007
Or, you could use Horizon Blogging Software, which uses way less resources that WordPress and handles the digg effect a lot more nicely. Even though it’s in beta, it’s a nice package. I’m currently using it on my blog. I think the project page is http://www.edgedrive.com, check it out.
By jeffery on Jul 18, 2007
I was thinking this was going to be a completely lame article, the tips are pretty good.
By PENIX on Jul 18, 2007
What kind of server are you running? Specs would be nice.
By Mark on Jul 18, 2007
I’ve seen a few people respond saying just get a hosting company and they’ll do this for you, but I don’t think I know a single hosting company that has WP-Cache installed or is pulling that cache from active memory. Those two tips alone will deliver impressive gains.
By Justin Kistner on Jul 18, 2007
Good info alot of it that i’m currenty implementing myself.
cheers
Paul
By Paul knight on Jul 18, 2007
_How_ do you have half a million hits per day? I would learn and maybe pay something for this information..thanks in advance and best regards!..Petra
By Petra on Jul 18, 2007
You should also check out this:
http://elliottback.com/wp/archives/2007/04/15/why-my-wordpress-site-is-so-much-faster-than-yours/
It goes into a bit more detail
By Elliott Back on Jul 18, 2007
I cant seem to find a place to confirm this. But when using WP-cache, arent those pageviews bypassed by Google Analytics?
ie if you use WP-cache, and you have x amount of visitors who view your site via cached info, none of those visits registers with your statistic setup….
Or am i wrong?
By Stephen on Jul 18, 2007
Some of these advices are great, but one of my advice would be to perform stress testing on your server before making it “live”. This would help you tweak everything and maybe make your own optimisation in the code if you can (that’s one of the thing that cannot easily be done with 3rd part software, even if open source)
By Helixum on Jul 19, 2007
eaccelerator does wonders in speeding php codes. tried it before with squirrelmail php based webmail and wow. what a difference does it makes!
By syahid ali on Jul 19, 2007
very good!! thankz!!
i make a portuguese translation here!
http://torradeira.net/como-suportar-milhares-no-de-usuarios-no-seu-blog/
By Daniel Oliveira on Jul 19, 2007
Stephen, WP-Cache does not bypass Google Analytics. The Google Analytics code will still be in the cached page, WP-Cache does not remove any code from the page. So all visits will be counted.
By Sylvain on Jul 21, 2007
Load balancing with multiple front-end servers also do help a lot.
By Josh LA on Dec 27, 2007