Drupal: Billions of Page Views per Month
Tuesday, September 16, 2008 at 09:20AM I am in fact still here. I have just been busy having started a new job recently. One of the more interesting projects I've been working on lately is actually a Drupal 6.2 project.
As a Scale Consultant on of my first new projects was working w/ a client to scale and test their Drupal installation to being capable of well over 2 BILLION page views per month! Well over... That's been quite fun.
In the spirit of sharing. I put the info up on the wiki at Joyent.
http://wiki.joyent.com/all-accelerators:kb:drupal
There's a nice diagram at the end that yours truly made. Here's a smaller version.
Drupal
I have some other really neat projects I've been working on in a new program I'm running called Joyent Labs. This is probably by far where I get to have the most fun during the week. In running the labs I'm essentially getting to work with some really cutting edge cloud computing companies, CMS developers, business partners, and much more. I've got some great things brewing in the labs and I can't wait to unleash the creations!
I have several articles on the back burner for here that I've been working on a while as well. Now that think I'm settling into my new groove a bit more I should be able to get back to blogging more consistently. I miss it!
So, that's where I've been. Sadly, not on vacation, but having fun none the less.
scale 
Email Article to Friend
Reader Comments (4)
Interesting article -
I'm curious as to what you would expect the peak load in pages per second you'd expect with the system as you've diagrammed it. In other words - are you figuring the 2b views as 1000/sec at constant load for the month, or does the system peak at significantly higher loads?
Well, it's always a bit subjective to say things like this but I'd say for a reasonably well configured drupal node (no crazy module non-sense) you should be able to be able to do 250 req/sec sustained as drawn here but with 3 1 GiB, 2 x 4 GiB, 2 x 8 GiB. That would be on 27 GiB worth of Accelerators at Joyent (~$3400 /month + setups) But, really, there is no way to truly answer that question without testing the actual page itself.
The system I ran my tests on had 8 application nodes and I did a 1000 req/sec on a well configured node over 20 minutes. But, in reality, it runs about 250 req/sec and peaks for about an hour at 4 to 5 times that.
We are working on the newest set of bottle necks to further optimize. As the bottle necks move, we work on them.
I think we can make this thing go a lot faster. I'm getting some specialists to help me with some other parts.
Interesting post. ration price/perf is pretty sexy. One question: with joyent can you put database and apache on the same accelerator? Are they pre-installed or do you have to install them?
Denis
@Denis
Yes, that is indeed how most of them start. Then, they split off the database. Then they split off the static content silo. Then they add more application servers. Then, the add read slaves. In summary, get what you need when you need it over time.
The accelerators come with everything pre-installed ready to run Drupal. They don't come with Drupal installed and configured at this time.