The Drupal community has been growing steadily over time, and recently started to outgrow our shared, single server system (Pentium Xeon 3Ghz with 1 GB of RAM). The server hosts a variety of Drupal websites, including http://www.drupal.org/, http://cvs.drupal.org/, http://www.drupaldocs.org/ and a number of other sites.

As mentioned earlier, we have been talking to the Open Source Lab (OSL) at Oregon State University and they generously offered to provide free rack space, bandwidth, power, backup facilities and on-site support. OSL is responsible for taking care of some of the largest projects in the Open Source community, including the Apache Foundation and the Mozilla Foundation. In order to take advantage of this generous offer, we need to supply our own equipment. Our initial plan was to purchase a modest dedicated server to replace our current shared system. However, thanks to the success of last week's donations, we will be able to roll out a more sophisticated server architecture to support the growth of our community. Read on for the details, and an extra "sun-prise" ...

A surprise from Sun Microsystems

Tim Bray (Sun Microsystems employee, W3C member, co-inventor of XML, and Drupal fan) noticed we were in trouble, and asked around if Sun Microsystems could donate a server. On Sunday Sun Microsystems' Software CTO, Hal Stern mailed me saying they wanted to ship us a free Sun Fire V20z server. Monday morning Scott Kveton, Associate Director of the Open Source Lab, got in touch with Sun and completed some paperwork on our behalf. Much to our surprise, the Sun server arrived at OSL the morning after! Don't believe me? Check the shipping details (see below, picture on the left) or have a look at the pictures snapped by OSL personnel (or see below, picture on the right). Sun managed to stun all of us, that's for sure!

The Sun Fire V20z server has the following specifications:

• 2 x AMD Opteron
• 4 x 1 GB DDR-2 400 RAM
• 2 x 73 GB U230 SCSI drives (RAID1)

Props to Tim Bray (Software CTO office), John Fowler (Executive Vice President of the Network Systems Group) and Hal Stern (Software CTO) for making this possible. Thanks!

Proposed server infrastructure

On Monday, we had a meeting with OSL. We decided all the donated money (just over $10,000 USD) would be transferred to OSL, and OSL and Firebright would work together to spec out a good infrastructure for our money. Later this week, they followed up with a proposal and quotes for the infrastructure. In short, the OSL will order us 3 Dell 1850s. Each Dell 1850 has the following specifications:

• 2 x Intel Xeon
• 2 x 1 GB DDR-2 400 RAM
• 2 x 73 GB U230 SCSI drives (RAID1)
• Dual power supplies
• Remote access card

Scott Kveton of OSL explains:

With 4 total servers, we will not only be more scalable but also much more reliable. The Sun Fire V20z will act as a database server, and we will have 2 Dell 1850 application servers, each handling half of the load. We'll start with a DNS round-robin and then move to using Linux Virtual Server (LVS) for full redundancy of the drupal.org websites. The great part about using LVS is it will enable us to scale even further should the need arise without a massive rearchitecture of the existing gear.

We will be using the third Dell 1850 to handle the CVS/SVN and mail services for Drupal. This is technically a single-point of failure but we did make sure to order redundant power supplies and will be doing RAID1 with the 2 - 73G disks.

In the short term, we'll backup the database every day and off-load to the OSL backup server. In the long term, we can look at doing replication with another box if the need arises and as we grow.

The Dell 1850s will be ordered as soon the donated money is transferred to OSL. The transfer is in process but it takes Paypal and the local banks a number of business days to get the money from A to B.

Once the hardware arrived, we'll probably be installing Gentoo on them as it gets us the most bang for our buck. Also, OSL has a lot of Gentoo experience: two people of Gentoo's board of trustees work for the OSL, and the OSL provides hosting for Gentoo.

Community services

In the next few weeks we'll try to setup a system administrators team. We'll grow our team of system administrators (currently Kjartan, Steven and myself), we'll create a chart of who administers what, and who to contact when something is out of whack. In other words, we'll get ourselves organized. Tune in on the Drupal infrastructure mailing list if you want to get involved.

In addition, we have offered to sponsor a student intern at the OSL for 8 weeks. We all have day jobs and in order to get this up and running in a timely matter, the intern will help with the intensive work on the servers, help with the transition from the existing site to the new one, as well as focus on some new functionality the Drupal community has been clamoring for. OSL interns work directly for OSL's system engineers from whom they will get guidance. Furthermore, several of OSL's students are doing - or have done - internships at NASA, Intel, Gentoo and Mozilla.

Note also that quite a few of us will be attending O'Reilly's Open Source Convention and the Drupal conference in Portland (1-7 august). We'll visit the OSL and meet with the OSL folks face-to-face. Feel free to join us!

Once the existing services have been migrated to the new infrastructure, we'll look into extending the services we offer the Drupal community. With the help from our intern and OSL personnel, we'll setup the infrastructure so we can give people levelled access. For example, we'll make it so Ber Kessels can move the Drupal theme garden from his personal server to the new Drupal server infrastructure, and he has the permissions it takes to maintain the Drupal theme garden. Similarly, we intend to provide a Subversion mirror of our CVS repositories. (Possibly, a first step in moving from CVS to Subversion?) We might also be able to provide hosting for other Drupal-related communities, like language-specific communities.

That is it for now. We'll follow up with more information as soon one or more servers are in place. Thanks go out to Sun, OSL and the Drupal community for their/your fantastic support!

Updates

July 19, 2005 (Dries): I had to wait for some donations to be cleared so I couldn't transfer the money until today. As of two hours ago, the donated money is on its way to OSL.