Why we moved Drupal.org to a CDN

As of a little after 19:00 UTC on 2 July 2014, Drupal.org is now delivering as many sites as possible via our EdgeCast CDN.

Why a CDN?

We are primarily concerned with the network level security that a CDN will provide Drupal.org.

The CDN enables us to restrict access to our origin servers and disallow directly connecting to origin web nodes (which is currently possible). The two big advantages are:

  1. Accelerate cacheable content (static assets, static pages, etc).
  2. Allow us to easily manage network access and have a very large network in front of ours to absorb some levels of attacks.

Here are some examples of how the CDN helps Drupal.org:

  • We were having issues with a .js file on Drupal.org. The network was having routing issues to Europe and people were complaining about Drupal.org stalling on page loads. There was basically nothing we could do but wait for the route to get better. This should never be a problem again with EdgeCast's global network.
  • We constantly have reports of updates.drupal.org because blacklisted because it serves a ton of traffic coming in and out of a small number of IP addresses. This should also not happen again because the traffic is distributed through EdgeCast's network.
  • A few months ago we were under consistent attack from a group of IPs that was sub-HTTP and was saturating the origin network's bandwidth. We now have EdgeCast's large network in front of us that can 'take the beating'.

updates.drupal.org

By enabling EdgeCast's raw logs, rsync, and caching features, we were able to offload roughly 25 Mbps of traffic from our origin servers to EdgeCast. This change resulted in a drastic drop in origin network traffic, which freed up resources for Drupal.org. The use of rsync and the raw log features of EdgeCast enabled us to continue using our current project usage statistics tools. We do this by syncing the access logs from EdgeCast to Drupal.org’s utility server that processes project usage statistics.

Drupal.org

Minutes after switching www.drupal.org to use the CDN, there were multiple reports of faster page load times from Europe and North America.

A quick check from France / webpagetest.org:
Pre-CDN results: first page load=4.387s. repeat view=2.155s
Post-CDN results: first page load=3.779s, repeat view=1.285s

Why was the www.drupal.org rename required?

Our CDN uses a combination of Anycast IP addresses and DNS trickery. Each region (Asia, North America, Europe, etc.) has an Anycast IP address associated with it. For example cs73.wac.edgecastcdn.net might resolve to 72.21.91.99 in North America, and 117.18.237.99 in Japan.

Since 72.21.91.99, 117.18.237.99, etc. are Anycast IPs, generally their routes are as short as possible, and the IP will route to whatever POP is closest. This improves network performance globally.

Why can't drupal.org be a CNAME?

The DNS trickery above works by using a CNAME DNS record. Drupal.org must be an A record because the root domain cannot be a CNAME. MX records and any other records are not allowed by the RFC on CNAME records. To work around this DNS limitation, Drupal.org URLs are now redirected to www.drupal.org.

 

 

Related issueshttps://www.drupal.org/node/2087411https://www.drupal.org/node/2238131

Original Article: