Look for links to our Strategic Roadmap highlighting how this work falls into our priorities set by the Drupal Association Board and Drupal.org Working Groups.

Preparing for a Release Candidate

After a month of planning and organizational introspection in July, the Association spent August getting down to brass tacks. Since May, progress on Drupal 8 criticals has been rapid and we focused on doing our part to clear blockers for the upcoming Drupal 8 release candidate.

On August 13th, the Drupal Association engineering team met with core maintainers to discuss Drupal.org infrastructure blockers to a Drupal 8 release candidate. Since May, core developers have been rapidly clearing critical issues, and several Drupal.org services are blockers to the release. Fortunately, our meeting with core maintainers confirmed what we knew: DrupalCI and Localize.drupal.org are the two pieces of infrastructure that need work to support Drupal 8 release candidates. We also briefly discussed infrastructural issues to support a full 8.0.0 release as well as the path forward for supporting Composer.

We’re very excited for the upcoming release candidate and hope to celebrate with the community soon!

The Roadmap

Drupal 8 blocker: Localize.drupal.org

In August we completed the migration of Localize.drupal.org to Drupal 7. This was the culmination of a tremendous amount of work by the community over the last year, and quite a bit of work by the Association over the last several months. As always with a major site migration we had to work closely with the active users to ensure there were no functional regressions and do quite a bit of permissions and security review as well as some of the fundamental modules which power the site, such as organic groups, work quite differently in their Drupal 7 implementations.

Completing this upgrade also allowed us to focus on Drupal 8 release blockers in the localization system. First, we need a server side version fall back system for translations. Secondly, we need to support contrib by supporting configuration translatables with external dependencies. We’ve made solid progress on the first issue, but there is still work remaining.

Our initial work on these issues was reviewed towards the end of August, and we hope to have the remaining work completed in the first couple weeks of September.

Many thanks to Gábor Hojtsy for helping us review this final work.

Drupal 8 blocker: Drupal CI

In our last update we laid out two milestones that the Association was pushing hard to reach for DrupalCI: DrupalCI must meet the testing requirements for Drupal 8 Core and contrib specified by core developers. DrupalCI must also meet or exceed the existing functionality of the PIFT/PIFR testbots for testing Drupal 7 and Drupal 6 so that the old testbot system can be retired.

We’re very pleased to report that as of the end of August the DrupalCI meets the testing requirements set out by Drupal core developers. There are no blockers to a Drupal 8 release in the DrupalCI infrastructure!

We’ve also made major progress in several other areas:

• We made significant strides towards testing for Drupal 7 and Drupal 6. Simpletest jobs for these versions have a very different structure so this required some careful work.
• We now display test results directly on Drupal.org. This new display makes it much easier to see these results, and will also make it easier for us to provide email notifications for test failures.

However, there is still work to do. There is a flaw in test discovery for contrib modules that must be resolved for Drupal 8 contrib testing to be complete (in the first week of September a core patch for run_tests.sh was commited thanks to jthorson).

Once contrib testing is stable and functional, our next goal is to begin phasing out the old testing infrastructure. We’ve identified several key issues that will allow us to disable the old testbots in a phased way. Once Drupal 8 core and contrib testing are burned in and core and contrib developers have had time to affirm that the PIFT/PIFR bots are no longer needed for D8 testing, we will phase out PIFT/PIFR’s Drupal 8 testbots. Because Drupal 8 testing represents the majority of testing volume on our infrastructure at this time, being able to disable the redundant bots will provide a significant cost savings for the Association.

We’ll then focus on making sure that Drupal 7 and Drupal 6 testing are equally functional before phasing those bots out as well. Finally we’ll need to transition Qa.drupal.org to a static archive of the past test results.

In the meantime we are still asking project maintainers to enable DrupalCI for their projects and provide us with their feedback in this issue.

Search Improvements

The next item on our roadmap is improving the search experience. To make significant improvements in this area there was some pre-work we needed to do, both in planning and infrastructure. Earlier in the year the Association sat down with a consultant to provide recommendations on ways to improve our Solr configuration. At the same time, there were a number of new features of Solr itself in version 5 that we would not be able to take advantage of with our existing Solr 3 installation.

So we began our pre-work by creating a pre-production environment that would allow us to test our changes to search - evaluating what it would take to reindex Drupal.org with Solr 5 - and ultimately upgrading our production search servers to support Solr 5 and performing that index.

In parallel, we expanded the criteria that we would use to evaluate the success of our search improvements - drafting user stories that concretely define what a better search means for the variety of types of content that a user might be searching for.

Going into September we’ll then be implementing small iterative changes to our Solr configuration to tune our search results to meet these user stories.

Incremental Improvements to Drupal.org

Issue Queues

We’ve also made several incremental improvements to the issue queues during the month of August. We started by making an automatic first comment on issues when the reporter first creates an issue. This will allow the reporter of an issue to be credited by the maintainer when the issue is closed, even if that reporter does not make additional comments on the issue.

As we were making this change we took the opportunity to change a subtle detail about the issue summary. Previously the issue summary was attributed to the initial reporter. However, because Drupal.org issue summaries can be edited by anyone, this attribution was misleading. We have removed this attribution from the summary, and instead added the original reporter attribution to the issue meta data in the sidebar.

There is some follow up work to do to allow the initial reporter to adjust their organization/customer attribution in that automatically generated first comment. We will likely also look into allowing credit attributions for users who did not comment in the issue.

Many thanks to the community members who provided their feedback on these changes as we were making them. Making your voices heard allowed us to improve on these changes even as we were making them.

Performance Profiling for the new Content Model

Another key deployment that is very close for Drupal.org is the first iteration of the new content model. Content Strategy for Drupal.org has been a major initiative for most of the past year, and we have built out the first iteration of that featureset. It should enable a new content organization model on Drupal.org with sections that have individual governance and maintainership, and lay the groundwork for a new navigational paradigm for the site. These changes won’t be immediately apparent in terms of visual changes to Drupal.org, but instead provide structural tools to make it easier to govern and maintain content on the site. This work will be the basis for our improvements to Documentation on Drupal.org.

Before making this deployment we wanted to ensure that the new features would be performant. Drupal.org has a tremendous amount of content, is exceptionally highly trafficked, and provides services that are essential for the community to develop Drupal itself. We need to be sure that the new modules and features we deploy will be performant before rolling these new features out. We set up a new integration environment on which we could run some performance profiling tests, and in light of that testing we feel confident we’ll be able to deploy this first iteration very soon.

Revenue-related projects (funding our work)

DrupalCon Asia

August also saw the full site launch of DrupalCon Asia. It’s a beautiful site and we hope you’ll check it out and join us in highlighting the strength of the community in India. This is our third site launch on the unified Events.drupal.org subsite and the new multi-event system is paying dividends in allowing us to launch these Con sites more quickly, consistently, and on schedule than ever before.

The call for papers is open now - so if you’re going to join us for the first DrupalCon in Asia, please submit your session proposals now! We’re also accepting applications for grants and scholarships, and looking for volunteers to mentor the sprints.

Improvements to Jobs.drupal.org

We’d like to give a special thanks to community member , CTO of DoSomething.org, who volunteered his time in August to help us making improvements to the Drupal Jobs board. He helped to improve the way job postings are listed, helped us adjust how renewed postings would be sorted, and helped improve the data we gather so we can provide a better home for Drupal careers.

Thanks, Matt!

Sustaining Support and Maintenance

Every month there is infrastructure to be maintained and improved, and August was no exception. We performed a number of tasks including updating the Drupal.org SSL cert, rebulding one of our Solr servers to support the upgrade to Solr version 5, improving stability and redundancy of our load balancers.

We also made adjustments to how we maintain our dynamically scaling infrastructure for DrupalCI test bots on Amazon. In August we upsized and rebuilt the dispatcher instance to take advantage of on the fly compression and provide us more capacity for the high volume of testing we’ve already been doing on the new system. Providing testing to the community represents a significant infrastructural cost for the Drupal Association, so we have also been focusing on ways to improve the efficiency of testing and reduce our expenses.

---

As always, we’d like to say thanks to all volunteers who are working with us and to the Drupal Association Supporters, who made it possible for us to work on these projects.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra