Look for links to our Strategic Roadmap highlighting how this work falls into our priorities set by the Drupal Association Board and Drupal.org Working Groups.

July was an action packed month at the Drupal Association - we had our quarterly prioritization with the Working Groups, our annual all-hands summer staff meeting, and mentored two tremendously dedicated interns throughout.

Our primary engineering focus was on DrupalCI and Localize.drupal.org - though we also found time to make some iterative changes to Drupal.org in a few areas, namely: issue credit refinements, performance, groundwork for the new content model.

Strategic Planning

Prioritization for Q3 2015

The Drupal.org Working Groups help to provide governance for Drupal.org and to set priorities for the work of Association staff. Each quarter we evaluate our priorities with the Working Groups and update our Roadmap.

On July 15th we updated our roadmap based on the Working Groups input. Our main priorities for Q3 are Drupal.org services that are required to support the Drupal 8 release, and functional improvements to Drupal.org:

1. The port of localize.drupal.org to Drupal 7, as well as a few issues that support Drupal 8 localization.
2. Making sure DrupalCI meets the MVP spec set out by the core developers for providing test coverage for Drupal 8, and that it meets the functionality required to replace the old testbot system.
3. Improving Drupal.org search.
4. A new documentation section based on our content strategy work that will provide better organization and governance of documentation.

Additional priorities were identified for Association Staff to tackle as time permits.

Drupal Association Summer Staff Week

July was also the time for the annual all-hands staff meeting for Drupal.org. For one week, we gathered all our local and remote staff in our Portland office to discuss:

• The mission, vision, and values of the Association.
• Our ever-evolving relationship with the Drupal project itself.
• Setting engineering and design principles for the team.
• Finding sustainable revenue that will fund our work.

Internships at the Association

For 8 weeks beginning in mid-June the Association staff hosted and mentored two interns who had just completed Epicodus’ inaugural Drupal curriculum.

Bojana Skarich(BabaYaga64) and Daniel Toader(CountPacMan) worked with us on bug fixes, features, and theme work for the Conference Organizing Distribution and several related modules that allow the Association to run our DrupalCon websites.

We’d also like to thank our Supporting Partner ThinkShout for funding Daniel and Bojana’s work with us. This is just one small example of how the supporting partner program fosters our mission by promoting Drupal as part of a software development training curriculum and giving these new members of our community a great head start.

Drupal.org

Issue Credit Updates

We deployed two small updates to continue to refine and iterate on the Issue credit system that we implemented in the beginning of this year.

Firstly, to support an earlier change to allow explicit attribution as a volunteer, we’ve updated the attribution :hover state display. Previously unattributed comments and volunteer attributed comments would both simply display the username in attribution, though the distinction was being made in the comments themselves. Now that distinction exists not just in the data but in the display on comments.

Since releasing the issue credit system in March, there have been over 9,500 issue credits awarded on over 5,200 issues. Over 2,400 unique users and 250 unique organizations have been awarded issue credits. Over 1,000 projects (modules, themes, distributions) have credits that have been awarded. The last 90 days of issue credits can be viewed on each user and organization profile.

Secondly, we deployed a small change that will automatically generate the first comment on a newly created issue.

This automatically generated initial comment serves two purposes: It allows the original author of an issue to be credited when the issue is resolved, even if they did not leave any subsequent comments on an issue. It provides a link to the original issue summary providing a better at-a-glance view of what the original reporter wrote, even if the summary has since been edited a large number of times by other participants in the issue.

There are still additional refinements to be made as we find time - in particular providing a ui to edit the attribution that will be made for the automatically generated first comment.

Entityreference_prepopulate Module

The new content model for Drupal.org requires a number of new modules on Drupal.org. To ensure that the site remains performant we are serializing these changes as much as we can. The first new module to be deployed on Drupal.org was entityreference_prepopulate.

As we work to build out the Documentation section we’ll be installing additional modules, creating some new content types, and providing a number of new resources for maintaining documentation on the site.

Advanced Aggregator

Improving performance of Drupal.org is an ongoing concern, particularly as we look to adding new modules that while powerful may also be somewhat heavy on a site of our scale. Utilizing advanced css/js aggregation is something we began to gradually implement towards the end of June, and in July we completed the majority of the changes laid out in this issue.

With these changes we’ve largely completed the work that will be done here for the foreseeable future, though there may be a few more performance gains to be found here and there. Thanks again to mikeytown2 for his assistance.

Drupal 8 Blockers

DrupalCI

July was a huge month for DrupalCI. There are two major milestones for the Association’s work on DrupalCI.

1. DrupalCI must meet the testing requirements for Drupal 8 Core and Contrib specified by core developers.
2. DrupalCI must also meet or exceed the existing functionality of the PIFT/PIFR testbots for testing Drupal 7 and Drupal 6 so that the old testbot system can be retired.

The first milestone was our primary goal in July - while the second will be our hard focus in August.

We made tremendous strides towards the first goal, starting with a reformat of the test result output to better display in Jenkins. This new format more logically organizes the test output by:

Test Group -> Test Class -> Test Method -> Output/Result

This should make understanding the results of testing easier in the long run, and is also a precursor to displaying test result information directly on Drupal.org - which we hope to complete in August.

We also made improvements to the test history pages - so that project maintainers can make better comparisons of any given test result to the status of a branch when an issue was created, for example, or against the most recent branch test. These test history pages also allow maintainers to see which user triggered the test, and are the portal to the test results.

July also saw the deployment of patch level testing with DrupalCI - which can be enabled on a per environment basis for projects on Drupal.org.

Towards the end of July we also enabled testing for Contrib projects - allowing any project maintainer on Drupal.org to begin using DrupalCI. We are asking project maintainers to enable DrupalCI for their projects and provide us with their feedback in this issue. This will be critical for us to retire the old testing infrastructure.

We also focused on improving the performance and efficiency of the tests. Minimizing the time it takes to initiate a test and complete a full test run both improves efficiency for developers and maximizes the reach of the Association budget for automated testing.

The new DrupalCI architecture automatically scales up and down the number of bots dependent on need, which will hopefully present a cost savings once we disable the redundant old testing infrastructure.

In addition to the architectural work above - we also upgraded our base environments to php 5.5 to support the change in Drupal 8 minimum requirements.

Finally, we improved the documentation for project maintainers for enabling DrupalCI testing on their projects.

Endless thanks to jthorson for his help.

Localize.Drupal.org

After our community testing of the localize.Drupal.org Drupal 7 port in June we identified a critical path of remaining issues that needed to be resolved to allow us to complete the upgrade. Many of the issues were related to user roles and permissions do to the differences between Drupal 6 organic groups and the Drupal 7 version.

We put a hard focus on resolving as many of these issues in July as we could, so that we would be ready to perform the final upgrade in August (which was completed successfully on August 12th).

Much thanks to Gábor Hojtsy and Sébastien Corbin for helping us with this process.

Revenue-related projects (funding our work)

DrupalCons

Events.drupal.org saw a few small deployments in July - primarily to support the launch of the Session Schedule, BOF Schedule, and Social Event calendar.

We also added the ability for event attendees to purchase or renew their Drupal Association memberships while purchasing their tickets for DrupalCon.

Finally we are in the planning phase for some additional work to support our payment processing needs in India, and to support having the registration process live for multiple events simultaneously.

Sustaining Support and Maintenance

New Git Infrastructure Deployment

As mentioned in our June update - we put the bow on a long-standing project to migrate our git infrastructure to new servers in July. Much of the work to provision the new servers was completed in June - but the cutover itself was scheduled in the early weeks of July.

The new git infrastructure is now both redundant and highly available, greatly increasing the stability of a critical part of our infrastructure.

Serving Files from a Separate Domain

In July we also acquired a new domain and wildcard cert for *.drupalcontent.org. This new domain will be used to serve static files across the Drupal ecosystem of websites, providing benefits for security and reducing the size of http requests by serving these resources from a domain without cookies.

Work to serve files from the new domain name is ongoing, and many thanks to mlhess for helping us implement this change.

Load balancer stability

After continuing the debug the decreasing stability of our load balancers, we decided to swap hardware and rebuild the load balancers using different hardware. The new hardware is also using an updated configuration and operating system, which has proven to be more stable. The second load balancer is in the process of being built out using the new configuration and different hardware. The project should be completed by the end of August, bringing stability back to one of our key infrastructure components.

Many thanks to nnewton for helping us diagnose and make this change.

Updates to Updates

One of the core services that Drupal.org provides is updates.drupal.org. In essence this is a feature of Drupal itself. Because Drupal.org is the home of updates information for the entire project, we analyze our updates traffic as part of our project usage statistics.

Unfortunately the project usage stats have been somewhat unreliable - so in July (and continuing into August) we’ve given this system some attention.

Changes we’ve made or are in the process of making include:

• We moved the updates system to a CDN (and then migrating to a different CDN provider).
• We updated our processing to work with centralized logs on our loghost.
• We are improving the performance of the process from a 3-4 hour run per day’s worth of data to a 1 hour run per month of data.
• We are simplifying the process by removing an intermediate MongoDB deduplication/key-value store.

Work to improve the performance and stability of the updates stats system will be ongoing.

As always, we’d like to say thanks to all volunteers who are working with us and to the Drupal Association Supporters, who made it possible for us to work on these projects.

Follow us on Twitter for regular updates: @drupal_org, @drupal_infra.

Personal blog tags: whats new on Drupal.org