Community driven development

29 Jul

Stuart Clark

Realityloop has a long history with the Melbourne Drupal community; We’ve
been heavily involved in the monthly Drupal meetups and began the monthly
mentoring meetups. However, the monthly mentoring only came about after a
failed experiment in community based web development.

While that experiment may have failed, the idea of community driven Drupal
development has been of great interest to me as it truly embraces the spirit
of open source.

In the last two weeks I have release two websites for the Drupal Melbourne
community, a DrupalMelbourne
community portal and a landing page for the upcoming DrupalCampMelbourne2015. In this tutorial I will be
demonstrating how anyone can get involved with the development of these sites,
or how the process can work for the benefit of any other community based
website.

 

The workflow:

  1. Build the codebase
  2. Setup and install the site
  3. Make your changes
  4. Update features and makefiles
  5. Test the changes
  6. Fork repository / push changes / pull request

 

Build the codebase

As per usual for myself and Realityloop, these sites are built using a slim
line profile / makefile approach with the GIT repository tracking custom code
only, which means you will require Drush to build the site codebase.

If you are not familiar with Drush (DRUpal
SHell), I highly recommend familiarising yourself as it’s not
only incredibly useful in everyday Drupal development, it is also a
requirement for this tutorial. Installation instructions can be found at http://www.drush.org/en/master/install/

Assuming you have Drush ready to go, building the codebase is as simple as
running the following, relevant command:

  • DrupalMelbourne​
    drush make --working-copy=1 https://raw.githubusercontent.com/drupalmel/drupalmel/master/stub.make drupalmel-7.x
  • DrupalCampMelbourne2015
    drush make --working-copy=1 https://raw.githubusercontent.com/drupalmel/drupalcampmel/2015.x/stub.make dcm-2015.x

The resulting codebase contains the specified Drupal core (currently 7.38)
along with the relevant install profile containing all custom and contrib code
(modules, themes and libraries).

Note: --working-copy=1 is used to retain the link
to the GIT repository.

 

Setup and install the site

Once your codebase is built you simply need to install a site as you would
normally do so, ensuring that you use the relevant installation profile
(DrupalMelbourne or DrupalCampMelbourne).

The sites are constructed in such a way that there is absolutely no need to
copy down the production database, any content is either aggregated from
external sources, or dummy content is created via the Devel generate module
for the purposes of development. This means that there is no laborious data
sanitization processes required, allowing contributors to get up and running
in as short of time as possible.

For more details on how to setup a Drupal site, refer to the Installation
Guide or your *AMP stack.

 

Make your changes

No change is insignificant, and a Community driven site thrives on changes;
if you think you can make the site look better, found a bug that you can fix,
or want new functionality on the site, the only thing stopping you is you!

 

Update features and makefiles

Once you’re happy with the changes you’ve made, be it content, theme or
configuration, you need to ensure that it’s deployable, and as we’re not
dealing with databases at all, this means that you need to update the
codebase; features and makefiles.

If you’re not familiar with Features or Makefiles, much like Drush, I highly
recommend them, as again, they are required for this particular approach of
Community driven development.

You can find more details on Features, Makefiles and Drush at my DrupalSouth
2012 talk "Ezy-Bake Drupal:Cooking sites with Distributions".

 

Features

Features allows you to capture most of your configuration in the filesystem,
allowing to be deployed via GIT.

In the case of these sites, there is only one feature which encapsulates all
configuration, as these sites have a relatively straight forward purpose. Some
sites may warrant more, that is a discussion for another day.

To update the feature, it’s an extremely simple process:

  1. Navigate to the relevant path within your site:
    • DrupalMelbourne:
      admin/structure/features/drupalmel_core/recreate
    • DrupalCampMelbourne2015:
      admin/structure/features/drupalcampmel_core/recreate
       
  2. Add or remove any required components (Page manager, Strongarm, Views,
    etc).
     
  3. Expand the Advanced options fieldset and click the Generate feature
    button.

More information can be found at the Features
project page.

 

Makefiles

Makefiles are recipes of modules, themes and libraries that get downloaded by
Drush make, including their versions and patches.

Updating a makefile is relatively straightforward, it’s just a matter of
opening the file in your IDE / text editor of choice and updating the
entries.

There are two makefile in these sites, stub.make and drupal-org.make; the
stub.make contains Drupal core and the install profile (and any relevant
patches) and the drupal-org.make contains all third-party (contrib) code.

Any new or updated modules, themes or libraries (and any relevant patches)
need to be added to this file, as no third-party code is tracked in the GIT
repo.

The makefiles are organized into three primary sections; Modules, Themes and
Libraries. Below are some examples of how an entry should be defined:

 

Bean module, version 1.9:

projects[bean][version] = 1.9

 

Reroute Email module, specific GIT revision with patch
applied:

  1. projects[reroute_email][download][revision] = f2e3878
  2. ; Variable integration - http://drupal.org/node/1964070#comment-7294928
  3. projects[reroute_email][patch][] = http://drupal.org/files/reroute_email-add-variable-module-integration-1964070-2.patch

Note: It is always important to include version, if you need
a development release then use a GIT revision as otherwise what you build
today may be drastically different from what you build tomorrow.

 

Bootstrap theme, version 3.1-beta2:

  1. projects[bootstrap][type] = theme
  2. projects[bootstrap][version] = 3.1-beta2

Note: As the default projects 'type' is set to module,
themes need to specify their type. This is a personal choice in the Drush make
file configuration, as it is highly likely you will always have more modules
than themes.

 

Backbone library, version 1.1.2:

  1. libraries[backbone][download][type] = get
  2. libraries[backbone][download][url] = https://github.com/jashkenas/backbone/archive/1.1.2.zip

Note: As libraries are not projects hosted on Drupal.org (in
general), you need to specify the URL of which the files is downloadable, or
cloneable, from.

More information can be found on the Drush make
manual page.

 

Other changes / hook_update_N()

Sometimes changes don’t fall under the realm of features or makefiles, either
due to a modules lack of integration with features, or when dealing with
content rather than configuration. This still needs to be deployable via the
codebase, and can be done with the use of a hook_update_N() function.

A hook_update_N() is a magic function that lives in a
modules .install file, where hook is the machine name of the
module and N is a numeric value, formatted as a 4 digit
number in the form of XY##, where X is the
major version of Drupal (7), Y is the major version of the
module (1) and ## is a sequential value, from 00 to 99.

Example: drupalmel_core_7100() / drupalcampmel_core_7100()

The contents of a hook_update_N() is whatever you wish it to be, and Drupal
API function or PHP.

An example of one such function is:

  1. /**
  2. * Assign 'ticket holder' role to ticket holders.
  3. */
  4. function drupalcampmel_core_update_7105() {
  5. $query = new EntityFieldQuery();
  6. $results = $query->entityCondition('entity_type', 'entityform')
  7. ->entityCondition('bundle', 'confirm_order')
  8. ->execute();
  9.  
  10. if (!empty($results['entityform'])) {
  11. $entityforms = entityform_load_multiple(array_keys($results['entityform']));
  12. foreach ($entityforms as $entityform) {
  13. $user = user_load($entityform->uid);
  14. $user->roles[3] = 'ticket holder';
  15. user_save($user);
  16. }
  17. }
  18. }

For more details, refer to the hook_update_N()
API documentation.

 

Test the changes

Once you’ve made your changes and prepared your features and makefiles, it’s
ideal to ensure that everything is working as expected before you push it up
to the GIT repo.

This is a multi-step process, but it’s easy enough, especially given that we
don’t have a database that we have to worry about.

 

Features:

  1. Take a database dump of your local (development) site; Safety first.
     
  2. Re-install the site with the the relevant install profile:
    • DrupalMelbourne:

      drush si drupalmel -y

    • DrupalCampMelbourne2015:

      drush si drupalcampmel -y

  3. Test to ensure your changes are present and working as expected.

 

Makefiles:

Testing your makefile can be a little bit trickier than testing your
features, as when you download a module, theme or library there are various
places they can be stored, and it’s easy to get a false positive.

  1. Build a --no-core version of the drupal-org.make file
    into a temporary directory. A --no-core build is exactly
    what it sounds like, build the makefile excluding Drupal core.Example:
    1. cd ~/temp
    2. drush make --no-core --no-gitinfofile ~/Sites/drupalmel-7.x/profiles/drupalmel/drupal-org.make dm-temp
  2. Run a diff/merge tool over the --no-core build’s sites/all directory and
    your local (development) site’s relevant profile directory (e.g.,
    profiles/drupalmel).
    I personally use Changes on OS X, but there are
    different free and paid diff/merge tools for different operating
    systems.
     
  3. Ensure that all third-party (contrib) code is identical on both sides of
    the diff/merge, any discrepancies imply that you may be missing an entry in
    your makefile, or that your local version of the code is located in an
    incorrect location.

If your changes aren’t working as expected, or something is missing, simply
restore your database dump and go back to the Update features and makefiles
step.

 

Fork repository / push changes / pull request

Now that you have made your changes and everything is good to go, it’s time
to push those changes back to the repository.

For the sake of a manageable review process, it’s preferable that all changes
should be made in a fork with a pull request back to the master
repository.

If you’ve only ever lived in the realm of Drupal.org, then this may be an
entirely alien process, but it is again a relatively straight forward
process.

Note: If you don’t have a Github account, you will need one.
Signup for free at https://github.com/join

  1. Go to the relevant Github repository:
  2. Click the Fork button (top right of the page) and follow
    the onscreen instructions.
     
  3. Click the Copy to clipboard button on the clone
    URL     field (in the right sidebar).
     
  4. Add a new GIT remote to your local (development) site with the copied URL.Example:
    1. cd ~/Sites/drupalmel-7.x/profiles/drupalmel
    2. git remote add fork git@github.com:Decipher/drupalmel.git
  5. Commit and push the changes to your fork.
     
  6. Create a pull request via your Github fork by clicking the Pull
    request     button, providing as much detail as possible of what your
    changes are.

If all goes well, someone will review your pull request and merge the changes
into the relevant website.

 

The review process

So this is the not so community friendly part of the process; In a perfect
world the community should be able to run itself, but Github isn’t necessarily
setup this way, nor is Drupal.org. Someone has to specifically approve a Pull
request. Currently this is only myself and Peter Lieverdink (cafuego).

I’m absolutely open to suggestion on how to improve this, comment below if
you have any thoughts on how this could be improved.

 

The uncommitables

Not everything should be committed, especially in a public repository. A
perfect example of something that shouldn’t be committed is an API key.

The DrupalMelbourne website integrates with the Meetup.com API to pull in all
DrupalMelbourne Meetups, but exposing the API key to the codebase would open
the DrupalMelbourne meetup group to abuse and spam.

To deal with this, API keys and other sensitive items can be dealt included
directly on the server or in the database, and placeholders can be user for
local development.

 

Open source your site?

Exposing your website codebase is definitely not the normal practice, and
it's absolutely not for everyone. I couldn't imagine trying to convince a
client to go down this road. But for a community site, especially a Drupal
based community site, it just makes sense. While I wouldn't expect every
visitor with Drupal knowledge to volunteer their time to help with the sites
development, any who do is 100% more than you'd get otherwise.

drupaldrupal planet

Original Article: