Drupal Helpers: Tools for devops and deployment

Topics: 

As I've been architecting Drupal solutions for almost ten years now, I've accumulated quite a bit of knowledge on devops best practices, which constitutes a sizeable amount of the consulting that I do. This includes documentation, configuration management, development processes and deployment processes. In this article, I'll be introducing Drupal Helpers, a collection of standard scripts and configurations that I use on all of my client projects (where applicable).

At the time of the writing, the repository provides support for the following operations and Drush set-up. Additional tools are always welcome.

Refreshing a development site's database and files

The resync-drupal-db-for-dev script essentially deploys the database and files from a staging site (Staging) or production site (Prod) onto a development site, but it also does many other things that should be done as part of that process, devifying it.

It takes source and destination Drush aliases as arguments and the third one, a list of modules to disable, is optional. Here is the usage example:

  • resync-drupal-db-for-dev <SOURCE_DRUSH_ALIAS> <DESTINATION_DRUSH_ALIAS> [<MODULES_DISABLE>]

It performs the following tasks:

  1. Saves a cache-cleared dump of the destination database (DB) as a backup.
  2. Overwrites the destination DB with the source's.
  3. Rebuilds the registry in case PHP file locations have changed.
  4. Updates the DB schema.
  5. Reverts all features to the code in Features modules.
  6. Reverts all views to those defined in code.
  7. Disables modules that shouldn't be enabled during development.
  8. Enables modules that are helpful for development.
  9. Disables CSS and JavaScript caching.
  10. Sets the files and temporary directories to standard locations.
  11. Enables on-screen error reporting.
  12. Disables user-initiated cron runs. This should really be disabled everywhere for performance reasons.
  13. Clears all caches.
  14. Overwrites the destination's files directory with the source's.
  15. Runs cron.

Developers should be doing all of this every time they refresh their DBs. Because it's tricky and time-consuming to do all of these manually, some of the steps are often missed. This leads to configuration mismanagement issues between development sandboxes and other environments. I'd recommend that this script, or another one like it, be run frequently on the authoritative development/integration site (Dev) and local development sites to prevent such mishaps.

The script tries to be as versatile as possible, working in a variety of GNU/Linux environments. If it doesn't work for yours, please submit a merge request so that we can get support added.

Deploying code to a development/integration site

The deploy-drupal-code-dev script is useful for deploying the latest development code that's been merged to the development branch to Dev. As part of that process, it does everything else that's necessary after a code deployment, including clearing external Varnish caches. It doesn't produce output to the screen when running it directly, as it's assumed to be run as a cron job. All output gets redirected to a log file. A best practice is to have it run nightly so that Dev is kept up-to-date.

Deploying code to staging or production sites

The two (2) scripts deploy-drupal-code-qa and deploy-drupal-code-prod basically follow the same idea as the Dev script above, except that:

  • They require a Git release tag as an argument, as only tagged releases should be deployed to Staging and Prod.
  • They produce output directly to the screen, as they are intended to be run manually.

PHP configuration for Drush

The drush.ini configuration file (see Configuring php.ini for details) adds Drush-specific configuration to PHP, differentiated by its being run from the command line; Drush's PHP doesn't go through a Web server.

As Drush is often called upon for batch processing, it requires more resources than Web-server PHP. Also, there aren't usually multiple instances of it running so we don't need be as concerned about overflowing resource limits. With Web servers, there could be a huge number of PHP processes running on public-facing sites.

The configuration file does the following:

  • Increases the memory limit.
  • Ensures all errors are shown.
  • Sets the time zone.
  • Increases the maximum execution time. You may need to do this for MySQL/variants as well.

Make sure this is set up by placing the file (or a symlink to it) in your /etc/php5/cli/conf.d/ directory (or the equivalent for different systems).

Default Drush alias configuration

The default.alias.drushrc.php Drush alias file is a standard location for storing all common Drush configuration. It needs to be set in each of your site alias' configuration stanzas to use it.

The line is:

  • <span style="color: #000000"><span style="color: #0000BB">&lt;?php<br>$aliases</span><span style="color: #007700">[</span><span style="color: #DD0000">'dev'</span><span style="color: #007700">] = array(<br>&nbsp; </span><span style="color: #DD0000">'parent' </span><span style="color: #007700">=&gt; </span><span style="color: #DD0000">'@default'</span><span style="color: #007700">,<br>&nbsp; ...<br></span><span style="color: #0000BB">?&gt;</span></span>

It could actually use an update now that Better handling of structure-tables and skip-tables options (including cache_* support!) is done. We no longer need to explicitly state which cache tables we need to skip during certain operations.

Backing up databases

The backup-drupal-db script will back up a Drupal DB whenever it's run. It can be run stand-alone or as a Cron job. To save storage space, cache tables will not be included in the compressed dump files. Each is timestamped, and the "LATEST" file will always be a symbolic link to the most recent backup. Old backups are automatically deleted after a set number of days; the default is 60 (~2 months).

Rebuilding a site with its latest Drush makefile

The drupal-remake script will rebuild a Drupal site's document root to reflect recent changes in its Drush makefile. It performs the following tasks:

  1. Takes the site off-line.
  2. Backs up the existing database.
  3. Updates the Git repository with the latest code.
  4. Purges the old code base.
  5. Rebuilds it.
  6. Re-installs the previous sites directory (with site-specific configuration).
  7. Properly sets all file permissions.
  8. Updates the DB schema.
  9. Turns the site back on-line.
  10. Clears all caches.

Deploying Solr onto the GlassFish application server

The deploy-solr-on-glassfish script sets up the Solr search engine to run on the GlassFish application server.

With Solr 5, it's no longer necessary to run the search engine in an application server. It can run as a stand-alone application. See Install Apache Solr 5 and Drupal Search API on your laptop in minutes for an example of how to do this. For earlier versions though, this is helpful in getting the Java stack set up to run alongside the PHP one.

A special thanks goes out to Jamon Camisso for posting the original version of this on GitHub earlier.

This article, Drupal Helpers: Tools for devops and deployment, appeared first on the Colan Schwartz Consulting Services blog.

Original Article: