A composer based workflow for Drupal 8

Single image:

20.08.2014
Tobias Stöckler

Body:

Every developer had a fight once with dependency management of big Drupal projects, which use a couple of modules and maybe one theme with base themes etc.
At the same time there are maybe patches to Drupal core itself, modules are developed internally on some GIT server etc.. It's a pain in the ass to get the code from various places and keep it up to date.

In the node.js world, for example, people use a tool called npm
which fetches all the versions from various sources in a simple and reproducable way.

Naive approach

The first approach, you could also call it poormans dependency managment, is to put everything into one gigantic git repository: All of core, the various contrib and custom modules as well as themes and install profiles. On the one hand this makes it trivial to get the "current" version of the project but on the other hand there is no workflow if you have to update some of the dependencies.

Let's have a look at an example: Drupal releases a security release. Sadly you have some critical performance patch applied as well. Now you have to download the release, apply your patch upon it etc. without loosing time.

Another problem is that you cannot really develop on the dependencies itself. If you have a shared custom module between projects you have to change the code of the custom module in another place. Copy over the new files etc. Yet another pain in the ass.

drush makefile

One way of solving it is to provide a makefile. There you specify both the version of core, contrib modules, themes, some git repositories and applied patches. Your main repository will then just contain one drush.make file. Calling drush makethen fetches the code.
While this looks really nice to quickly get some code, you will find problems on the daily work.

makefiles are not meant to update existing dependencies. It just works if you want to start from scratc
It is a drupal only solution, so you cannot use it for other external libraries etc.

Composer

For sure the Drupal community is not the only PHP community with the dependency managment problem. Therefore a group of people worked on composer, a tool similar to npm.

When you start a new project you create a composer.json file and put some information
in there:

{
"name": "My great drupal project",
"description": "One composer file to rule them all, one autoloader to bind them",
"license": "GPL-v2+"
"require": {
"php": ">=5.4.2",
"symfony/http-foundation": "2.5.*"
}
}

Once you have greated such a file all you need is to run composer install and it will fetch you the dependencies of symfony/http-foundation.

Sadly Drupal core as well as modules aren't typical PHP libraries, so you need some additional work to put them into the proper directory layout.

Main Project

In order to move modules inside the proper folder we use some tricks in our project composer.json file, you will see in the following example

{
name: "Example project",
"minimum-stability": "dev",
"require-dev":{
"composer/installers": "~1.0",
# Specify the used branch of the drupal module.
"drupal/menu_link_config": "dev-8.x-1.x"
},
"repositories": [
{
# Tell composer about the drupal module.
"type": "vcs",
"url": "<a href="<a href="http://git.drupal.org/project/menu_link_config.git">http://git.drupal.org/project/menu_link_config.git</a>"
">http://git.drupal.org/project/menu_link_config.git">http://git.drupal.or...</a> }
],
"extra": {
"installer-paths": {
# Force composer to install the contrib module in a specific location.
"web/modules/contrib/{$name}": ["drupal/menu_link_config"]
}
}
}

Drupal 8 core
The Problem

Drupal 8's new directory layout is built in such a way, that every file that is changed through a Drupal core update lies inside the top-level core directory. For people manually managing their files (via FTP, for example) this is great news. Delete the old core directory, put the new one in place, and voilá: Drupal core is updated. If you pull in Drupal core via Composer, however, the situation plays out a bit differently. The Drupal core repository is not the core directory itself, but instead contains a bunch of top-level files (such as index.php, .htaccess, etc.) as well as the core directory. And contributed or custom modules and themes should be placed into the top-level modules and themes directories which are also part of the Drupal repository. Not only does this mean that updating Drupal core via Composer might overwrite local changes to .htaccess. Even worse: you cannot also pull in contributed (or custom) modules via Composer as those would need to be placed inside the code of the Drupal core package.

The solution

What one really wants is a Git repository that is exactly the core directory of the Drupal repository, but keeps up with the main repository over time. Fortunately, Git provides a nifty tool to make this happen: git subtree. This command - from within a given repository - can extract the version history of a specified directory. You can see the result of such a subtree split at tstoeckler's Drupal core repository. The version history of the 8.0.x branch of that repository is identical to that of Drupal core's 8.0.x branch except that commits that only touched files outside of the core directory are ignored. There is also one additional commit that brings in the Composer files from the Drupal repository, as those are located in the Drupal root. This allows to pull in Drupal via Composer.

How to use

{
name: "Example project",
"minimum-stability": "dev",
"require":{
# Use a custom version of the Composer installers that support Drupal core.
"composer/installers": "dev-drupal-core",
# Specify the used branch of Drupal core.
"drupal/drupal-core": "8.0.*"
},
"repositories": [
{
"type": "vcs",
"url": "<a href="<a href="https://github.com/tstoeckler/installers">https://github.com/tstoeckler/installers</a>"
">https://github.com/tstoeckler/installers">https://github.com/tstoeckler/...</a> },
{
"type": "vcs",
"url": "<a href="<a href="https://github.com/tstoeckler/drupal-core">https://github.com/tstoeckler/drupal-core</a>"
">https://github.com/tstoeckler/drupal-core">https://github.com/tstoeckler...</a> }
],
"extra": {
"installer-paths": {
# Force composer to install Drupal in a specific location.
"web/core": ["tstoeckler/drupal-core"]
}
}
}

With this composer.json file in the project root, running composer install will fetch Drupal's core directory exactly in the way we wanted. When initializing the repository the files that are in the root of Drupal repository, such as index.php and .htaccess need to be fetched manually. Although this is a little bit more work initially it also means that local changes to .htaccess will no longer be overwritten (although if Drupal's default .htaccess changes you should verify whether you want to include those changes in your repository as well). Speaking of Drupal core updates: composer update is all you need with this method.

Drupal 8 modules

In order to use composer properly, each "entity" of the environment has to specify its information, so each
module you use, both contrib and custom, should have such a small composer.json file.

{
"name": "drupal/menu_link_config",
"description": "muh",
"type": "drupal-module",
"license": "GPL-2.0+",
"minimum-stability": "dev",
"require": { }
}

The important bit here is "type": "drupal-module"

Summary

Once you have a proper composer based workflow, it feels a bit like the holy grail. You can work on contrib modules inside your normal project workflow.
Installing the dependencies is also blazing fast etc.. We hope that more and more people will use. Feedback and other approaches are welcomed.

Example

https://gist.github.com/dawehner/dd6bf5b07cdd28d95f6c provides a working example for a composer file

Schlagworte/Tags:
planet
php
drupal-planet
Drupal 8
dc-planet
composer

#03

09.09.2014 Andrew (anonym)

Does not work for Drupal 7? …

Frustrated with drush make, I came up with a similar approach but for Drupal 7.
The only problem I encountered was that I am only able to fetch the modules via their files (http://ftp.drupal.org/files/projects/strongarm-7.x-2.0.tar.gz) rather than the git repository (http://git.drupal.org/project/strongarm.git).
So this method works up until the point you want to receive a specific commit.
Do you know if it's possible to fetch them via their repo?

Antworten

#04

04.10.2014 donquixote (anonym)

download the package, enable the module …

So far in Drupal we have a distinction of packages vs modules. Packages can be downloaded, can contain more than one module, and modules can be individually installed or uninstalled.
There is also a distinction between module dependencies and package dependencies. Module dependencies tell you to enable other modules. Package dependencies (in Composer) tell you to download other packages.
So far I always had the idea that 3rd party libraries should be downloaded depending on enabled modules. This is also how Composer manager works.
I think it becomes easier if we change the thinking. This is already close to what you are proposing here, but maybe not exactly.
1. Instead of drush dl, you add the package to the composer.json and run composer update.
2. Packages can be 3rd party libraries or Drupal packages.
3. Packages can contain any number of Drupal modules, even if they come from other sources. The composer.json might have an extra key where to look for those.
4. You can only enable those modules where the module itself and all dependencies are in the downloaded subfolders.
5. Modules can depend on other modules even if the containing package composer.json does not force these to be downloaded. This can be used for optional integration modules.
6. Some magic could be implemented for module packages that don't contain a composer.json.
7. module namespaces are registered in the autoloader depending on the package status via composer.json, not the module installation status.
8. Plugin discovery only runs in enabled modules.

Antworten

Ihr Name

Kommentar/Comment

Save

Leave this field blank

Original Article:

A composer based workflow for Drupal 8