Note: This blog post is based on Drupal 8.1.x. It is an updated version of a previous tutorial based on Drupal 8.0.x. While the concepts are largely the same as 8.0.x, a refactoring of the core migrate modules took place in Drupal 8.1.x (migrations will become plugins in 8.1.x). This updated tutorial updates the previous example to work with Drupal 8.1.x, as well as demonstrates how to specify a migration group and run the migration with Drush. If you're familiar with the previous tutorial, you may want to skip to the "Rolling up our sleeves" section below.

Even if you're only casually acquainted with Drupal 8, you probably know that the core upgrade path to Drupal 8 has been completely rewritten from the ground-up, using many of the concepts of the Migrate and Drupal-to-Drupal migration modules. Using the Migrate upgrade module, it is possible to migrate much of a Drupal 6 (or Drupal 7) site to Drupal 8 with a minimum of fuss (DrupalEasy.com is a prime example of this). "Migrate upgrade" is similar to previous Drupal core upgrade paths - there are no options to pick-and-choose what is to be migrated - it's all-or-nothing. This blog post provides an example of how to migrate content from only a single, simple content type in a Drupal 6 site to a Drupal 8.1.x site, without writing any PHP code at all.

Setting the table

First, some background information on how the Drupal 8 Migrate module is architected. The Migrate module revolves around three main concepts:

• Source plugins - these are plugins that know how to get the particular data to be migrated. Drupal's core "Migrate" module only contains base-level source plugins, often extended by other modules. Most Drupal core modules provide their own source plugins that know how to query Drupal 6 and Drupal 7 databases for data they're responsible for. For example, the Drupal 8 core "Node" module contains source plugins for Drupal 6 and Drupal 7 nodes, node revisions, node types, etc… Additionally, contributed and custom modules can provide additional source plugins for other CMSes (WordPress, Joomla, etc…), database types (Oracle, MSSQL, etc…), and data formats (CSV, XML, JSON, etc.)
• Process plugins - these are plugins designed to receive data from source plugins, then massage it into the proper form for the destination on a per-field basis. Multiple process plugins can be applied to a single piece of data. Drupal core provides various useful process plugins, but custom and contributed modules can easily implement their own.
• Destination plugins - these are plugins that know how to receive data from the process plugins and create the appropriate Drupal 8 "thing". The Drupal 8 core "Migrate" module contains general-purpose destination plugins for configuration and content entities, while individual modules can extend that support where their data requires specialized processing.

Together, the Source -> Process -> Destination structure is often called the "pipeline".

It is important to understand that for basic Drupal 6 to Drupal 8 migrations (like this example), all of the code is already present - all the developer needs to do it to configure the migration. It is much like preparing a meal where you already have a kitchen full of tools and food - the chef only needs to assemble what is already there.

The configuration of the migration for this example will take place completely in two custom .yml files that will live inside of a custom module. In the end, the custom module will be quite simple - just a .info.yml file for the module itself, and two .yml files for configuring the migration.

Reviewing the recipe

For this example, the source Drupal 6 site is a large site, with more than 10 different content types, thousands of nodes, and many associated vocabularies, users, profiles, views, and everything else that goes along with an average Drupal site that has been around for 5+ years. The client has decided to rewrite the entire site in Drupal 8, rebuilding virtually the entire site from the ground-up - but they wanted to migrate a few thousand nodes from two particular content types. This example will demonstrate how to write a custom migration for the simpler of the two content types.

The "external article" content type to be migrated contains several fields, but only a few of consequence:

• Title - the node's title
• Publication source - a single line, unformatted text field
• Location - a single line, unformatted text field
• External link - a field of type "link"

Some additional notes:

• The "Body" field is unused, and does not need to be migrated.
• The existing data in the "Author" field is unimportant, and can be populated with UID=1 on the Drupal 8 site.
• The node will be migrated from type "ext_article" to "article".

Several factors make this a particularly straight-forward migration:

• There are no reference fields at all (not even the author!)
• All of the field types to be migrated are included with Drupal 8 core.
• The Drupal 6 source plugin for nodes allows a "type" parameter, which is super-handy for only migrated nodes of a certain type from the source site.

Rolling up our sleeves

With all of this knowledge, it's time to write our custom migration. First, create a custom module with only an .info.yml file (Drupal Console's generate:module command can do this in a flash.) List the Migrate Drupal (migrate_drupal) and Migrate Plus modules as dependencies. The Migrate Drupal module dependency is necessary for some of its classes that contain functionality to query Drupal 6 databases, while the Migrate Plus module dependency is required because custom migrations are now plugins that utilize the MigrationConfigEntityPluginManager provided by Migrate Plus (full details in a blog post by Mike Ryan).

Next, create a "migration group" by creating a migrate_plus.migration_group.mygroup.yml file. The purpose of a migration group is to be able to group related migrations together, for the benefit of running them all at once as well as providing information common to all the group migrations (like the source database credentials) in one place.

The "shared_configuration -> source -> key" value of "legacy" corresponds to a database specified in the Drupal 8 site's settings.php file. For example:

Next, create a new "migrate_plus.migration.external_articles.yml" file in /config/install/. Copy/paste the contents of Drupal core's /core/modules/node/migration_templates/d6_node.yml file into it. This "migration template" is what all node migrations are based on when running the Drupal core upgrade path. So, it's a great place to start for our custom migration. Note that the file name begins with "migrate_plus.migration" - this is what allows our custom migration to utilize the Migrate Plus module's MigrationConfigEntityPluginManager.

There's a few customizations that need to be made in order to meet our requirements:

• Change the "id" and "label" of the migration to something unique for the project.
• Add the "migration_group: mygroup" to add this migration to the group we created above. This allows this migration access to the Drupal 6 source database credentials.
• For the "source" plugin, the "d6_node" migration is fine - this source knows how to query a Drupal 6 database for nodes. But, by itself, it will query the database for nodes, regardless of their type. Luckily, the "d6_node" plugin takes an (optional) "node_type" parameter. So, we add "ext_article" as the "node_type".
• We can remove the "nid" and "vid" field mappings in the "process" section. The Drupal core upgrade path preserves source entity ids, but as long as we're careful with reference fields (in our case, we have none), we can remove the field mappings and let Drupal 8 assign new node and version ids for incoming nodes. Note that we're not migrating previous node revisions, only the current revision.
• Change the "type" field mapping from a straight mapping to a static value using the "default_value" process plugin. This is what allows us to change the type of the incoming nodes from "ext_article" to just "article".
• Similarly, change the "uid" field mapping from a straight mapping to a static_value of "1", which assigns the author of all incoming nodes to the UID=1 user on the Drupal 8 site.
• Since we don't have any "body" field data to migrate, we can remove all the "body" field mappings.
• Add a mapping for the "Publication source". On the Drupal 6 site, this field's machine name is "field_source", on the Drupal 8 site, the field's machine name is field_publication_source. Since it is a simple text field, we can use a direct mapping.
• Add a direct mapping for "field_location". This one is even easier than the previous because the field name is the same on both the source and destination site.
• Add a mapping for the source "External link" field. On the Drupal 6 site, the machine name is "field_externallinktarget", while on the Drupal 8 site, it has been changed to "field_external_link". Because this is a field of type "link", we must use the "d6_cck_link" process plugin (provided by the Drupal 8 core "Link" module). This process plugin knows how to take Drupal 6 link field data and massage it into the proper form for Drupal 8 link field data.
• Finally, we can remove all the migration dependencies, as none of them are necessary for this simple migration.

The resulting file is:

Note that .yml files are super-sensitive to indentation. Each indentation must be two spaces (no tab characters).

Serving the meal

To run the migration, first enable the custom module. The act of enabling the module and Drupal core's reading in of the migration configuration could trigger an error if the configuration isn't formatted properly. For example, if you misspelled the "d6_node" source plugin as "db_node", you'll see the following error:

[Error] The "db_node" plugin does not exist.

If the module installs properly, the Drush commands provided by the Migrate Tools (8.x-2.x-dev - 2016-Apr-12 or later) module can be used to manage the migration. First, use the Drush "migrate-status" command (alias: ms) can be run to confirm that the migration configuration exists. This is similar to functionality in Drupal 7's Migrate module.

~/Sites/drupal8migrate $ drush ms Group: mygroup          Status  Total  Imported  Unprocessed  Last imported       enternal_articles       Idle    602    602       0            2016-04-29 16:35:53

Finally, using Drush, the migration can be run using the "migrate-import" (alias: mi) command:

~/Sites/drupal8migrate $ drush mi external_articlesProcessed 602 items (602 created, 0 updated, 0 failed, 0 ignored) - done with 'external_articles'

Similarly, the migration can be rolled back using the drush "migrate-rollback" (alias: rm) command:

~/Sites/drupal8migrate $ drush migrate-rollback external_articlesRolled back 602 items - done with 'external_articles'

Once the migration is complete, navigate over to your Drupal 8 site, confirm that all the content has been migrated properly, then uninstall the custom module as well as the other migrate-related modules.

Note that the Migrate module doesn't properly dispose of its tables (yet) when it is uninstalled, so you may have to manually remove the "migrate_map" and "migrate_message" tables from your destination database.

Odds and ends

• One of the trickier aspects about writing custom migrations is updating the migration configuration on an installed module. There are several options:
  • The Configuration development module provides a config-devel-import-one (cdi1) drush command that will read a configuration file directly into the active store. For example: drush cdi1 modules/custom/mymodule/config/install/migrate.migration.external_articles.yml
  • Drush core provides a config-edit command that allows a developer to directly edit an active configuration.
  • Finally, if you're a bit old-school, you can uninstall the module, then use the "drush php" command to run Drupal::configFactory()->getEditable('migrate.migration.external_articles)->delete();, then reinstall the module.
• Sometimes, while developing a custom migration, if things on the destination get really "dirty", I've found that starting with a fresh DB helps immensely (be sure to remove those "migrate_" tables as well!)

Additional resources

• Migrations are now plugins change record
• Migration Update for Drupal 8.1 - blog post by Mike Ryan.
• Migrate a custom JSON feed in Drupal 8 with Migrate Source JSON blog post by Jeff Geerling.
• Custom Drupal-to-Drupal Migrations with Migrate Tools - blog post by William Hetherington.

Thanks to Mike Ryan and Jeffrey Phillips for reviewing this post prior to publication.