Upgrading Drupal 6 and i18n Content Translation to Drupal 8

Our first Drupal 6 to Drupal 8 Upgrade

We recently had to perform our first Drupal 6 to Drupal 8 upgrade. The Drupal 6 site was a multilingual site with three content types, a couple of CCK fields and a straightforward number of common modules. Nothing special. In this blog post I'll describe how we upgraded this site. 

The Migrate module has replaced update.php as the default upgrade path to Drupal 8. There are two modules in Drupal Core (migrate and migrate_drupal) and one module in Contrib (migrate_upgrade) that allow for upgrading Drupal 6 or 7 to Drupal 8. Here is a nice overview of the different Migrate modules.

Important:

  • Not all modules already provide an upgrade path.
  • All modules that should be upgraded have to be enabled in Drupal 8.
  • Migrate automatically creates content types, fields, text formats, etc.

The upgrade can either be executed using the Drupal Web interface or Drush (preferred). In both cases, one needs to have at hand a path to the old Drupal installation (with private files a local file system path, otherwise the URL can be provided) and credentials to access the old Drupal database. The following Drush command automatically performs the upgrade:

drush migrate-upgrade
  --legacy-root='/var/www/hostname/web'
  --legacy-db-url='mysql://user@hostname/database'

First attempt

We used the above command to perform our first upgrade attempt on a clean Drupal 8 install, that is, without preparing content types, fields, text formats, etc. The upgrade ran smoothly, but there were still a couple of things that did not work as expected. Most noticeably:

  • Even though the Core Date module was enabled in Drupal 8, CCK Date fields were not migrated.
  • The node body was migrated but not displayed (apparently some problem with the text format).
  • Migration from Contrib i18n Content Translation to Core Content Translation in Drupal 8 was not successful, that is, there were separate nodes and not a single node for a translation set. There is an issue on drupal.org

Then we realized that there were other things that we wanted to improve during the upgrade:

  • Some CCK Textfields, configured as select lists with a PHP snippet providing the possible values (ouch!), should now be taxonomy terms.
  • We switched from a multisite installation to a single site installation, so the path to the public files directory changed from sites/multisite/files to sites/default/files. Inline images were not displayed and inline links did not work anymore.
  • One content type came with a dedicated teaser field and we now wanted the content of this field to be available in the summary of the content type's body field.

The client not only hired us to perform the upgrade to Drupal 8, but also wanted us to implement a redesign. We realized that many of the migrations prepared by the Migrate module were not necessary. For instance, the imagecache presets from Drupal 6 did not have to be migrated to Drupal 8 image styles because the image dimensions in the new design were very different. This insight, in conjunction with the aforementioned migration problems, led us to the conclusion to only run the migrations that were really necessary and to adapt them to our needs.

Second attempt

So, instead of running the automatic upgrade on a clean Drupal 8 install, we created content types, text formats, image styles etc. as we usually do when building a Drupal site. We also installed two more Contrib Migrate modules that provide additional features and Drush commands:

  • Migrate Tools
  • Migrate Plus

Technically, migrations in Drupal 8 are config entities. By adding the --configure-only option to the drush migrate-upgrade command, we created the config entities without executing the migration:

drush migrate-upgrade
  --legacy-root='/var/www/hostname/web'
  --legacy-db-url='mysql://user@hostname/database'
  --configure-only

Then, we exported the active configuration into YAML files in the configuration directory specified in the $config_directories array in the settings.php file:

drush config-export

We added a config directory named migrate to the $config_directories array and copied, renamed and edited the config files – respectively migrations – that we really needed, namely users, files and nodes. These are the files we ended up with in our migrate config directory:

  • migrate.migration.d6_user_custom.yml
  • migrate.migration.d6_file_custom.yml
  • migrate.migration.d6_node__page_custom.yml
  • migrate.migration.d6_node__news_custom.yml
  • migrate.migration.d6_node__project_custom.yml

In order to avoid collisions with the existing migrations, _custom was added to the file names and to the config entity id in the yaml data. The uuid values as well as most dependencies were removed from the files, so that unwanted migrations were not pulled in. Another configuration was added in order to group our migrations (grouping is a feature of the Migrate Plus module):

  • migrate_plus.migration_group.custom.yaml

We also created a couple of custom migration plugins to solve the problems we encountered, included them in our migration configuration, and finally imported our configuration into the active configuration store:

drush config-import --partial migrate

The migrate argument specifies the name of our config directory. The --partial option ensures that missing configuration is not deleted (since there is only migration configuration in the migrate config directory).

Finally, we executed the migration:

drush migrate-import --group=custom

Code and comments

As a result, we only migrated the content that we needed. Compared to the automatic upgrade, we also solved various problems and added own improvements (especially the content translation migration). Below is a listing of the config files (migrations) and the (migration) plugins we created for the upgrade. In general, the custom migration group was added, config uuids as well as the dependencies on other default migrations were removed from the config files and the id was changed. The node migrations received dependencies on the custom user and file migrations. In a custom module, custom migration process plugins were created to fix or prepare input data and a custom destination plugin was added to handle the migration from i18n content translation to Drupal 8 content translation.

Migration configuration

migrate_plus.migration_group.custom.yml

id: custom
label: Custom
description: 'Custom Migrations.'

migrate.migration.d6_user_custom.yml

id: d6_user_custom
migration_tags:
  - 'Drupal 6'
label: 'User accounts'
source:
  plugin: d6_user
  database_state_key: migrate_upgrade_6
process:
  uid: uid
  name: name
  pass: pass
  mail: mail
  created: created
  access: access
  login: login
  status: status
  timezone:
    plugin: user_update_7002
    source: timezone
  preferred_langcode: language
  init: init
destination:
  plugin: 'entity:user'
  md5_passwords: true
template: d6_user
migration_group: custom

migrate.migration.d6_file_custom.yml

dependencies:
  config:
    - migrate.migration.d6_user_custom
id: d6_file_custom
migration_tags:
  - 'Drupal 6'
label: Files
source:
  plugin: d6_file
  database_state_key: migrate_upgrade_6
process:
  fid: fid
  filename: filename
  uri:
    plugin: file_uri
    source:
      - filepath
      - file_directory_path
      - temp_directory_path
      - is_public
  filemime: filemime
  filesize: filesize
  status: status
  changed: timestamp
  uid: uid
destination:
  plugin: 'entity:file'
  urlencode: true
  source_base_path: /var/www/hostname/web/
template: d6_file
migration_dependencies:
  required:
    - d6_user_custom
migration_group: custom

migrate.migration.d6_node__page_custom.yml

id: d6_node__page_custom
migration_tags:
  - 'Drupal 6'
label: 'Nodes (page)'
source:
  plugin: d6_node
  database_state_key: migrate_upgrade_6
  node_type: page
process:
  nid:
    # Custom process plugin to determine the nid that should
    # be used by the custom destination plugin (see below),
    # which solves the problem with the upgrade from i18n
    # content translation to Drupal 8 content translation.
    plugin: tnid
    source: tnid
  type: type
  langcode:
    plugin: default_value
    source: language
    default_value: de
  title: title
  uid: node_uid
  status: status
  created: created
  changed: changed
  promote: promote
  sticky: sticky
  body/format:
    # Set default text format as source text formats
    # are not migrated.
    plugin: default_value
    default_value: basic_html
  body/value:
    # Use custom process plugin that fixes urls that 
    # changed due to new public files path.
    plugin: body
    source: body
  revision_uid: revision_uid
  revision_log: log
  revision_timestamp: timestamp
destination:
  # Use custom destination plugin that handles
  # migration from i18n content translation to 
  # d8 content translation.
  plugin: 'custom_node'
template: d6_node
migration_dependencies:
  required:
    - d6_user_custom
    - d6_file_custom
migration_group: custom

migrate.migration.d6_node__news_custom.yml

id: d6_node__news_custom
migration_tags:
  - 'Drupal 6'
label: 'Nodes (news)'
source:
  plugin: d6_node
  database_state_key: migrate_upgrade_6
  node_type: news
process:
  nid:
    plugin: tnid
    source: tnid
  type: type
  langcode:
    plugin: default_value
    source: language
    default_value: de
  title: title
  uid: node_uid
  status: status
  created: created
  changed: changed
  promote: promote
  sticky: sticky
  body/format:
    plugin: default_value
    default_value: basic_html
  body/value:
    plugin: body
    source: body
  body/summary:
    # Use value from source field field_news_teaser 
    # as summary value and do not migrate the field itself.
    # Additionally, the field_news_teaser plugin removes
    # all markup.
    plugin: field_news_teaser
    source: field_news_teaser/0/value
  revision_uid: revision_uid
  revision_log: log
  revision_timestamp: timestamp
field_news_date:
    # Process the date value in a custom
    # process plugin so that it works in d8.
    plugin: field_news_date
    source: field_news_date
destination:
  plugin: 'custom_node'
template: d6_node
migration_dependencies:
  required:
    - d6_user_custom
    - d6_file_custom
migration_group: custom

migrate.migration.d6_node__project_custom.yml

id: d6_node__project_custom
migration_tags:
  - 'Drupal 6'
label: 'Nodes (project)'
source:
  plugin: d6_node
  database_state_key: migrate_upgrade_6
  node_type: project
process:
  nid:
    plugin: tnid
    source: tnid
  type: type
  langcode:
    plugin: default_value
    source: language
    default_value: de
  title: title
  uid: node_uid
  status: status
  created: created
  changed: changed
  promote: promote
  sticky: sticky
  body/format:
    plugin: default_value
    default_value: basic_html
  body/value:
    plugin: body
    source: body
  revision_uid: revision_uid
  revision_log: log
  revision_timestamp: timestamp
  field_project_type:
    # Use a static map to map source text field values 
    # to destination term ids.
    plugin: static_map
    source: field_project_type/0/value
    map:
      arena: 3
      stadium: 4
  field_project_status:
    plugin: static_map
    source: field_project_status/0/value
    map:
      0: 1
      1: 2
  field_project_images:
    plugin: d6_cck_file
    source: field_project_images
  field_project_boxes: field_project_boxes
  field_project_standing_places: field_project_standing_places
  field_project_seats: field_project_seats
  field_project_business_seats: field_project_business_seats
  field_project_all_seater: field_project_all_seater
  field_project_total_seats: field_project_total_seats
  field_project_construction_time: field_project_construction_time
  field_project_completion:
    # Process the date value so that it works in d8.
    plugin: field_project_completion
    source: field_project_completion
destination:
  plugin: 'custom_node'
template: d6_node
migration_dependencies:
  required:
    - d6_user_custom
    - d6_file_custom
migration_group: custom

Migration plugins

The tnid process plugin replaces the source nid by the tnid so that nodes belonging to a translation set have the same nid and not different ones.

/**
 * @file
 * Contains \Drupal\migrate_custom\Plugin\migrate\process\Tnid.
 */

namespace Drupal\migrate_custom\Plugin\migrate\process;

use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\Row;

/**
 * @MigrateProcessPlugin(
 *   id = "tnid"
 * )
 */
class Tnid extends ProcessPluginBase {

  /**
   * {@inheritdoc}
   */
  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
    return empty($value) ? $row->getSourceProperty('nid') : $value;
  }
}

The custom_node destination plugin creates a new node if the nid does not exist yet or adds a translation if it does exist.

/**
 * @file
 * Contains \Drupal\migrate_custom\Plugin\migrate\destination\CustomNode.
 */

namespace Drupal\migrate_custom\Plugin\migrate\destination;

use Drupal\Core\Entity\EntityStorageInterface;
use Drupal\Core\Plugin\ContainerFactoryPluginInterface;
use Drupal\Core\TypedData\TypedDataInterface;
use Drupal\migrate\Plugin\migrate\destination\DestinationBase;
use Drupal\migrate\Entity\MigrationInterface;
use Drupal\migrate\Row;
use Drupal\node\NodeInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;

/**
 * @MigrateDestination(
 *   id = "custom_node"
 * )
 */
class CustomNode extends DestinationBase implements ContainerFactoryPluginInterface {

  /**
   * The entity storage.
   *
   * @var \Drupal\Core\Entity\EntityStorageInterface
   */
  protected $storage;

  /**
   * {@inheritdoc}
   */
  public function __construct(array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration, EntityStorageInterface $storage) {
    parent::__construct($configuration, $plugin_id, $plugin_definition, $migration);
    $this->storage = $storage;
    $this->supportsRollback = TRUE;
  }

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition, MigrationInterface $migration = NULL) {
    return new static(
      $configuration,
      $plugin_id,
      $plugin_definition,
      $migration,
      $container->get('entity.manager')->getStorage('node')
    );
  }

  /**
   * {@inheritdoc}
   */
  public function getIds() {
    $ids['nid']['type'] = 'integer';
    $ids['langcode']['type'] = 'string';
    return $ids;
  }

  /**
   * {@inheritdoc}
   */
  public function fields(MigrationInterface $migration = NULL) {
    return [
      'nid' => 'The node id.',
      'langcode' => 'The node language code.',
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function import(Row $row, array $old_destination_id_values = array()) {
    $nid = $row->getDestinationProperty('nid');
    $type = $row->getDestinationProperty('type');
    $langcode = $row->getDestinationProperty('langcode');
    $existing_node = $this->storage->load($nid);

    if ($existing_node instanceof NodeInterface) {
      $node = $existing_node->addTranslation($langcode);
    }
    else {
      $node = $this->storage->create([
        'nid' => $nid,
        'type' => $type,
        'langcode' => $langcode,
      ]);
    }

    foreach ($row->getDestination() as $field_name => $values) {
      $field = $node->$field_name;

      if ($field instanceof TypedDataInterface) {
        $field->setValue($values);
      }
    }

    $node->save();
    return array($nid, $langcode);
  }

  /**
   * {@inheritdoc}
   */
  public function rollback(array $destination_identifier) {
    $entity = $this->storage->load(reset($destination_identifier));
    if ($entity) {
      $entity->delete();
    }
  }
}

The body process plugin fixes the change in the public files path.

/**
 * @file
 * Contains \Drupal\migrate_custom\Plugin\migrate\process\Body.
 */

namespace Drupal\migrate_custom\Plugin\migrate\process;

use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\Row;

/**
 * @MigrateProcessPlugin(
 *   id = "body"
 * )
 */
class Body extends ProcessPluginBase {

  /**
   * {@inheritdoc}
   */
  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
    return str_replace('sites/multisite/files/', 'sites/default/files/', $value);
  }
}

The field_news_date process plugin fixes the date format so that it works with the Drupal 8 date field format.

/**
 * @file
 * Contains \Drupal\migrate_custom\Plugin\migrate\process\FieldNewsDate.
 */

namespace Drupal\migrate_custom\Plugin\migrate\process;

use Drupal\Component\Utility\Unicode;
use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\Row;

/**
 * @MigrateProcessPlugin(
 *   id = "field_news_date"
 * )
 */
class FieldNewsDate extends ProcessPluginBase {

  /**
   * {@inheritdoc}
   */
  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
    if (!empty($value['value'])) {
      $value['value'] = Unicode::substr($value['value'], 0, 10);
    }

    return $value;
  }
}