In early February a gathering of developers came together in Acquia’s offices to decide the fate of the Drupal 8 initiative known as WSCCI (Web Services and Core Context Initiative). This event has been blogged about a handful of times already by Dries Buytaert, Larry “Crell" Garfield and Daniel “sun" Kudwien so I won’t rehash what has already had a lot of robust and detailed coverage.

One of the conclusions from the sprint was that an entire class of problems that WSCCI was trying to solve—mainly around improving Drupal core’s layout and block system—was a large enough problem that it really deserved attention as its own proper, separate initiative.

Why a new initiative?

While rescoping WSCCI, it became obvious that there were three separate scopes into which WSCCI was creeping.

• Web Services
• “Frameworkification" of Drupal core
• Page Layout

Page layout, and the various foundational work required to get there, are all well within the scope of what Drupal 8 would like to deliver. But this can’t all fall under WSCCI, and as such, a new initiative was decided upon, and I was asked to lead it.

Ok, so... what is this new initiative then?

This initiative aims to bring unity to a system of disjointed output components (blocks, page callbacks, menus, theme settings, and more) and provide a standardized mechanism of output, new tools for placing content on a page, and a potential for performance gains amongst other benefits.

Blocks as they currently exist have out-lived their usefulness as a basic Drupal component. Placement of blocks is a common shortcoming of the core system, as is the use of multiple instances of a single block, which is impossible without contributed modules. A number of other very popular solutions have been around for a while (e.g. Context, Panels, Display Suite) that change the notion of how blocks work, and what can be done with them. The goal of this initiative then is to build a best of breed solution leveraging the tools and experience developed in Drupal Contrib.

Currently, Drupal doesn’t treat all content equally, and there are so many different types of things on screen at the same time (page callback output, blocks, theme settings, etc.) that it can be daunting to keep up. Instead of building pages via a page callback as “primary" content, and then blocks and other bits of “secondary" decorating the page, the goal is to make any output on the page anywhere a block. Not a “block" as traditionally thought of in the limited Block module in Drupal 7 and below, but a “smart block" that is context-aware and utilizes per-instance block configuration.

Additionally, instead of regions being theme-specific, the capability to add regions would move to the hands of the site builder. A palette of layout options will be offered to select from (3-column, grid, etc.), including the ability to nest layouts within layouts, and layouts could factor in contextual data (e.g. a different layout for articles vs. page nodes).

How do we get there?

First and foremost, I need your help! :)

A lot of thought has gone into the steps involved in doing this long before the WSCCI sprint in Boston; being offered this position has encouraged me to really focus and prioritize that list of things into what I hope are deliverables that are practical for Drupal as a whole, and can stand alone should the next phase of development not land in time for our Dec 1st feature freeze. What follows is an outline of sorts, first detailing the systems we need in core in order to even consider this initiative achievable, and second the work involved in delivering on this initiative.

Step 0: Dependencies outside of this initiative:

1. Contribution to WSCCI’s foundational work with Request/Routing/Response.
   • We’re going to leverage Symfony’s Request, Routing and Response systems from the WSCCI initiative. Since HTML is just one type of REST response, once this is enabled, we will take over the HTML response system and build a new set of tools for delivering Drupal’s content. We are essentially the text/html web service response of web services. Step one is getting Symfony’s Request object in core.
2. Configuration Management Initiative.
   • The new block system we are architecting depends heavily on custom configuration management objects per block instances, and without CMI, we will be completely crippled. We need to get this framework reviewed and committed to core as soon as possible. UPDATE: Committed 3/2/12
3. Finalization of the Plugin subsystem.
   • The Plugin system is a unified way of dealing with various “pluggable" elements within Drupal: for example, blocks, swappable subsystems (e.g. cache layer), and text formats, etc.
   • The plugin system has been modeled after CTools but with an OO approach. CTools’ solution for this was always very OO inspired (even supporting OO plugins). The system is very robust, providing generic factory solutions for multiple use cases with the ability to extend as necessary, and a wrapper system for calling plugins of various type easily.
   • As soon as the initiative’s repo is setup and ready to go plugin development will continue in a branch specifically for plugins. Until that time you can get a sneak peak in my repo that I’ve been using as a git submodule.

Blocks Everywhere Phases:

1. Create a standard interface and class specification (plugin) that works to satisfy existing needs of blocks.
   • Includes meta data about the block, configuration forms, validation, submission, rendering and a few other things like contextual awareness.
2. Migrate existing blocks to the new structure.
   • Includes a lot of work in the blocks administration system.
   • First real check point for the whole system. Until this much is done the system is not a complete enough product to do anything.
3. Move page elements (title, logo, primary links, etc) to the block system.
   • There’s already issues in the core queue for some of this, such as Convert primary and secondary menus to blocks. We need to decide which of these could most benefit from being worked on now, with core’s current block system, and then refactor to the new block system later, and which ones to wait on until the new block system is in place.
4. Integrate forms with the new block system.
   • Many current “page callbacks" in Drupal are drupal_get_form() with a particular $form_id passed as a “page argument". How do we best integrate all these forms into the new block system? Should we convert each form into a block plugin class, or create a single “Form" block plugin class that can take a form id as a configuration or context argument, or something else? We’ll need to do some experimentation, and this will be subjected to lots of code review and iterating based on feedback.
   • Similarly, the Symfony approach relies on reflection of variable names to do a lot of its magic, and drupal_get_form() totally breaks this with its typical use case.
   • We’ll need to consider how best to make form redirects and dynamic content configurable. We’ll also need to integrate form POST handling with the WSCCI initiative (which plans to implement separate routing for POST vs. GET requests). While implementation details are still unclear at this time, the end goal is to enable our administrative tools to become plastic and re-arrangeable in such a way as to allow install profiles and distributions to provide a more custom tailored administrative experience.
5. Integrate non-form “page callbacks" with the new block system.
   • Some page callbacks, like node_page_default() (the list of node teasers shown on the front page in a default Drupal installation), will need to be made into first class block types, complete with their own unique configuration options (e.g., how many teasers to display).
   • For the rest, there’s an open question on whether to make all of them first class block types, or leave them similar to the functions they are now, but passed the contextual information they need to ensure their dynamic content isn’t tightly coupled to their default URL, so that custom sites and distributions can rearrange their URLs and page placement just like with any other block. I am currently biased towards the former, but the pros and cons of each of these with respect to API consistency, impact on page configuration UX, and impact on core and contrib developers still needs to be discussed in more detail.
6. New layout system
   • In order to deliver a layout administration tool many helping hands will be required during the previous 4 phases so that this phase, and those following it can progress as close to simultaneously as possible.
   • Layouts are a bit of a new concept for core and what will be done here is breaking the dependency between a “theme" and it’s “page.tpl". Page.tpls will largely go away in favor of reusable layout components that can be controlled and populated through the administrative interface.
   • Layouts are a lightweight plugin type intended to be easy to create for “designers". They will contain their own js, css, administrative css (if necessary), some sort of image representation of what they look like, a configuration xml definition (for the CMI config system) and a tpl file with their appropriate output. The objective is to remove the need for site builders to need to be building their own HTML, and to make building custom HTML for designers insanely simple.
   • Layouts are intended to be nestable and reusable, not only in the sense that you could have multiple of the same layout on a page together with different content, but also in the sense that a pre-configured layout (for example: a header or footer) could be easily reused and nested into a container layout.
7. Logically a library of layouts will develop here and things like grid or responsive layout plugins could exist hopefully lowering the barrier to entry on getting sites that depend on these systems up and running more easily. A strong starter library in core would be a big win.
   • Mentioned previously a theme becomes completely disconnected from a layout, so changing the theme is really about individual elements a theme may care about, and the css intended to exist for given layouts.
   • As core (and other modules) begin providing various layouts, it seems natural that themes would begin to supply css that corresponds to the most popular layout plugins, and/or supply their own layout plugins. Logically layout packages begin to take the place of “base themes".
8. In order to setup all of this work with layouts, a new administrative system will need to come into being that allows for default layouts for a site to be defined, and then some sort of variation on layouts based on url and other potential criteria.

Conclusion

In short, we have a ton to do and I will need as much help as I can get from people familiar with the concepts I’m discussing here or those who are willing to learn and help convert the existing system. My employer, Commerce Guys has agreed to donate 20% of my time to this initiative, which is insanely generous and I am lucky to be in a position to help push this initiative forward.

If you’re interested in getting involved this effort we will probably work in #drupal-wscci on freenode. I’m EclipseGc on IRC, drupal.org and twitter. I look forward to engaging the community in this initiative.

I think we have the opportunity to do something really special here.