D8FTW: REST-aware Routing

My official role in Drupal 8 was as Lead of the Web services Initiative (WSCCI). That is, our mandate was to make Web services better in Drupal. Or, as we phrased it in our mission statement:

Drupal needs to evolve from a first-class CMS to a first-class REST server with a first-class CMS on top of it.

It was that need, to better handle Web services, that drove most of the changes that followed. So what exactly was the problem? How is Drupal 7 not a good Web services platform? Clearly people have implemented Web services using Drupal, right?

They have, but to do so they've had to hack around many assumptions baked into Drupal itself. Drupal 7 and earlier make a number of assumptions that, while they may have been true in 2005, are simply not true today. Specifically, it assumes that every incoming request is for an HTML page, and specifically for the complete page, from &lt;html> tag to &lt;/html> tag. The code for that is rooted in hook_menu:

<?php function example_menu() {   $items['my/page'] = array(     'title' => 'My page',     'page callback' => 'my_page_function',     'access arguments' => array('access content'),   );   return $items; } ?>

In this example, we define a single menu item (or "route", as it's called in most systems including Drupal 8) fixed at the path http://mysite.com/my/page. That is the only route we get for that path, and it maps to my_page_function as the code that will handle the content area of the page. Whatever comes back from my_page_function will be wrapped up in a themed page, unconditionally, because of course the page callback is the body of a full HTML page. We also have only a single access callback (implicitly in this case for a user permission), and cookie-based authentication is the only supported way to identify a request.

Although there are ways around some of those limitations, they are all essentially hacks. Technically Drupal 8 does have a concept of "delivery callbacks" that are invoked on the page callback result, but they're almost never used as they don't happen until after the page callback has already run, rendering it of only very limited use.

And then of course there's the regular use of global variables. That includes PHP's superglobals such as $_GET or $_SESSION, setting HTTP headers or page headers with global function calls (which themselves wrap global variables), and many other patterns that assume one PHP process is producing one complete HTML page in one known, predictable way. (They also make unit testing impractical or impossible in most cases, but that's neither here nor there.)

Drupal 7 is a page-building tool. Everything else is an after-thought. The web in 2015/2016 is not limited to pages.

The cornerstone of WSCCI was fixing that problem, and removing the page-centric assumptions from Core. Routes are now defined in such a way that we can have multiple routes per path, which in turn means that we can map to different controllers (formerly "page callbacks") depending on more than just the path. Here's what the the previous example looks like in Drupal 8, as a YAML file:

example.route:   path: /my/page   defaults:     _controller: '\Drupal\example\Controller\ExampleController::page'     _title: 'Example page'   requirements:     _permission: 'access_examples'     _day: 'Tuesday'     _method: 'GET'

The route now has a machine name, example.route, rather than using the path as an identifier. We're also restricting it to only apply for HTTP GET requests. We can have another route, with its own machine name, that has the same path but matches a different HTTP method like PUT or POST, that maps to an entirely different controller and thus have different logic.

We're also able to stack multiple access checks on the same route. In this case, we're flagging that we should check for the user permission "access_examples" and for a day of "Tuesday". (There is code elsewhere that leverages that information to perform the check; we won't go into details here.)

Core supports many, but not all, of the ways to select different routes based on incoming HTTP information, such as the path, the HTTP method, the format of the request, the domain, or the scheme (HTTP vs HTTPS). It does so in a pluggable fashion, though, so contrib modules can inject their own logic, too. A somewhat over-simplified diagram of the process is below:

The RouteProvider service does the initial lookup for the request compared against all routes defined in the entire system, using the path only and logic very similar to that from Drupal 7. It may return multiple routes, though, if multiple routes are defined with the same path. One or more Route Filters, then, can filter that list down even further. Contributed modules are able to add additional Route Filters with any arbitrary logic they want (although they run on every request, so be very wary of performance). Finally, the FinalMatcher applies the last checks and selects the one matching route that will get used for the request.

Modules like Page Manager, Services, or RESTful in Drupal 7 needed to, in essence, build their own routing system on top of the one provided in core in order to handle their non-path-centric use cases. In Drupal 8, such modules need only slot into the existing architecture in a clearly defined and standard way.

The other big change (at least that's relevant for our purposes) is what happens to the result from the controller. In Drupal 7, recall, it was always treated as the body of a page. In Drupal 8, the controller may return a render array, which means it will be used as the body of a page, but it can also return any PHP classed object, including a complete Response object. See below.

If the controller returns a Response, we're done. If not, a series of "view listeners" run in order until one decides it knows what to do with the result, producing a Response. Core provides one that recognizes a render array, does the usual theming and page layout, and produces an HTML page that gets shoved into a Response object. That's what most requests use in practice. However, we can also return any object we want as long as there's a listener that knows how to translate into a Response. That Response could be a page, but could also be a page fragment, or a JSON response, or XML, or whatever. In either case, Response Listeners can then apply their own logic to the Response, uniformly, to handle caching and so forth.

In essence, the page-rendering logic is now opt-in rather than opt-out (or really, hack-around. That makes the whole system cleaner and removes assumptions we need to work around for Web services.

Incidentally, the bulk of this architecture wasn't Drupal-original. The core pipeline is the Symfony HttpKernel component and Routing component. The Route Filters concept was co-developed by Drupal developer and Symfony CMF in the CMF extended Router component, and is now used by Drupal 8, Symfony CMF, and ezPublish. It was that outsourcing and collaboration that helped spur Drupal to finally break out of its Not-Invented-Here habits and embrace Proudly-Invented-Elsewhere (PIE).

In Drupal 7, everything is a page (unless you dance around a bit). In Drupal 8, everything is an HTTP response, which is sometimes a page.

Why is this a win for Web services?

The core pipeline now lets us address all of HTTP. We can provide different logic at the same path depending on other criteria than the path, and we can easily return responses that are not HTML pages without Drupal getting in our way.

A single REST resource has a single URI, but that URI may fork to different controllers depending on the HTTP method, the content type of a POST or PUT request, the requested type for a GET request, or anything else we want. And we can bypass the "page" logic, returning a complete HTTP response object either from the controller or downstream in a view listener, depending on what fits our use case.

Just that alone allows us to build up as far as level 2 of the Richardson Maturity Model quite easily. But core goes a step further, giving us tools to simplify the process even more. We'll cover that in our next episode…

A note about this series: Web Services in today's applications and websites have become critical to interacting with third parties, and a lot of Drupal developers have the need to expose content and features on their site via an API. Luckily for us, Drupal 8 now has this capability built right into Core. Some contrib modules are attempting to make such capabilities even better, too.

To shed some light onto these new features, we've worked with Acquia to develop a webinar and subsequent series of blog posts to help get you up to speed with these exciting, new features. The first of these blog posts, Web Services 101, has been published on the Acquia Developer Center previously, written by our very own Senior Architect and Community Lead Larry "Crell" Garfield.

Larry follows up with this post in the series by continuing his comprehensive explanation of exactly what Web services are, providing a necessary and strong foundation for you to approach the exciting Web services developments new to Drupal 8. Look for a follow-up next week on the Palantir.net blog.

This second post is part of a 4-part series written by Larry, and Kyle Browning, of Acquia, based on a webinar that Larry and Kyle recently gave: Drupal 8 Deep Dive: What It Means for Developers Now that REST Is in Core

Original Article:

D8FTW: REST-aware Routing