Is it time to give Drupal another look?

Ed. note: Sometimes we feature guest authors whose industry experience can help illuminate what it's like to be developer today. This post is written by Tim Lehnen, the Chief Technology Officer of the Drupal Association (501c3), the non-profit organization dedicated to accelerating the Drupal software project, fostering the community, and supporting its growth.

Drupal 9 has just been released, and the open source CMS is approaching its 20th year. For many developers, 'CMS' conjures specters of the late 2000s—a decade in which Internet Explorer still held 65% of browser market share, the HTML5 specification had not yet been proposed, Git was just establishing dominance in the VCS space, and jQuery was the JavaScript library de rigeur. But if that's the image in your head when you think about Drupal, it's time to take another look.

The elePHPant in the room

As I write this post, I'm cognizant that Drupal has the infamous distinction of having earned the place of second Most Dreaded Web Framework in Stack Overflow's 2020 developer survey, and PHP, the underlying language it is written in, is sixth on the Most Dreaded language list.

Certainly that's not a badge that any open source community would wear with pride—so what is it about Drupal 9 that makes it worth a second look?

We could try to shrug this off as simply a result of how widely used Drupal is, but we have better data than that. A sentiment survey of Drupal and other open source and proprietary CMSs found that Drupal's most negative sentiment comes from beginning users, but that sentiment becomes increasingly positive with users' expertise on the platform. It's telling that only the expert level users have a positive impression of the clarity of using Drupal and would generally recommend it to others.

The Drupal community identified a few key areas that drove this sentiment among the developers and end-users surveyed:

• Ease-of-use and total cost of ownership (especially, difficult major version upgrades).
• Improving the beginner experience (for developers, contributors, and end-users).

Let's look at how the Drupal project has addressed the concerns of these beginner and intermediate users, and then look at the fundamental features that make the learning curve worthwhile to users at the expert level.

Even if this exploration doesn't convince you to use Drupal for your next web development project, the lessons in managing a mature open source project should be valuable to contributors in any open source community.

Let's get started.

Making Upgrades Easy

There have been two schools of thought about managing large-scale software projects across many years of development and new major versions:

• Projects can opt to prioritize backwards compatibility as a primary focus. Wordpress for example, "strives to never break backwards compatibility" with the goal that it is "very easy for users to be able to update without worrying."
• • Pro:This extreme focus on backwards compatibility is very friendly to beginner users. For more advanced users or engineering teams managing larger scale deployments, this ethos can make the task of staying up to date less daunting.
  • Con:Maintaining backwards compatibility at this extreme can lead to a lot of codebase cruft, result in a confusing array of APIs that perform the same function, and prevent adoption of new best practices or technologies.
• Projects can choose to focus on the latest architectural innovations and best practices, at the expense of total incompatibility between major versions.
• • Pro: The project can adopt the latest technical standards and innovations, rewriting the technology with all the lessons learned from years of experience with the prior version. Legacy code can be removed, and contributor time is preserved for new features rather focused on compatibility.
  • Con: Any upgrade between major versions is effectively a complete replatforming. Data being moved from one version to another must be migrated, rather than updated in place. The user base has to relearn the platform between major versions.

From its inception through version 7, Drupal followed the latter development philosophy, sacrificing backwards compatibility and easy updates in favor of reinventing the architecture to keep abreast of the latest standards and innovations.

Beginning with Drupal 8, the project made a fundamental shift in development philosophy in order to bridge these two philosophies: adopting semantic versioning (SemVer) and a rigorous deprecation policy. SemVer is still widely misunderstood by many developers, so it's worth reiterating what it means:

• The SemVer version schema is: MAJOR.MINOR.PATCH, where each of the three version components has (naturally) a semantic meaning:
• • Any incompatible API changes must increment the MAJOR version.
  • Adding new functionality in a backwards compatible manner requires incrementing the MINOR version.
  • Backwards compatible bug fixes that don't fundamentally alter functionality increment the PATCH version.

• By adopting SemVer in combination with a carefully designed deprecation policy, Drupal tries to split the difference between these two philosophies
• • Pro: Drupal can now maintain a well-defined window of backwards compatibility that allows for more predictable upgrades, and at the same time can continuously adopt new technologies and best practices.
  • Con: Software developers are not perfect logicians, and there is sometimes subjectivity or ambiguity in what constitutes a backwards-compatibility breaking change. Human error may sometimes mean that the backwards compatibility promise is betrayed in a patch or minor release.

Improving the beginner experience

Improving the beginner experience is a more subjective undertaking, further complicated because there are a number of personas to consider:

• Developers: building on Drupal as a framework.
• Site Builders: assembling and configuring modules and customizing structural components like content types, all in the no-code context of the administrative interface.
• End-users: the content editors that are typically the constituents that developers are building the experience for.

We'll set aside the end-user experience for this examination of Drupal's capabilities because the quality of that experience is heavily dependent on the particular implementation by the development team and site builders who create that experience. Drupal is not opinionated out of the box, so configurations for different use-cases and environments can look vastly different. However, we'll highlight structural features that give Drupal developers a leg up on building those experiences.

Let's look at the capabilities Drupal offers developers and site builders through the lens of what content management systems should offer in 2020.

Content management today is not what it was ten years ago

The problem space of content management today is very different from what it was ten years ago. A decade ago, most implementations were still focused on the basics: providing a monolithic framework that could act as both the editorial content store and as the display layer—typically with a snap-together approach to using third-party modules and themes.

This early understanding of what a CMS should be was widely shared by established CMS projects, and in recent years, has been commoditized by inexpensive proprietary services like Wix.com, Squarespace, and others.

Modern content management systems must provide much more robust capabilities:

• A structured data engine, providing rich metadata, and making content available as reusable components. Structured data forms the basis of all the other capabilities that make a modern CMS robust. When the data itself is structured as atomic components, it can be published into different endpoints or displays, whether that's a straightforward RSS feed for a blog or the structured response of a GraphQL query.
• Robust APIs with complete feature parity between features available via API vs. those available in the administrative UI. Exposing the capabilities of a CMS programmatically is the basis of enabling digital experiences that go beyond traditional web pages: these APIs enable headless or decoupled experiences, where the CMS manages the data but one or more end-points for displaying that data are built as separate front-end applications.
• Editorial and workflow tools, as well as robust roles and permission management to support the content editors that will populate the platform. Why not simply build your own MEA/RN stack, and take responsibility for structuring the data in your database directly and building the full application in a framework like React or Vue.js?

If your use case is well-defined, unlikely to change, and the number of people involved in managing the content is small, that might be a fine solution. However, as soon as you need to define more complex editorial workflows and enforce them with roles and permissions, using a CMS like Drupal can save significant development time.

• Dependency and configuration management.

In any use case of significant scale, managing both dependencies (i.e: third-party libraries) and managing the site's configuration (i.e: defined roles and permissions) in a way that is reproducible and/or version controlled is vital.

• Performance that can compete with app-centric experiences with low time before First Contentful Paint (or newer metrics like Largest Contentful Paint). One of the existential threats to the open web is that closed-garden, app-based experiences offer a dramatically more performant and responsive user experience than what we're used to on the web. But this is a challenge that can be overcome.

Where Drupal stands as a modern CMS

Having defined the requirements of a modern CMS, let's look at how Drupal handles each.

Structured Data (and Metadata)

Drupal's core strength has always been its power as a structured data engine.

Very simply, entities are the base unit of content in Drupal. Entities include things like: nodes (Drupal's genericized content item), comments, users, blocks, taxonomy terms, etc.

Fields represent the data that is stored within an entity. Any individual field can hold one kind of data: plain text, formatted text, images, other files, dates, etc.

Developers can also define their own entities. This example shows the definition of a new Entity called Event which could be used to store structured data about Drupal meetups, and the definition of some basic fields for ID and Universally Unique Identifier.

<?php

namespace Drupal\event\Entity;

use Drupal\Core\Entity\ContentEntityBase;
use Drupal\Core\Field\BaseFieldDefinition;
use Drupal\Core\Entity\EntityTypeInterface;
use Drupal\Core\Entity\ContentEntityInterface;

/**
 * Defines the event entity.
 *
 * @ingroup event
 *
 * @ContentEntityType(
 *   id = "event",
 *   label = @Translation("event"),
 *   base_table = "event",
 *   entity_keys = {
 *     "id" = "id",
 *     "uuid" = "uuid",
 *   },
 * )
 */

class Event extends ContentEntityBase implements ContentEntityInterface {

  public static function baseFieldDefinitions(EntityTypeInterface $entity_type) {

    // Standard field, used as unique if primary index.
    $fields['id'] = BaseFieldDefinition::create('integer')
      ->setLabel(t('ID'))
      ->setDescription(t('The ID of the Event entity.'))
      ->setReadOnly(TRUE);

    // Standard field, unique outside of the scope of the current project.
    $fields['uuid'] = BaseFieldDefinition::create('uuid')
      ->setLabel(t('UUID'))
      ->setDescription(t('The UUID of the Event entity.'))
      ->setReadOnly(TRUE);

    return $fields;
  }
}
?>

Because entities and fields are architecturally central to Drupal, they cleanly inherit all the capabilities that Drupal offers—becoming automatically available in Drupal's APIs, being revisionable and translatable, subject to access control from the central roles and permissions system, and, critically, available in Drupal's structured query builder Views.

What are Views?

Conceptually, Views are a core component of Drupal that allows you to fetch content from the database and structure it into a feed or display.

How is this distinct from just writing a SQL query? Beyond simply querying the structured data, Views also inherit all of the other features of Drupal, allowing you to define multiple displays, make query parameters contextual, and enforce access control with Drupal's core roles and permission system.

While all of this functionality can be managed through the Views UI in the Drupal admin interface, for more granular control Views can be defined in code.

An example of the power of Views

Let's imagine that our Drupal instance stores sports statistics. Perhaps we've built a typical website experience where users can browse these statistics, but our instance also serves content to a digital kiosk installed in a sports stadium. This kiosk does not run on Drupal, but queries our Drupal site using RESTful web services for stats about whatever team a sports fan using the kiosk taps on.

We can define a View of the Sports Team entity within our Drupal site, which gathers all of the Fields containing the team statistics that might be requested by the Kiosk.

We can then configure the Views output with a REST export path that the kiosk will call to for the content, and choose whether to serialize the output as XML or JSON.

Notice that we need to know nothing about how the kiosk is built, except what kind of output format it can consume.

Views in Drupal can save tremendous amounts of development time when displaying structured content across channels and endpoints.

Robust APIs for Content Presentation

Drupal 9 was built with an API-first development approach. This means both that all the application functions of Drupal are available as APIs and that all the structured data stored in Drupal can be made available via Drupal's RESTful web services and JSON:APIs. Support for additional API extensions (such as GraphQL) is available through contributed modules. Critically, these APIs respect all of the other Drupal core features like revisioning, permission management, etc.

We talked about Views above, Drupal's core component that enables the "Compose once, publish anywhere" capability of the CMS. While it's traditionally used to create flexible displays of structured data within Drupal's own presentation layer, we've shown in the example above that it can also create API endpoints to allow content to be consumed and displayed in a decoupled front-end application.

It's increasingly common to see Drupal used in this way. The front-end applications that consume data from a decoupled Drupal installation are most often built in Node.js, React, Vue.js, Gatsby, and similar frameworks.

The choice of whether to use Drupal as a standalone, 'coupled' application, or across the spectrum of decoupled options depends on the editorial and development needs of the project.

Editorial and workflow tools

A content management system is only as useful as its ability to empower content editors to do their work, and development teams tasked with building on Drupal are often given specific requirements for editorial features that editors should be able to use without developer involvement. In Drupal 9, there are structural tools for defining editorial workflows.

Workspaces allow you to define content staging environments for your editorial workflow, previewing multiple content changes, which can then be simultaneously deployed to the public environment.

Within an individual content type, you can define States for content which can have their own attributes, controlling who has access to view or edit the content, changing the default revision, etc.

Transitions define the allowable State changes that users can put content through and these again can be further restricted by roles and permissions.

The Workspace and Workflow configuration can be imported/exported and version controlled by Drupal's configuration management system, which we'll describe below, vastly simplifying the task of managing these editorial workflows across a fleet of installations.

For example, if a developer was tasked with adding a new content approval step to the editorial workflow for multiple editorial teams across a whole suite of publications each on their own instance of the codebase, the configuration can be created on a single environment, exported, and then imported into all of the other environments.

Dependency and configuration management

Composer is the PHP ecosystem's dependency management tool, comparable to npm or Yarn for the JavaScript ecosystem. Drupal 8 first introduced support for Composer-based dependency management, but with Drupal 9, it becomes the preferred way of managing a Drupal installation. Drupal.org's release packaging system ensures that whether a Drupal project is started on the command line or via downloading the .tar.gz, the project is ready to use Composer.

Starting a Drupal project with Composer is straightforward. Once you have a development environment that meets the system requirements (typically a standard LAMP stack), you can install Drupal using the recommended Composer template:

$ composer create-project drupal/recommended-project my_site_name_dir

Additional Drupal modules or themes can be added to your installation with the composer require command, following the appropriate version constraint syntax.

If, for example, you wanted to add the Token module (providing expanded options for inserting dynamically generated values into content) as well as the GraphQL module (because your content is being consumed by another front-end application), you can add these modules to your project with:

$ composer require 'drupal/token'
$ composer require 'drupal/graphql'

This general workflow should be familiar to developers who've used other dependency management tools.

Configuration management

Drupal uses a standard format for all site configuration—whether configuration of Drupal core components, or installed modules. This means that Drupal configuration can be exported and imported as YAML, which allows for staging of configuration changes, deploying configuration between sites, and easy version control.

Within the configuration management UI you can view the active (in database) configuration as well as the staged (in YAML) configuration that is currently available. The active configuration can be exported to override the current staged configuration, or alternately the active configuration can be reverted to the settings in the staged YAML.

As alluded to above, any custom extension of Drupal can define its own configuration as well. The schema for the configuration YAML is quite simple:

# /modules/example/config/schema/example.schema.yml
example.settings:
  type: config_object
  label: 'Example config'
  mapping:
    message:
      type: text
      label: 'Message'

As a developer, this ability to export and import the configuration—and to manage it with version control—makes the site configuration easier to control, auditable, and much easier to deploy from a staging environment to production.

Performance and scalability

Competing with near-instant responsiveness of walled-garden mobile apps means putting a strong emphasis on modern performance and rendering techniques.

Drupal includes a dynamic page cache, requiring no configuration, which uses the declarative cache contexts of each rendered item on a page to determine if it can be cached. This means that even pages with dynamic, personalized content can be cached in part or in whole, whether a user is anonymous or authenticated.

The key to this is in detecting what page content can be cached and what should not be. The process of detecting highly-dynamic elements of a page (i.e: poor candidates for caching) so they can be rendered later is called auto-placeholdering.

By default, Drupal will perform auto-placeholdering of elements that match these conditions in the renderer config:

 renderer.config:
    auto_placeholder_conditions:
      max-age: 0
      contexts: ['session', 'user']
      tags: []

These conditions can be extended for your specific use case. As an example, if you want to eliminate caching for dynamic content that is not related to the user or the current session (perhaps a visualization of real-time activity) you could define a new cache context—let's call it 'dataviz'. Content given that context will be handled via auto-placeholdering, rather than caching.

-  contexts: ['session', 'user']
+ contexts: ['session', 'user', 'dataviz']

Similarly, you may want to do the opposite: defining a cache context for something that Drupal would normally not cache by default. For example, if you had content that was contextual to your user entities, but was largely immutable, you could give it a new cache context to enforce caching on that content without having to cache all other content that might vary dynamically based on the user context.

Drupal also includes the BigPipe core module, the most advanced optimization technique for page rendering available today, which was first proposed and implemented by Facebook. This technique builds on the cache contexts and auto-placeholdering described above to allow the page to render with an early flush (sending the initial content to the browser before the entire page is ready), and then streaming in the placeholder content.

This is a dramatic improvement time to Largest Contentful Paint. BigPipe functionality is enabled by default with no additional configuration required, though it may be extended.

Richness and reach

The increase in technical sophistication in the CMS ecosystem has simultaneously increased the richness of digital experiences and also increased the complexity of building and maintaining those systems. That's changed who Drupal is for.

Within the Drupal community, we say that Drupal is for ambitious digital experiences. This is not exclusively or even primarily a measure of size—but rather a measure of how much a web experience needs to integrate these modern content management concepts we've explored above.

The purpose of a robust CMS like Drupal is no longer in building blogs, portfolio sites, or brochure-ware, and hopefully after this exploration, that is no longer what comes to mind when you think of the CMS ecosystem. Instead, if the modern feature set we've explored here resonates with your use case, it's time to give Drupal 9 a look.

Quick start

If you have LAMP development environment, you can quickly explore a quick-start Drupal environment with Composer and PHP:

$ composer create-project drupal/recommended-project try9
$ php try9/web/core/scripts/drupal quick-start demo_umami

To learn more about Drupal, please explore the Drupal project's documentation, consider joining the Drupal community online or in your area, or check us out at DrupalCon.