PreviousNext: Vite and Storybook frontend tooling for Drupal

We’ve just completed an extensive overhaul of our frontend tooling, with Vite and Storybook at the centre. Let’s go over it piece by piece.

by jack.taranto / 2 October 2024

The goal of the overhaul was to modernise all aspects of the build stack, remove legacy dependencies and optimise development processes.

Tooling is split into four pieces: asset building, styleguide, linting and testing.

Asset building for Drupal with Vite

We have always utilised two separate tools to build CSS and JS assets. Until now, this was PostCSS and Rollup, in the past Sass and Webpack have been in the mix. 

With Vite it’s one tool to build both types of assets. To introduce Vite to anyone not already familiar with it, I would say it’s a super fast version of Rollup without the configuration headaches. 

Moving to Vite sped up our development build times and production build times (in CI), simplified our config files and removed a huge number of NPM dependencies.

Vite library mode

A typical Vite build pipeline is most suitable for single-page apps. It involves an index.html file where Vite dynamically adds CSS and JS assets. However, with Drupal, we do not have an index.html file; we have the Drupal libraries system to load assets, with which Vite has no way of communicating.

Luckily, Vite ships with something called Library mode, which is seemingly tailor-made for Drupal assets! Library mode allows us to output all our frontend assets to a single directory, where we can include them in a libraries.yml file or via a Pinto Theme Object.

To use our config, you’ll first need a few dependencies. 

npm i -D vite postcss-preset-env tinyglobby browserslist-to-esbuild

Our vite.config.js looks like this:

import { defineConfig } from 'vite' import { resolve } from 'path' import { globSync } from 'tinyglobby' import browserslist from 'browserslist-to-esbuild' import postcssPresetEnv from 'postcss-preset-env' const entry = globSync(['**/*.entry.js', '**/*.css'], { ignore: [   '**/_*.css',   'node_modules',   'vendor',   'web/sites',   'web/core',   'web/libraries',   '**/contrib',   'web/storybook', ], }) export default defineConfig(({ mode }) => ({ build: {   lib: {     entry,     formats: ['es'],   },   target: browserslist(),   cssCodeSplit: true,   outDir: resolve(import.meta.dirname, './web/libraries/library-name'),   sourcemap: mode === 'development', }, css: {   postcss: {     plugins: [       postcssPresetEnv(),     ],     map: mode === 'development',   }, }, }))

We define entry points as any *.css file and any *.entry.js file. We exclude certain directories, so we aren’t building assets that are included with core or contrib. Additionally, we exclude CSS partials, which use an underscore prefix. This allows us to add asset source files anywhere in our project. They could be added in the theme, a module, or (as we have been doing recently) inside a /components directory in the project root.

The Vite config itself enables library mode using build.lib, passing all source assets through using build.lib.entry and building JS assets using the es format.

build.cssCodeSplit is required when passing CSS files through to build.lib.entry.

build.outDir specifies a folder inside the Drupal libraries directory where all built assets will be sent. Drupal libraries.yml definitions are then updated to include files from this directory.

build.sourcemap will output JS sourcemaps in development mode only.

Finally, we pass through any PostCSS plugins with css.postcss.plugins. Vite includes postcss-import by default, so you do not need to add that. It will also handle resolving to custom directories without including resolve options for postcss-import, meaning you’ll only need to add your specific plugins. In this case, we reduced ours to just postcss-preset-env. Add more as needed!

We also enable CSS sourcemaps with css.postcss.map.

This config allowed us to completely remove the PostCSS config file, PostCSS CLI, Rollup, its config and all Rollup plugins.

The config file above is a starting point—a minimum viable setup you’ll need to build assets using Vite’s library mode. Add to it as you need to, and familiarise yourself with Vite’s documentation.

Using Browerslist with Vite

Vite uses ESBuild to determine the output feature set based on the build.target. For many years now, we have used Browserslist to determine feature sets for both PostCSS and Rollup, and it works really well. We weren’t ready to lose this functionality by moving to Vite.

This is where the browserslist-to-esbuild dependency comes in. We added the following .browserlistrc file to our project root:

> 1% in AU

By calling browserslist() in build.target we get our browser feature set provided by Browserslist instead of ESBuild.

NPM scripts for development mode and production builds

We use NPM scripts for consistent usage of non-standard commands both locally and on CI for production builds.

"scripts": { "dev-vite": "vite build -w -m development", "build-vite": "vite build" },

To watch and build source assets whilst developing locally, we use npm run dev-vite. Unlike Vite’s dev command, this still uses Rollup under the hood (instead of ESBuild), so we miss out on the extreme speed of Vite’s dev mode. However, it’s still very fast—faster than default Rollup. It’s a tradeoff that provides what we need, which is building our assets while we are editing them in a way that works with Drupal. We lose hot reloading, but that’s less important when we have Storybook at our disposal.

Production builds happen on CI using npm run build-vite.

Using Storybook with Drupal

Although we had been using Storybook in our projects for some time now, we hadn’t yet standardised on it or provided a default setup. And with Vite now baked into Storybook, it seemed like an excellent time to provide this.

If you have a spare 15 minutes, I would first suggest checking out Lee Rowland’s lightning talk from Drupal South to see just how fluid a frontend development experience Storybook brings to Drupal.

Storybook is easy to setup using its wizard with:

npx storybook@latest init

It will present you with a few choices. Just make sure you choose HTML and Vite for your project type. When using Vite with Storybook, Storybook provides its necessary config to Vite; however, it will still read your projects vite.config.js file for any additional config. This includes the PostCSS config we setup above and any additional functionality you provide.

Now, install Lee’s Twig plugin. This plugin will allow us to write components using Twig that can be imported into our stories.js files. First, install the plugin:

npm i -D vite-plugin-twig-drupal

Then register the plugin by adding the following lines to the vite.config.js default export:

plugins: [ twig(), ],

See the vite-plugin-twig-drupal documentation for more details, including how to set up Twig namespaces.

Writing stories

To use Twig in Storybook, it’s quite similar to any other framework. Here’s an example story of a card component:

import Component from './card.html.twig' const meta = { component: Component, args: {   title: `<a href="#">Card title</a>`,   description:     'Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam eu turpis molestie, dictum est a, mattis tellus. Sed dignissim, metus nec fringilla accumsan, risus sem sollicitudin lacus.', }, } export default meta export const Card = {}

We import the twig file as Component and then add that to the stories meta. We can pass through args, which will show up in the Twig file as variables, and we can use HTML here.

Writing stories is covered in more detail in our front-end nirvana blog post.

NPM scripts for developing with Vite and Storybook at once

Our standard development practice involves building and testing components in Storybook and then integrating them with Drupal using Pinto. To do this, we need to run Storybook and our Vite tooling at once so we have both Storybook dev mode and our built frontend assets available to us.

Running two NPM scripts in parallel can be a pain, so we have implemented concurrently to streamline this approach.

npm i -D concurrently

Then we use the following in our NPM scripts:

{ "scripts": {   "dev": "concurrently -k -n \"VITE,STORYBOOK\" -c \"#636cff,#ff4785\" \"npm run dev-vite\" \"npm run dev-storybook\"",   "build": "concurrently -n \"VITE,STORYBOOK\" -c \"#636cff,#ff4785\" \"npm run build-vite\" \"npm run build-storybook\"",   "dev-storybook": "storybook dev -p 6006 --no-open",   "build-storybook": "storybook build -o web/storybook",   "dev-vite": "vite build -w -m development",   "build-vite": "vite build" },

With npm run dev we get coloured output so we can see which tool is running and what it’s doing. npm run build is used on CI.

Linting with Prettier, Stylelint and ESLint

These three tools have been a staple on our projects for a long time, but with ESLint introducing a new flat configuration method, it seemed like a good time to review the tooling.

First, we’ll need some more dependencies.

npm i -D prettier stylelint stylelint-config-standard eslint@8.57.0 @eslint/js@8.57.0 eslint-config-prettier eslint-config-drupal

Formatting source assets with Prettier

We are using Prettier to format both CSS and JS files. With PHPStorm, you can set this to happen on file save. We also have an NPM script to do this on demand and before committing. NPM commands are at the end of this section.

Reducing Stylelint configuration

Past iterations of our Stylelint tooling involved extensive configuration on each project. Using Stylelints latest standard configuration, it sets sensible defaults, which lets us remove most config options. We’re left with the following:

const config = { extends: ['stylelint-config-standard'], rules: {   'custom-property-empty-line-before': null,   'no-descending-specificity': null,   'import-notation': 'string',   'selector-class-pattern': [     '^([a-z])([a-z0-9]+)(-[a-z0-9]+)?(((--)?(__)?)([a-z0-9]+)(-[a-z0-9]+)?)?$',     {       message:         'Expected class selector to be BEM selector matching either .block__element or .block--modifier',     },   ],   'selector-nested-pattern': '^&', }, } export default config

We added a custom rule to ensure project BEM selectors are used.

Like prettier, we also use a .stylelintignore file to exclude core and contrib folders.

Moving to ESLint flat config

The new config format isn’t yet supported by all plugins (there’s a compatibility tool to help with this), but where it is, it’s much simpler.

The following config can be used in conjunction with Prettier.

import js from '@eslint/js' import globals from 'globals' import prettier from 'eslint-config-prettier' import drupal from 'eslint-config-drupal' export default [ js.configs.recommended, prettier, {   languageOptions: {     globals: {       ...globals.browser,       ...globals.node,       ...drupal.globals,       dataLayer: true,       google: true,       once: true,     },   }, }, {   rules: {     'no-console': 'error',     'no-unused-expressions': [       'error',       {         allowShortCircuit: true,         allowTernary: true,       },     ],     'consistent-return': 'warn',     'no-unused-vars': 'off',   }, }, {   ignores: [     'node_modules',     'vendor',     'bin',     'web/core',     'web/sites',     'web/modules/contrib',     'web/themes/contrib',     'web/profiles/contrib',     'web/libraries',     'web/storybook',   ], }, ]

This includes linting for Storybook files and tests as well. Additionally, it ignores core and contrib files.

NPM scripts for linting

We use the following NPM scripts to run our linting commands locally and on CI.

"scripts": { "format": "prettier --write \"**/*.{css,ts,tsx,js,jsx,json}\"", "lint": "npm run lint-prettier && npm run lint-css && npm run lint-js", "lint-prettier": "prettier --check \"**/*.{css,ts,tsx,js,jsx,json}\"", "lint-css": "stylelint \"**/*.css\"", "lint-js": "eslint ." },

These commands work so well because we have excluded all Drupal core and contrib folders using ignore files. 

Testing using Storybook test runner

Storybook test runner provides the boilerplate-free ability to run automated snapshot and accessibility tests on each story in Storybook. Our previous test tooling involved using Jest and Axe to handle this, but we needed to manually write tests for each component. With Storybook test runner, this is handled automatically.

To set it up, first, install some dependencies.

npm i -D @storybook/test-runner axe-playwright

Then create the following test-runner.js file inside your .storybook directory.

import { waitForPageReady } from '@storybook/test-runner' import { injectAxe, checkA11y } from 'axe-playwright' import { expect } from '@storybook/test'; /* * See https://storybook.js.org/docss/writing-tests/test-runner#test-hook-api * to learn more about the test-runner hooks API. */ const config = { async preVisit(page) {   await injectAxe(page) }, async postVisit(page) {   await waitForPageReady(page)   // Automated snapshot testing for each story.   const elementHandler = await page.$('#storybook-root')   const innerHTML = await elementHandler.innerHTML()   expect(innerHTML).toMatchSnapshot()   // Automated accessibility testing for each story.   await checkA11y(page, '#storybook-root', {     detailedReport: true,     detailedReportOptions: {       html: true,     },   }) }, } export default config

This config will loop through all your stories, wait for them to be ready, then snapshot them and run Axe against them. You’ll get great output from the command, so you can see exactly what’s going on.

NPM scripts for testing Storybook locally and on CI

First, install a few more dependencies:

npm i -D http-server wait-on

The following scripts will run the complete Storybook test base and update snapshots as needed.

"scripts": { "test-storybook": "test-storybook", "test-storybook:update": "test-storybook -u", "test-storybook:ci": "concurrently -k -s first -n \"SERVER,TEST\" -c \"magenta,blue\" \"npm run http-server\" \"wait-on tcp:6006 && npm run test-storybook\"", "http-server": "http-server web/storybook -p 6006 --silent" },

To run tests on CI we use http-server to serve the built version of Storybook and wait-on to delay the test run until the server is ready. The concurrently command smooths the output of both these commands.

Wrapping up

See the complete workflow, including all config and ignore files in the pnx-frontend-build-tools-blog repository I've setup for this post.

The repository and this blog post have been designed to provide the necessary pieces so you can implement this workflow on your existing (or new) projects. However, a lot more functionality can be gained, including easily adding support for Typescript, React and Vitest.

Tagged

Storybook, Vite

Drupal blog: State of Drupal presentation (September 2024)

This blog has been re-posted and edited with permission from Dries Buytaert's blog.

DrupalCon Barcelona 2024 Driesnote presentation

Approximately 1,100 Drupal enthusiasts gathered in Barcelona, Spain, last week for DrupalCon Europe. As per tradition, I delivered my State of Drupal keynote, often referred to as the "DriesNote".

If you missed it, you can watch the video or download my slides (177 MB).

In my keynote, I gave an update on Drupal Starshot, an ambitious initiative we launched at DrupalCon Portland 2024. Originally called Drupal Starshot, inspired by President Kennedy's Moonshot challenge, the product is now officially named Drupal CMS.

The goal of Drupal CMS is to set the standard for no-code website building. It will allow non-technical users, like marketers, content creators, and site builders, to create digital experiences with ease, without compromising on the power and flexibility that Drupal is known for.

A four-month progress report

Image removed.

A preview of Drupal.org's front page with the updated Drupal brand and content.

While Kennedy gave NASA eight years, I set a goal to deliver the first version of Drupal CMS in just eight months. It's been four months since DrupalCon Portland, which means we're halfway through.

So in my keynote, I shared our progress and gave a 35-minute demo of what we've built so far. The demo highlights how a fictional marketer, Sarah, can build a powerful website in just hours with minimal help from a developer. Along her journey, I showcased the following key innovations:

  1. A new brand for a new market: A brand refresh of Drupal.org, designed to appeal to both marketers and developers. The first pages are ready and available for preview at new.drupal.org, with more pages launching in the coming months.
  2. A trial experience: A trial experience that lets you try Drupal CMS with a single click, eliminating long-standing adoption barriers for new users. Built with WebAssembly, it runs entirely in the browser – no servers to install or manage.
  3. An improved installer: An installer that lets users install recipes – pre-built features that combine modules, configuration, and default content for common website needs. Recipes bundle years of expertise into repeatable, shareable solutions.
  4. Events recipe: A simple events website that used to take an experienced developer a day to build can now be created in just a few clicks by non-developers.
  5. Project Browser support for recipes: Users can now browse the Drupal CMS recipes in the Project Browser, and install them in seconds.
  6. First page of documentation: New documentation created specifically for end users. Clear, effective documentation is key to Drupal CMS's success, so we began by writing a single page as a model for the quality and style we aim to achieve.
  7. AI for site building: AI agents capable of creating content types, configuring fields, building Views, forms, and more. These agents will transform how people build and manage websites with Drupal.
  8. Responsible AI policy: To ensure responsible AI development, we've created a Responsible AI policy. I'll share more details in an upcoming blog, but the policy focuses on four key principles: human-in-the-loop, transparency, swappable large language models (LLMs), and clear guidance.
  9. SEO Recipe: Combines and configures all the essential Drupal modules to optimize a Drupal site for search engines.
  10. 14 recipes in development: In addition to the Events and SEO recipes, 12 more are in development with the help of our Drupal Certified Partners. Each Drupal CMS recipe addresses a common marketing use case outlined in our product strategy. We showcased both the process and progress during the Initiative Lead Keynote for some of the tracks. After DrupalCon, we'll begin developing even more recipes and invite additional contributors to join the effort.
  11. AI-assisted content migration: AI will crawl your source website and handle complex tasks like mapping unstructured HTML to structured Drupal content types in your destination site, making migrations faster and easier. This could be a game-changer for website migrations.
  12. Experience Builder: An early preview of a brand new, out-of-the-box tool for content creators and designers, offering layout design, page building, basic theming and content editing tools. This is the first time I've showcased our progress on stage at a DrupalCon.
  13. Future-proof admin UI with React: Our strategy for modernizing Drupal's backend UI with React.
  14. The "Adopt-a-Document" initiative: A strategy and funding model for creating comprehensive documentation for Drupal CMS. If successful, I'm hopeful we can expand this model to other areas of Drupal. For more details, please read the announcement on drupal.org.
  15. Global Documentation Lead: The Drupal Association's commitment to hire a dedicated Documentation Lead, responsible for managing all aspects of Drupal's documentation, beyond just Drupal CMS.

The feedback on my presentation has been incredible, both online and in-person. The room was buzzing with energy and positivity! I highly recommend watching the recording.

Attendees were especially excited about the AI capabilities, Experience Builder, and recipes. I share their enthusiasm as these capabilities are transformative for Drupal.

Many of these features are designed with non-developers in mind. Our goal is to broaden Drupal's reach beyond its traditional user base and reach more people than ever before.

Release schedule

Our launch plan targets Drupal CMS's release on Drupal's upcoming birthday: January 15, 2025. It's also just a couple of weeks after the Drupal 7 End of Life, marking the end of one era and the beginning of another.

The next milestone is DrupalCon Singapore, taking place from December 9–11, 2024, less than 3 months away. We hope to have a release candidate ready by then.

Now that we're back from DrupalCon and have key milestone dates set, there is a lot to coordinate and plan in the coming weeks, so stay tuned for updates.

Call for contribution

Ambitious? Yes. But achievable if we work together. That's why I'm calling on all of you to get involved with Drupal CMS. Whether it's building recipes, enhancing the Experience Builder, creating AI agents, writing tests, improving documentation, or conducting usability testing – there are countless ways to contribute and make a difference. If you're ready to get involved, visit https://drupal.org/starshot to learn how to get started.

Thank you

This effort has involved so many people that I can't name them all, but I want to give a huge thank you to the Drupal CMS Leadership Team, who I've been working with closely every week: Cristina Chumillas (Lullabot), Gábor Hojtsy (Acquia), Lenny Moskalyk (Drupal Association), Pamela Barone (Technocrat), Suzanne Dergacheva (Evolving Web), and Tim Plunkett (Acquia).

A special shoutout goes to the demo team we assembled for my presentation: Adam Hoenich (Acquia), Amber Matz (Drupalize.me), Ash Sullivan (Acquia), Jamie Abrahams (FreelyGive), Jim Birch (Kanopi), Joe Shindelar (Drupalize.me), John Doyle (Digital Polygon), Lauri Timmanee (Acquia), Marcus Johansson (FreelyGive), Martin Anderson-Clutz (Acquia), Matt Glaman (Acquia), Matthew Grasmick (Acquia), Michael Donovan (Acquia), Tiffany Farriss (Palantir.net), and Tim Lehnen (Drupal Association).

I also want to thank the Drupal CMS track leads and contributors for their development work. Additionally, I'd like to recognize the Drupal Core Committers, Drupal Association staff, Drupal Association Board of Directors, and Certified Drupal Partners for continued support and leadership. There are so many people and organizations whose contributions deserve recognition that I can't list everyone individually, partly to avoid the risk of overlooking anyone. Please know your efforts are deeply appreciated.

Lastly, thank you to everyone who helped make DrupalCon Barcelona a success. It was excellent!

The Drop Times: SystemSeed Explores Human-Centered Design at DrupalCon Barcelona 2024

At DrupalCon Barcelona 2024, Elise West of SystemSeed presented a session on Human-Centered Design (HCD), explaining its growing relevance in Drupal projects. The session highlighted HCD’s role in aligning development with user needs, making it essential for project managers, developers, and product leads. More insights from SystemSeed's experience will follow.

ComputerMinds.co.uk: Automatically generate forms from config schema

Image removed.

Drupal's form API has been brilliant for many years. Still, recently I found myself wondering why I needed to build a configuration form if I already had a schema for my config. Defining a schema facilitates API-first validation (including some pretty smart constraints), specific typing (e.g. actual booleans or integers instead of '0' or '1' strings), and even translation in Drupal. 

That last part got me thinking; if Drupal automatically provides translation forms for typed configuration, why must I build a form? I started diving into the code and found config_translation_config_schema_info_alter() which maps certain config data types to element classes. The ConfigTranslationFormBase::buildForm() class fetches the schema for each config property from the 'config.typed' service (\Drupal\Core\Config\TypedConfigManager) before building the appropriate elements. So Drupal core automatically provides this translation form - notice the long textarea for the 'body' property:

Image removed. Screenshot of a config translation form from Drupal core

I had built a block plugin that needed some regex-based validation on a couple of its configuration properties. Validation constraints seemed like a natural fit for these, as an inherent part of the property definitions, rather than just on the form level. Drupal has had good support for validation constraints on configuration since version 10.2. This allows forms to be simpler, and config to be fully validatable, even outside the context of a form (e.g. for setting via APIs or config synchronisation). So I defined my config schema like this:

block.settings.mymodule_myblock: type: block_settings label: 'MyBlock block settings' mapping: svcid: type: string label: 'Service ID' constraints: Regex: pattern: '/^[a-zA-Z0-9_\-]+$/' message: "The %value can only contain simple letters, numbers, underscores or hyphens." default: 'abcde' locked: true envid: type: string label: 'Environment ID' constraints: Regex: pattern: '/^[a-zA-Z0-9_\-]+$/' message: "The %value can only contain simple letters, numbers, underscores or hyphens." default: 'x-j9WsahRe_1An51DhErab-C'

Then I set myself the challenge of building a configuration form 'automatically' from this schema - without using core's config_translation module at all, as this was for a monolingual site. 

I only had two string properties, which meant two textfields, but I wrote the code to use form elements that could be appropriate for other types of property that might get added in the future. The #title of each element could come directly from each label in the schema. (Why do we usually set these in both the schema and form element?!) I added custom default and locked properties to the schema to help encapsulate everything I considered 'inherent' to each part of the config in one place. This meant the configuration form for my block could be fairly simple:

public function blockForm($form, FormStateInterface $form_state) { // Each config property will be returned with its schema from $this->getConfigurables(). foreach ($this->getConfigurables() as $key => $schema_info) { $form[$key] = [ '#type' => match ($schema_info['type']) { 'string', 'label' => 'textfield', 'text' => 'textarea', 'boolean' => 'checkbox', 'integer', 'float' => 'number', 'email' => 'email', }, '#title' => $schema_info['label'], '#default_value' => $this->configuration[$key], '#required' => empty($schema_info['nullable']), '#disabled' => $schema_info['locked'] ?? FALSE, ]; } return $form; }

Hopefully that gives an idea of how simple a config form could be - and this could really be reduced further by refactoring it into a generic trait. The code in core's config_translation module for mapping the type of each property to an element type could be much more useful than the fairly naïve match statement above, if it was refactored out to be available even to monolingual sites.

You can explore my full code at https://gist.github.com/anotherjames/bcb7ba55ec56359240b26d322fe2f5a5. That includes the getConfigurables() method which pulls the schema from the TypedConfigManager.

You'll see that I went a little further and picked up the regex constraints for each config property, for use in #pattern form API properties. This provides quick feedback to admins about what characters are allowed using the HTML5 pattern attribute:

Image removed.

Not all configuration constraints could be built into the form level. It's arguable that since the Regex constraint and HTML pattern attribute support slightly different regular expression features, this particular one shouldn't be included in a generic trait. Then again, the Choice constraint could be especially useful to include, as it could be used to populate #options for select, radios, or checkboxes elements. We've started using backed Enums with labels for fixed sets of options. Can we wire those up to choice constraints together, I wonder?

Whereas my example was for a configurable plugin's form (which I don't believe can use #config_target), Joachim Noreiko (joachim) has submitted a feature request to Drupal core for forms extending ConfigFormBase to get automatically built from schema. This idea of generating form elements from config schema is still in its infancy, so its limits and benefits need to be explored further. Please let us know in a comment here, or in Joachim's feature request, if you have done anything similar, or have ideas or concerns to point out!

Dries Buytaert: Solving the Maker-Taker problem

Image removed.

Recently, a public dispute has emerged between WordPress co-founder Matt Mullenweg and hosting company WP Engine. Matt has accused WP Engine of misleading users through its branding and profiting from WordPress without adequately contributing back to the project.

As the Founder and Project Lead of Drupal, another major open source Content Management System (CMS), I hesitated to weigh in on this debate, as this could be perceived as opportunistic. In the end, I decided to share my perspective because this conflict affects the broader open source community.

I've known Matt Mullenweg since the early days, and we've grown both our open source projects and companies alongside each other. With our shared interests and backgrounds, I consider Matt a good friend and can relate uniquely to him. Equally valuable to me are my relationships with WP Engine's leadership, including CEO Heather Brunner and Founder Jason Cohen, both of whom I've met several times. I have deep admiration for what they've achieved with WP Engine.

Although this post was prompted by the controversy between Automattic and WP Engine, it is not about them. I don't have insight into their respective contributions to WordPress, and I'm not here to judge. I've made an effort to keep this post as neutral as possible.

Instead, this post is about two key challenges that many open source projects face:

  1. The imbalance between major contributors and those who contribute minimally, and how this harms open source communities.
  2. The lack of an environment that supports the fair coexistence of open source businesses.

These issues could discourage entrepreneurs from starting open source businesses, which could harm the future of open source. My goal is to spark a constructive dialogue on creating a more equitable and sustainable open source ecosystem. By solving these challenges, we can build a stronger future for open source.

This post explores the "Maker-Taker problem" in open source, using Drupal's contribution credit system as a model for fairly incentivizing and recognizing contributors. It suggests how WordPress and other open source projects could benefit from adopting a similar system. While this is unsolicited advice, I believe this approach could help the WordPress community heal, rebuild trust, and advance open source productively for everyone.

The Maker-Taker problem

At the heart of this issue is the Maker-Taker problem, where creators of open source software ("Makers") see their work being used by others, often service providers, who profit from it without contributing back in a meaningful or fair way ("Takers").

Five years ago, I wrote a blog post called Balancing Makers and Takers to scale and sustain Open Source, where I defined these concepts:

The difference between Makers and Takers is not always 100% clear, but as a rule of thumb, Makers directly invest in growing both their business and the open source project. Takers are solely focused on growing their business and let others take care of the open source project they rely on.

In that post, I also explain how Takers can harm open source projects. By not contributing back meaningfully, Takers gain an unfair advantage over Makers who support the open source project. This can discourage Makers from keeping their level of contribution up, as they need to divert resources to stay competitive, which can ultimately hurt the health and growth of the project:

Takers harm open source projects. An aggressive Taker can induce Makers to behave in a more selfish manner and reduce or stop their contributions to open source altogether. Takers can turn Makers into Takers.

Solving the Maker-Taker challenge is one of the biggest remaining hurdles in open source. Successfully addressing this could lead to the creation of tens of thousands of new open source businesses while also improving the sustainability, growth, and competitiveness of open source – making a positive impact on the world.

Drupal's approach: the Contribution Credit System

In Drupal, we've adopted a positive approach to encourage organizations to become Makers rather than relying on punitive measures. Our approach stems from a key insight, also explained in my Makers and Takers blog post: customers are a "common good" for an open source project, not a "public good".

Since a customer can choose only one service provider, that choice directly impacts the health of the open source project. When a customer selects a Maker, part of their revenue is reinvested into the project. However, if they choose a Taker, the project sees little to no benefit. This means that open source projects grow faster when commercial work flows to Makers and away from Takers.

For this reason, it's crucial for an open source community to:

  1. Clearly identify the Makers and Takers within their ecosystem
  2. Actively support and promote their Makers
  3. Educate end users about the importance of choosing Makers

To address these needs and solve the Maker-Taker problem in Drupal, I proposed a contribution credit system 10 years ago. The concept was straightforward: incentivize organizations to contribute to Drupal by giving them tangible recognition for their efforts.

We've since implemented this system in partnership with the Drupal Association, our non-profit organization. The Drupal Association transparently tracks contributions from both individuals and organizations. Each contribution earns credits, and the more you contribute, the more visibility you gain on Drupal.org (visited by millions monthly) and at events like DrupalCon (attended by thousands). You can earn credits by contributing code, submitting case studies, organizing events, writing documentation, financially supporting the Drupal Association, and more.

Image removed.A screenshot of an issue comment on Drupal.org. You can see that jamadar worked on this patch as a volunteer, but also as part of his day job working for TATA Consultancy Services on behalf of their customer, Pfizer.

Drupal's credit system is unique and groundbreaking within the Open Source community. The Drupal contribution credit system serves two key purposes: it helps us identify who our Makers and Takers are, and it allows us to guide end users towards doing business with our Makers.

Here is how we accomplish this:

  • Certain benefits, like event sponsorships or advertising on Drupal.org, are reserved for organizations with a minimum number of credits.
  • The Drupal marketplace only lists Makers, ranking them by their contributions. Organizations that stop contributing gradually drop in ranking and are eventually removed.
  • We encourage end users to require open source contributions from their vendors. Drupal users like Pfizer and the State of Georgia only allow Makers to apply in their vendor selection process.
Image removed.A slide from my recent DrupalCon Barcelona State of Drupal keynote showcasing key contributors to Drupal Starshot. This slide showcases how we recognize and celebrate Makers in our community, encouraging active participation in the project.

Governance and fairness

To make sure the contribution credit system is fair, it benefits from the oversight from an independent, neutral party.

In the Drupal ecosystem, the Drupal Association fulfills this crucial role. The Drupal Association operates independently, free from control by any single company within the Drupal ecosystem. Some of the Drupal Association's responsibilities include:

  1. Organizing DrupalCons
  2. Managing Drupal.org
  3. Overseeing the contribution tracking and credit system

It's important to note that while I serve on the Drupal Association's Board, I am just one of 12 members and have not held the Chair position for several years. My company, Acquia, receives no preferential treatment in the credit system; the visibility of any organization, including Acquia, is solely determined by its contributions over the preceding twelve months. This structure ensures fairness and encourages active participation from all members of the Drupal community.

Drupal's credit system certainly isn't perfect. It is hard to accurately track and fairly value diverse contributions like code, documentation, mentorship, marketing, event organization, etc. Some organizations have tried to game the system, while others question whether the cost-benefit is worthwhile.

As a result, Drupal's credit system has evolved significantly since I first proposed it ten years ago. The Drupal Association continually works to improve the system, aiming for a credit structure that genuinely drives positive behavior.

[newsletter-blog]

Recommendations for WordPress

WordPress has already taken steps to address the Maker-Taker challenge through initiatives like the Five for the Future program, which encourages organizations to contribute 5% of their resources to WordPress development.

Building on this foundation, I believe WordPress could benefit from adopting a contribution credit system similar to Drupal's. This system would likely require the following steps to be taken:

  1. Expanding the current governance model to be more distributed.
  2. Providing clear definitions of Makers and Takers within the ecosystem.
  3. Implementing a fair and objective system for tracking and valuing various types of contributions.
  4. Implementing a structured system of rewards for Makers who meet specific contribution thresholds, such as priority placement in the WordPress marketplace, increased visibility on WordPress.org, opportunities to exhibit at WordPress events, or access to key services.

This approach addresses both key challenges highlighted in the introduction: it balances contributions by incentivizing major involvement, and it creates an environment where open source businesses of all sizes can compete fairly based on their contributions to the community.

Conclusion

Addressing the Maker-Taker challenge is essential for the long-term sustainability of open source projects. Drupal's approach may provide a constructive solution not just for WordPress, but for other communities facing similar issues.

By transparently rewarding contributions and fostering collaboration, we can build healthier open source ecosystems. A credit system can help make open source more sustainable and fair, driving growth, competitiveness, and potentially creating thousands of new open source businesses.

As Drupal continues to improve its credit system, we understand that no solution is perfect. We're eager to learn from the successes and challenges of other open source projects and are open to ideas and collaboration.

The Drop Times: Unlock Advanced Drupal Content Editing: Join Our CKEditor Webinar

Maximize your Drupal content creation with CKEditor's advanced features! Join the upcoming webinar on October 16, 2024, at 11:00 AM EDT to learn about the CKEditor 5 Drupal Plugin Pack, advanced editing tools, and productivity enhancements. Drupal developers, content managers, and site administrators will gain practical insights on improving content workflows. Don't miss this opportunity—register today!

1xINTERNET blog: DrupalCon Barcelona - A seminal moment in Drupal’s history

DrupalCon is the biannual conference where leaders in Drupal gather to coordinate, collaborate and present the latest developments in digital and web technologies. This week 1xINTERNET attended DrupalCon Barcelona in full force. But this conference was like none before it, this was Starshots World Premier and it exceeded all expectations.

Evolving Web: Dries Presents Drupal CMS at DrupalCon Barcelona - Leading the Way in AI-Assisted Site Building

Image removed.

At DrupalCon Barcelona 2024, Dries Buytaert, the creator and project lead of Drupal, delivered his 40th DriesNote, in which he shared an exciting vision of where the platform is heading, with a strong focus on Drupal CMS (the product being developed as part of the Starshot Initiative). As part of the Starshot Initiative’s Leadership Team, I’m the Marketing Lead, working to align the positioning of the product with the needs of our target audiences: primarily marketers, content teams, and site builders, but also evaluators, designers, and developers. 

 

Image removed. The Drupal Starshot Leadership team, featuring Dries Buytaert as the Product Lead, Tim Plunkett as the Technical Lead, Pamela Barone as the Product Owner, Cristina Chumillas as the UX Lead, Lenny Moskalyk as the Delivery Manager, Gabor Hojsy as the Contrib Coordinator and Suzanne Dergacheva (that's me!) as the Marketing Lead. 

 

I’m also excited to contribute along with my colleagues at Evolving Web— who are leading the Analytics track. We’re proud to contribute to Drupal CMS (formerly known as Starshot). With a focus on Drupal CMS, Experience Builder, AI innovations, and a commitment to responsible AI, Dries outlined how the platform is becoming more accessible to non-developers while pushing the innovation and flexibility that Drupal is known for. 

If you missed the DriesNote, I will share some key takeaways below. Check out the recording for the preview demo of Drupal CMS and all the exciting features ahead directly from Dries.

Drupal CMS

Drupal CMS is being designed to make the power of Drupal more accessible than ever, offering preconfigured solutions that allows marketers, web designers, and organizations to easily build and manage their own websites.

It’s important to note that Drupal CMS isn’t some reduced-functionality, beginner-friendly version—it’s built on the robust foundation of Drupal Core. This means that while it is designed to make getting started with Drupal easier for new users, it still offers all the power, flexibility, and scalability that experienced developers rely on. Drupal CMS will open  the door to greater adoption directly by marketers and non-developers, making it simpler to use without sacrificing any of the advanced capabilities that make Drupal such a powerful tool for complex websites.

Preview of Drupal CMS

In the DriesNote demo, Dries demonstrated just how easy it will be to get up and running with Drupal CMS, by showing how a fictional marketer named ‘Sarah’ could easily create a wine tour website with a calendar to highlight upcoming tasting tours.

What Are Recipes?

A standout feature of Drupal CMS is the introduction of Recipes, which are pre-configured functionality, like SEO optimization or event management, that can be easily added to your website. They package content types and other configuration into easily reusable bundles. 

The benefit is that you can add features quickly, use smart defaults that have been carefully selected by experts, and take advantage of best practices even if you’re not a technical user. 

Dries demonstrated how Sarah used an Events Recipe to set up an interactive calendar and SEO-friendly pages for her wine tour website. With just a few clicks, Sarah was able to accomplish in hours what would previously take days.

 

Image removed. The Recipes dashboard shows the status of the Recipes currently underway

AI-Driven Site Building and Migration

One of the most exciting moments of the DriesNote was the unveiling of AI Agents, which automate many complex tasks in Drupal. These AI agents, embedded within Drupal CMS, allow non-technical users to rename content types, create custom fields, and enforce image quality standards—all through simple, plain-language commands.
Dries showed how Sarah leveraged AI to make critical site changes quickly. From migrating content from a non-Drupal website to setting up structured content types, AI simplifies traditionally cumbersome processes. 

Additionally, Drupal’s AI is transparent and flexible. Users can see what language models are used and even swap out the default AI providers with their own models, offering an extra layer of customization and control.

A Commitment to Responsible AI

As part of Drupal’s commitment to the open web, Dries announcedDrupal’s new Responsible AI Policy, which is built around the following principles: 

  • Human in the Loop: AI decisions must be reviewable and reversible by a human.
  • Transparency: Users must know where and how AI is being used on their sites.
  • AI Model Flexibility: Drupal users can select different AI providers based on their needs, ensuring that privacy, ethics, and energy efficiency are prioritized.

This policy has been developed to not only drive the incorporation of AI features into Drupal CMS but to also encourage its adoption by other parts of the Drupal ecosystem including Drupal core and Drupal contrib.

Experience Builder

Another major highlight from the keynote was a preview of Experience Builder, a React-based interface for modifying the UX and UI of a website. It provides content editors and marketers with a no-code builder that gives end-users the ability to add components, select styling options, and apply branding to the website without writing custom code. What really makes it user-friendly is how fast the interface is. Thanks to the fact that it’s React-based, clicks feel instant. This encourages users to experiment and makes the overall experience much more positive. It’s built using Single Directory Components (SDCs), making it familiar to front-end developers. 

Experience Builder will be part of Drupal CMS but is also set to revolutionize the way that all Drupal websites are built. The plan is for it to eventually replace the current Drupal admin UI, providing a more intuitive way to edit all aspects of a Drupal website. During the keynote, Dries presented a demo of Experience Builder, showcasing its initial version, with drag-and-drop interface, as well as an overview of what it will look like in the future.

With 30 full-time developers working on it—making it the largest, fastest-moving initiative ever undertaken in Drupal—Experience Builder will significantly modernize how websites are built and managed in Drupal,  eventually replacing the current Drupal admin interface. 

 

Image removed. A preview of what Experience builder will look like

A Roadmap for Drupal CMS 

A pre-release version of Drupal CMS is available to install so you can try it out and a hosted version is also available so you can try out the AI capabilities for yourself right from your browser. The first release of Drupal CMS will happen on January 15th, 2025—Drupal’s 24th birthday. A release candidate will be launched at DrupalCon Singapore, happening on December 9-11, 2024.  Experience Builder will be included in version 2 of Drupal CMS sometime in 2025.

The “Adopt a Document” Program

In addition to these technical advancements, Dries introduced the Adopt-a-Document program, a new initiative aimed at improving Drupal’s documentation, not only for Drupal CMS but also for Drupal Core. Organizations can sponsor sections of documentation to help ensure that Drupal remains well-supported and accessible for all users through a partnership with Drupalize.me. In addition, the Drupal Association is hiring a full-time documentation lead. 

Get Involved 

The message of Dries’ keynote was clear: Drupal CMS is not just about making web development easier; it’s about making the open web accessible to everyone. With AI-driven tools, intuitive page building, and a commitment to responsible technology, Drupal CMS is set to bring the power of Drupal to a whole new generation of users.

Whether you're a developer, a site builder, or a marketer, this is a great time to get involved and help shape the future of Drupal. Want to join us in pushing these initiatives forward? Get in touch or learn more at drupal.org/about/starshot
 

+ more awesome articles by Evolving Web