We're thrilled to share that the Sovereign Tech Fund (STF), based in Germany, has generously entrusted the Drupal Association with a $300,000 USD service contract for work done to benefit the public. This funding is set to fuel two crucial projects that promise to strengthen security for Drupal and enhance the Drupal ecosystem.
The Sovereign Tech Fund (STF) supports the development, improvement, and maintenance of open digital infrastructure in the public interest. Its goal to strengthen the open source ecosystem sustainably, focusing on security, resilience, technological diversity, and the people behind the code. STF is funded by the German Federal Ministry for Economic Affairs and Climate Action (BMWK) and hosted at and supported by the German Federal Ag-ency for Disruptive Innovation GmbH (SPRIND).
The Drupal Association, along with the Drupal community, support Drupal with core support, community support, flagship programs, and new innovation. The Drupal Association is a unicorn in the software sector in terms of structure and true community - and is a leader for open source collaboration and an open web.
This project will optimize GitLab CI, streamline user authentication with Keycloak, migrate Drupal contribution credits from the old issue queue to a new GitLab integration, create a seamless opt-in process for Drupal.org hosted projects to transition to GitLab issues, and develop an accessible learning guide. The guide will be a valuable resource for project maintainers looking to shift from Drupal.org's custom tooling to GitLab.
This initiative aims to enhance the security of the Drupal ecosystem by securing the signing prototype, conducting a third-party security audit of the PHP-TUF client and Rugged server, and performing a third-party security audit of the Drupal integration code. Additionally, the project will deploy secure signing in a production environment, further bolstering the security measures in place.
This funding aligns perfectly with the Drupal Association's strategic priorities. It enables us to make significant strides towards our goals, particularly in terms of optimizing our workflows through GitLab and enhancing our security measures with secure signing. Both projects will conclude before 31 March 2024.
The partnership with STF allows us to make a positive difference in the Drupal community and advance the open source platform for all users. We are grateful to the Sovereign Tech Fund for their generous support. Their funding shows dedication to open source and their belief in the Drupal Association and the community's ability to innovate and ensure the future of web development.
Hey nonprofit Drupal users! The DA is interested in supporting community-driven content that is specifically relevant to nonprofit organization staff and related agencies at DrupalCon North America in Portland, Oregon, at the Nonprofit Summit on May 9, 2024.
We are looking for volunteers who would be interested in giving back to the community by contributing some subject matter expertise via a day of informal breakout sessions or other group activities. We are open to ideas!
Do you have some Drupal expertise or a recent experience with a Drupal project that you would like to share with others? Is there something about Drupal that you think is really cool that you would love to share with the nonprofit Drupal community?
You will not be required to make slides! You don’t need to have lots of (or any) speaking experience! All you need is a willingness to facilitate a discussion group or engaging activity around a particular topic, and some expertise or enthusiasm for that topic that you wish to share.
Please fill out this form by February 13th and we will get back to you as soon as we are able. Thank you! https://forms.gle/MJthh68rsFeZsuVc8
Discussion leaders will be selected by the Nonprofit Summit Planning Committee and will be notified by the end of February.
In Drupal 8 and later versions, the routing system allows developers to define routes for various pages in their applications. These routes can include placeholder elements in the path, which are essentially variables or dynamic values in the URL. These placeholders are enclosed in curly braces, such as {node} in the example I have provided.
These are parts of the URL that are variable or dynamic.In the example, /node/{node}, {node} is a placeholder indicating that this part of the URL can vary. The placeholders are named to make them identifiable. In the example, {node} is named to indicate that it represents a node ID.
Upcasting refers to the process of converting a placeholder value from the URL into an actual object instance.In the example, the system wants to convert the {node} placeholder value into an actual node object.
The Gutenberg Editor is a powerful page building tool for Drupal that utilizes a block building system to create and edit content.
This comprehensive guide walks you through each step of setting up and using the Gutenberg Editor.
From downloading and installing the Gutenberg module, to enabling it on your content types, and finally using it to create your content using blocks, this guide has you covered.
To get the most out of this guide, follow the video above.
The greatest advantage of Symfony Messenger is arguably the ability to send and process messages in a different thread almost immediately. This post covers the worker that powers this functionality.
by daniel.phin / 11 January 2024This post is part 3 in a series about Symfony Messenger.
The Symfony Messenger integration, including the worker, is provided by the SM project. The worker is tasked with listening for messages ready to be dispatched from an asynchronous transport, such as the Doctrine database transport. The worker then re-dispatches the message onto the bus.
Some messages may be added to a bus with no particular execution time, in which case they are serialised by the original thread. Then unserialised almost immediately by the consume command in a different thread.
Since Messenger has the concept of delaying messages until a particular date, the DelayStamp can be utilised. The consume command respects this stamp and will not redispatch a message until the time is right.
The worker is found in the sm console application, rather than Drush. When SM is installed, Composer makes the application available in your bin directory. Typically at /vendor/bin/sm
The command takes one or more transports as the argument. For example if you’re using the Doctrine transport, the command would be:
sm messenger:consume doctrine
Multiple instances of the worker may be run simultaneously to improve throughput.
The worker. Messages output to stdout for demonstration purposes.The worker allows you to prioritise the processing of messages by which transport a message was dispatched to. Transport prioritisation is achieved by adding a space separated list of transports as the command argument.
For example, given transports defined in a site-level services.yml file:
parameters:
sm.transports:
doctrine:
dsn: 'doctrine://default?table_name=messenger_messages'
highpriority:
dsn: 'doctrine://default?table_name=messenger_messages_high'
lowpriority:
dsn: 'doctrine://default?table_name=messenger_messages_low'
In this case, the command would be sm messenger:consume highpriority doctrine lowpriority
Routing from messages to transports must also be configured appropriately. For example, you may decide Email messages are the highest priority. \Symfony\Component\Mailer\Messenger\SendEmailMessage would be mapped to highpriority:
parameters:
sm.routing:
Symfony\Component\Mailer\Messenger\SendEmailMessage: highpriority
Drupal\my_module\LessImportantMessage: lowpriority
'*': doctrine
More information on routing can be found in the previous post.
The transport a message is sent to may also be overridden on an individual message basis by utilising the Symfony\Component\Messenger\Stamp\TransportNamesStamp stamp. Though for simplicity I’d recommend sticking to standard routing.
The sm worker listens and processes messages, and is designed to run forever. A variety of built in flags are included, with the ability to quit when a memory or time limit is reached, or when a certain number of messages are processed or fail. Flags can be combined to process available messages and quit, much like drush queue:run.
Further information on how to use the worker in production can be found in the Consuming Messages (Running the Worker) documentation.
The next post covers Cron and Scheduled messages, a viable replacement to hook_cron.
This post covers Symfony Messenger’s message and message handlers, which are the day to day code developers using features of Symfony Messenger typically will be working on.
by daniel.phin / 10 January 2024This post is part 2 in a series about Symfony Messenger.
The Symfony Messenger integration with Drupal provided by the SM project is the only requirement for the following examples.
A message itself is very flexible, as it doesn't require annotations, attributes, or specific class namespace. It only needs to be a class serialisable by Symfony. For simplicity, don’t include any complex objects like Drupal entities. Opt to store entity UUIDs instead.
At its most simple implementation, a message handler is:
Messenger\ namespace#[AsMessageHandler] class attribute__invoke method. Where its first argument is an argument typehinted with the message class.Example message and message handler:
namespace Drupal\my_module;
final class MyMessage {
public function __construct(public string $foo) {}
}
namespace Drupal\my_module\Messenger;
use Drupal\Core\State\StateInterface;
use Symfony\Component\Messenger\Attribute\AsMessageHandler;
#[AsMessageHandler]
final class MyMessageHandler {
public function __construct(StateInterface $state) {}
public function __invoke(\Drupal\my_module\MyMessage $message): void {
// Do something with $message.
$this->state->set('storage', $message->foo);
}
}And dispatch code:
$bus = \Drupal::service(\Symfony\Component\Messenger\MessageBusInterface::class);
$bus->dispatch(new MyMessage(foo: 'bar'));Non-autowirable dependency injection
Message handlers use autowiring by default, so you don’t need ContainerFactoryPluginInterface and friends.
In the rare case that dependencies are not autowirable, you can opt to define a message handler as a tagged service instead of a class with #[AsMessageHandler] attribute and define dependencies explicitly. The same __invoke and argument typehinting semantics apply.
services:
my_module.my_message_handler:
class: Drupal\my_module\Messenger\MyMessageHandler
arguments:
- '@my_module.myservice'
tags:
- { name: messenger.message_handler }
Typically, when setting up a Drupal queue, you’ll be putting together a rigid class with a verbose annotation. When compared to the functionality of the messenger and handler above, the equivalent @QueueWorker looks like:
namespace Drupal\my_module\Plugin\QueueWorker;
use Drupal\Core\Plugin\ContainerFactoryPluginInterface;
use Drupal\Core\Queue\QueueWorkerBase;
use Drupal\Core\State\StateInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;
/**
* @QueueWorker(
* id = "my_module_queue",
* title = @Translation("My Module Queue"),
* cron = {"time" = 60}
* )
*/
final class MyModuleQueue extends QueueWorkerBase implements ContainerFactoryPluginInterface {
private function __construct(
array $configuration,
$plugin_id,
$plugin_definition,
private StateInterface $state,
) {
parent::__construct($configuration, $plugin_id, $plugin_definition);
}
public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition): static {
return new static(
$configuration,
$plugin_id,
$plugin_definition,
$container->get('state'),
);
}
public function processItem(mixed $data): void {
// Do something with $data.
$this->state->set('storage', $data['foo']);
}
}And dispatch code
\Drupal::service('queue')
->get('my_module_queue')
->createItem(['foo' => 'bar']);Notice the hard-to-remember annotation, boilerplate dependency injection, and mixed-type processItem argument $data . In comparison, Symfony Messenger messages and message handlers are easier to use thanks to PHP attributes.
All messages will be handled synchronously by default. To route messages to specific transports, routing needs to be configured.
Behind the scenes, routing is a simple map of class/namespaces to transports defined in a container parameter.
parameters:
sm.routing:
Drupal\my_module\MyMessage: doctrine
Drupal\my_module\MyMessage2: synchronous
'Drupal\my_module\*': doctrine
'*': doctrineKeys are either verbatim class names, partial class namespace followed by asterisk, or a standalone asterisk indicating the fallback. The values are the machine name of a transport. SM includes a synchronous transport out of the box, which indicates messages are handled in the same thread as it is dispatched. The doctrine database transport is available as a separate module. I’d recommend always using an asynchronous transport like Doctrine.
SM includes a configuration UI submodule that allows site builders to build a routing map without needing to mess with YAML. The container parameter is set automatically as soon as the form is saved.
A common use case for adding stamps to a message is to delay the message for an amount of time. A stamp is created and attached to the envelope containing the message to be processed:
$envelope = new Envelope(
message: new MyMessage(foo: 'bar'),
stamps: [\Symfony\Component\Messenger\Stamp\DelayStamp::delayUntil(new \DateTimeImmutable('tomorrow'))],
);
$bus = \Drupal::service(\Symfony\Component\Messenger\MessageBusInterface::class);
$bus->dispatch($envelope);For more advanced use cases, multiple handlers can be configured for a message. Useful if you want to listen for messages that you do not own. For example, additional handling of the Symfony Mailer email message:
namespace Drupal\my_module\Messenger;
use Symfony\Component\Messenger\Attribute\AsMessageHandler;
use Drupal\Core\State\StateInterface;
#[AsMessageHandler]
final class MyMessageHandler {
public function __construct(StateInterface $state) {}
public function __invoke(\Symfony\Component\Mailer\Messenger\SendEmailMessage $message): void {
$this->state->set(
'sent_emails_counter',
$this->state->get('sent_emails_counter', 0) + 1,
);
}
}Both this custom handler and the original \Symfony\Component\Mailer\Messenger\MessageHandler::__invoke handler will be invoked.
Handlers can be configured to handle multiple message types. Instead of using the #[AsMessageHandler] attribute on the class, use it with methods.
namespace Drupal\my_module\Messenger;
use Drupal\Core\State\StateInterface;
use Symfony\Component\Messenger\Attribute\AsMessageHandler;
final class MyMessageHandler {
#[AsMessageHandler]
public function myHandler1(\Drupal\my_module\MyMessage $message): void {
// Do something with $message.
}
#[AsMessageHandler]
public function myHandler2(\Drupal\my_module\MyMessage2 $message2): void {
// Do something with $message2.
}
}The next post covers the worker, the heart of messenger’s real-time capabilities.
Part one in a series of posts introducing Symfony Messenger, its ecosystem, and unique Drupal integrations to Drupal developers.
by daniel.phin / 9 January 2024The SM project brings Symfony Messenger and its ecosystem together with Drupal.
Symfony Messenger is a powerful alternative to Drupal’s @QueueWorker, enabling real-time or precise execution of scheduled tasks. It’s also a viable replacement for hook_cron enabling you to schedule dispatch and processing of messages according to crontab rules, again in real-time, rather than waiting for server-invoked or request-termination cron.
Messenger can be used as a user-friendly alternative to Batch API, as the user's browser is not blocked, while integrations such as Toasty can be programmed to notify when the last of a “batch” of messages completes, communicating to the user via user-interface toasts.
The Drupal integration includes additional niceties, such as intercepting legacy QueueWorkers for processing data through the Symfony Messenger bus, and end-user UI notifying a user when tasks relevant to them have been processed.
During this and the following series of posts, we’ll be exploring the benefits of real-time processing and user-friendly features that improve the overall experience and outputs.
The workerToasty displaying notifications.First up, we’ll cover the main features of Symfony Messenger and how it works.
As a developer working with Messenger, the most frequent task is to construct message and associated message handlers. A message holds data, while a message handler processes the associated data.
A message is inserted into the bus. The bus executes a series of middleware in order, each of which can view and modify the message.
If a transport is configured, the message may be captured and stored for processing later.
Typically the bus, middleware, and transports are configured in advance and rarely changed. Message and message handlers are introduced often without needing other configuration.
The stars of the show are buses. One bus is ready out of the box, which comprises a series of ordered middleware. A message is dispatched into a bus, where each middleware has the opportunity to view and modify the message (and its envelope). It's unlikely you’ll need to think about middleware, as the default set may already be the perfect combination.
When a message is dispatched to a bus, you can choose to wrap it in an envelope and apply stamps like the DelayStamp . A message will always be wrapped in an envelope if you don’t do it explicitly.
Buses have a series of default middleware. The main middleware to note are the transport and message handler middlewares. When a transport is configured for messenger, the transport middleware will capture the message, serialise it, and store it somewhere. For example, a database in the case of the doctrine transport. Any middleware after the transport middleware are not executed, for now.
When running the worker, you are opting to choose which bus and transport to run. The command will listen for messages as they are stored, and if the time is right, messages will be unserialised and inserted into the bus. The message will begin its journey yet again, iterating through all the middlewares from the beginning. When the transport middleware is hit, it will detect the message has already been in the transport to prevent recursion. This is done by checking the ReceivedStamp stamp added to the message envelope.
Out of the box, when a message is dispatched into the bus in a CLI or web request, it will be processed synchronously. All middleware will operate on the message in a set order, including the message handler middleware.
The greatest advantage of using Messenger is the ability to asynchronously handle messages outside of the thread they were originally dispatched. That is: asynchronously. This can be useful for improving the web request response times and reducing the memory usage and limit of web requests (allowing for more FPM threads on a machine). Bulky business operations that would typically, or should be, constrained by the limits of the web thread have more breathing room. A CLI runner/container may be set up with a little more memory and processing capability with the explicit direction to listen for messages and handle them in real-time, either as soon as possible or as scheduled.
Upcoming posts in this series will dive into aspects of Symfony Messenger and SM:
The next post covers the implementation of a message and message handler, and a comparison with Drupal core’s @QueueWorker plugins.