The Drupal Business Survey invites business leaders worldwide to share their insights and metrics on the Drupal business ecosystem. Relevant topics to many digital service providers include business development, community, marketing, and human capital. The results of this survey are analyzed and aggregated into a comprehensive report and presented at DrupalCon Europe, the yearly Drupal summit. Participants can support anonymously.

The Drupal Business Survey is about sharing business insights between Drupal service providers worldwide. Drupal’s open source ecosystem thrives by a collaborative community of tens of thousands of professionals worldwide, working together on the popular digital experience platform. Drupal has an Open Web objective, allowing anyone to work with the web or contribute to it. The Drupal Business Survey gives meaningful data for Drupal experts to make business decisions for the coming years.

The survey allows Drupal service providers and independent professionals to share insights on various topics. Its comprehensive results on business outlook and customer engagement has been a valuable guide for digital service providers, even to those working with other technologies but Drupal. Data is coming in from all continents, with most of the companies being in business for 10 years or more.

Drupal business booming

In 2022, over 15% of the respondents indicated seeing a significant impact on their digital business by the pandemic. Covid-19 clearly was a catalyst for investments in the digital market. At the same time, the size of deals made grew by almost a third. Non-Profit and Government made the leading markets for Drupal last year, with Education at the top with over 60% of agencies working with these sectors. However, comparing all the industries over the past 5 years, the number of Drupal projects in Non Profit shows a significant decrease. Acquiring new talent is constantly challenging and business leaders need to find new ways to get the right people in.

The survey revealed that open source software, like Drupal, has maintained its popularity, which will not come as a surprise to those following the industry. Janne Kalliola: “One of the most significant advantages of open source compared to similar SaaS solutions is the flexibility and manageability of pricing. Organizations can easily adapt their use of Drupal to changing circumstances, such as the pandemic or the energy crisis, and they can use Drupal without large license fees.”

CEO Dinner

The Drupal market is looking forward to how digital service providers are using Drupal and how the Drupal market has evolved this year. The results of the 2023 Drupal Business Survey will be presented at the CEO Dinner at DrupalCon Europe. DrupalCon Europe is the yearly Drupal summit where over 1500 Drupal users and professionals meet to exchange ideas and further evolve Drupal. DrupalCon Europe is from 17-20 October in Lille, with the CEO Dinner on Tuesday, October 17th.

About the Business Survey

The Drupal Business Survey supports Drupal businesses worldwide and is organized by a team of industry experts Imre Gmelig Meijling (React Online), Janne Kalliola (Exove) and Michel van Velde (independent) in collaboration with the Drupal Association. The 2022 results can be viewed here.

Drupal is the open source Digital Experience Platform used organisations including Amnesty International, Tesla and DHL.

Participate and share your insights

Drupal experts are invited to share their Drupal business insights through the Business Survey anonymously and come to DrupalCon Europe to review the results together.

You can participate in the Drupal Business Survey anonymously here.
The survey closes on 4 September.

Site speed is crucial, particularly nowadays, when modern websites are more dynamic and interactive. The traditional approach of serving pages is notably inefficient in this context. Numerous techniques exist to achieve optimal performance, and one such method is the BigPipe technique, originally developed at Facebook. The good news is that the BigPipe module, which incorporates the same functionality, has been integrated into Drupal 8 core since version 8.1.

How does Drupal BigPipe work?

The general idea of the BigPipe technique is to decompose web pages into small chunks called pagelets and pipeline them through several execution stages inside web servers and browsers. 

At a high level, BigPipe sends an HTML response in chunks:

1. One chunk: everything until just before - this contains BigPipe placeholders for the personalized parts of the page. Hence this sends the non-personalized parts of the page. Let's call it The Skeleton.
2. N chunks: a tag per BigPipe placeholder in The Skeleton.
3. One chunk:  and everything after it.

This is conceptually identical to Facebook's BigPipe (hence the name).

How does Drupal BigPipe differ from Facebook's technique?

Drupal module differs significantly from Facebook's implementation (and others) in its ability to automatically figure out which parts of the web page can benefit from BigPipe-style delivery. 

Drupal's render system has the concept of “auto-placeholdering.” What does it mean? The content that is too dynamic is replaced with a placeholder that can be rendered later. 

On top of that, it also has the concept of “placeholder strategies.” By default, placeholders are replaced on the server side, and the response is blocked on all of them being replaced. But it's possible to add additional placeholder strategies. BigPipe is just another one. Others could be ESI, AJAX, etc. 

BigPipe implemented by Facebook, can only work if JavaScript is enabled. Instead, the Drupal BigPipe module makes it possible to replace placeholders without JavaScript “no-JS BigPipe.” This isn’t technically BigPipe at all, but it's just the use of multiple flushes. 

This allows us to use both no-JS BigPipe and “classic” BigPipe in the same response to maximize the amount of content we can send as early as possible.

So basically, that is happening during the page rendering process:

• The personalized parts are turned into placeholders &token=6NHeAQvXLYdzXuoWp2TRCvedTO2WAoVKnpW-5_pV9gk">. The placeholder contains information about which callback to call and the arguments to pass to it. The renderer then continues traversing the array and converting it to HTML. The resulting HTML, including placeholders, is cached. Then, depending on the rendering strategy being used, the placeholders are each replaced with their dynamic content.
• The replacement of the placeholders is done using JavaScript. The callback starts looking for replacement script elements once a special element is printed and found.
• At the very last moment, it’s replaced with the actual content. This new strategy allows us to flush the initial web page first and then stream the replacements for the placeholders.

When to use a lazy builder?

As a rule of thumb, you should consider using a lazy builder whenever the content you're adding to a render array is one of the following types.

• Content that would have a high cardinality if cached. For example, a block that displays the user's name. It can be cached, but because it varies by user, it's also likely to result in cached objects with a low hit rate.
• Content that cannot be cached or has a very high invalidation rate. For example, displaying the current date/time, or statistics that must always be as up-to-date as possible.
• Content that requires a lengthy and potentially slow assembly process. For example, a block displaying content from a third-party API where requesting content from the API incurs overhead.

Using lazy builders in practice

To provide a comprehensive example of implementing lazy builders, let's consider a scenario with a custom profile widget block placed on a web page. The block contains the user's picture, full name, and profile menu.

In order to ensure that every user sees their personalized information, we can implement specific strategies, such as setting the “max-age” to 0 or utilizing user contexts and tags. However, it's important to note that setting “max-age” to 0 will lead to the rest of the web page being uncached.

Thanks to the concept of “auto-placeholdering,” Drupal considers this block as a personalized part and turns it into a placeholder.

The only problem here is that we have the entire block replaced by a placeholder afterward:

=profilewidget&args%5B1%5D=full&args%5B2%5D&token=QzMTPnxwihEGO
itjJB_tahJj8V-L-KopAVnEjVEMSsk">

Nonetheless, it's worth noting that specific data within the block might remain static or consistent for all users, like this:

To make our block more granular, we can transform dynamic parts into placeholders while the other block content remains cacheable and loads during the initial page load.

Step 1. Creating lazy builders

Lazy builders are implemented using the render array of the #lazy_builder type, just like other elements. The render array must contain a callback as the first element and an array of arguments to that callback as the second element.

Lazy builder render elements should only contain the #cache, #weight, and #create_placeholder properties. 

public function build() {  $build['user_data'] = [    '#lazy_builder' => [      'd_profiles.builder:build',      [],    ],    '#create_placeholder' => TRUE,  ];  $build['user_menu'] = $this->buildMenu()  $build['user_img'] = [    '#lazy_builder' => [      'd_profiles.builder:build',      ['profile_widget_mini'],    ],    '#create_placeholder' => TRUE,  ];  return $build; }

Step 2. Implementing TrustedCallbackInterface

Before we go any further, we need to ensure that our lazy builder implementation can call the lazyBuilder() method. To do this, we need to implement the TrustedCallbackInterface to tell Drupal that our lazy builder callback is allowed to be called.

When implementing this interface, we need to add a method called trustedCallbacks(), which will be called automatically by Drupal through the detection of the interface. The return value of this method must be any methods within this class that can be used as callbacks.

Here is the basic implementation of this for our block:

/** * Provides a lazy builder for the profile block. */ class ProfileBuilder implements TrustedCallbackInterface { /** * {@inheritdoc} */ public static function trustedCallbacks() { return ['build']; } /** * Build profile details. */ public function build($view_mode = 'navbar_widget') { return $this->entityBuilder->build($this->loadProfile(), $view_mode); } }

As a result, the cached block will look like:

Profile

• Settings
• Logout

Normally the lazy builder callback will be executed on every page load, which is the intended behavior. But in certain cases, it may also be necessary to cache placeholders. To achieve this, we need to include cache keys along with the cache context, as in the example below:

$build['user_data'] = [   '#lazy_builder' => [     'em_profiles.builder:build',     [],    ],   '#create_placeholder' => TRUE,   '#cache' => [     'contexts' => ['user'],     'keys' => [       'entity_view',       'user',      'profile',      'navbar_widget',     ],   ], ];

Step 3. Ensuring a smooth visual page load experience

Because Drupal BigPipe lazily loads certain parts of the page, it could result in a jarring page load experience. It depends on our theme and the location of the lazily loaded content.

The simplest solution is to have the lazily loaded content appear in a space reserved for them that avoids reflowing content. Alternatively, we can apply a “loading” animation to all BigPipe placeholders in our theme with some CSS.

The last possible option is to define an interface preview that will be populated by BigPipe, using a Twig template.

Let’s compare the final result of the custom lazy-builder strategy (1) vs. “auto-placeholdering” strategy (2).

Custom lazy-builder strategy (1)
 

Drupal "auto-placeholdering" strategy (2)

Both strategies work just fine, but you can see the cons of auto-placeholdering, like jarring page load experience (drastic layout shift).

More examples:

1. Statistics block with static part loaded immediately and dynamic content loaded later:

2. Views with skeleton:

Troubleshooting lazy builders

If you’ve implemented a lazy builder and it isn't speeding up your Drupal page load or just isn't working as expected, then there are some things you can try:

• Be sure the Drupal BigPipe module is active. 
• Check the cache settings of your lazy builder callback method. By default, Drupal will make some assumptions about how it should be cached, which isn't always right for your use case. Instead, you can explicitly set the cache settings.
• An upstream CDN or Varnish layer might be caching your entire web page, so all of the output of the BigPipe rendering process will be served simultaneously. You'll need to find another mechanism to work around this.

Drupal BigPipe - summary 

In this article, we’ve explored an alternative rendering strategy that allows us to defer the rendering of highly dynamic content only after the static parts of the web page have already been loaded from the cache.

BigPipe, the Drupal module, can automatically enhance our website performance thanks to improved render pipeline and render API, particularly the cacheability metadata and auto-placeholdering.

However, it’s essential to note that using Drupal BigPipe doesn’t substitute for addressing underlying performance issues. Implementing lazy builders to mitigate the impact of slow code on your web page will only mask the problem rather than resolve it entirely. By implementing these techniques effectively, you can optimize performance and enrich the overall user experience on your Drupal-powered websites.

We're excited to share our experience implementing automated testing into our legacy site, where it has helped improve functionality and user experience through automated testing.

The digital world is a swift river, and staying current is of utmost importance. The advent of Drupal 10 marks a substantial leap towards improved performance, security, and user experience. As a pioneer in 360 Drupal services, LN Webworks is proud to offer seamless Drupal upgrade Services. 

 Let’s navigate the intricacies of Drupal 10's core attributes, the advantages of embracing this upgrade, and how LN Webworks stands as your steadfast partner for a successful migration.

Revisit the past with a 16-year-old ancient Drupal forum discussion. As we rediscover these hidden gems from the past, it's a reminder that the digital landscape is ever-changing. What was innovative in 2007 may not be cutting-edge in 2023, but the principles of adaptability and innovation remain constant.

Discover the Twin Cities DrupalCamp 2023, a two-day event showcasing an array of engaging sessions and peer discussions. This article delves into the highlights of the camp, its featured speakers, and the diverse spectrum of sponsors that contribute to its success.

August is traditionally a month when many in the northern hemisphere go on a much needed holiday from the heat and enjoy a well-deserved break from work.

However, this will not stop our Pitch-burgh contest winners, who were incredibly excited about starting the work and demonstrating how an open source competition like this can deliver cost-effective benefits for the Drupal project.

For the boring part, most or all of the agreements are in place (with some still being signed this second week of August), and the initial grants are starting to arrive in the awardee’s bank accounts. As previously mentioned, this is a first for the Drupal Association, and it has taken some time to figure out important things regarding contracts and how money is moved. In the end, we want to make sure none of us get in trouble with the tax collector.

In the meantime, for those keeping tabs on our Slack channels, you're likely aware of the substantial progress that's been made across the projects. It's remarkable that none of them were compelled to take action until the ink on the contracts had dried. This is the beauty of open source – a commitment driven by shared belief and trust among all contributors that transcends mere agreements and financial transactions. It's a challenge that proprietary software can hardly match.

I reached out to each project, asking for an update on their initial step. Here's what they had to say:

Mentor the mentor

The first iteration of the DrupalCon First Time Contributor workshop has been templated and made more accessible for folks who can't access the slides visually. The slide deck for local and regional camps is also primarily complete; awaiting feedback from other mentors. The next phase involves scripting the decks and incorporating documentation into Drupal.org's Contributor Guide.

Unsurprisingly, the issue queue etiquette deck is still in flux and changing as some contributors find new and unique ways to game the contribution system. AmyJune has contacted core and contrib maintainers, seeking insights into their most pressing pain points within the issue queue. The issue queue etiquette documentation is being shared as needed, but some of the community feels some points are antiquated or can be reworded into more productive dialog. The presentation depends on that etiquette. As most of the slides are complete, she will hold off on saying it's finished until the issue queue etiquette documentation discussion is in a healthy place.

Stay updated on the progress of this project by following the discussion in the #pitchburgh-mentoring-the-mentor channel in Drupal Slack.

Policy based access

Access policies in core now have a green patch for just the new API and a green patch for the API + integration in core. All I need to do now is add documentation and keep following up on the patch to get it into core.

To stay informed about the most recent developments, please keep an eye on the #pitchburgh-policy-based-access channel.

Drupal client

The Drupal API Client is currently in the process of finalizing our project planning phase and gearing up for implementation. Our intent is to begin by focusing on a limited-scope POC that will allow us to test our assumptions early and gather feedback. As part of this effort, we are in the final stages of addressing a related meta issue and eagerly welcome any insights you may have in the comments section.

Recently, we also conducted an initial async kickoff meeting in the #api-client channel in Drupal Slack. We're planning on establishing a bi-weekly Slack meeting on Thursdays at 15:00 UTC, and actively working on scheduling a one-off Zoom kickoff. Join #api-client for more info on these meetings, or just to say 'hi' and catch up on the backscroll.

The Slack channel is bustling with activity, encompassing daily discussions and regular catchup meetings. We invite you to participate and connect with our project leader, Brian Perry, in the #api-client channel within the Drupal Slack community.

Decoupled Layout Builder 

Discovery is complete, we’re all set up on Drupal.org with meta issues and sub tasks for the project. We’re planning to start development in September.

To stay informed about the latest developments, please track updates in the #pitchburgh-decoupled-layout-builder channel.

Gutenberg

The Gutenberg project is starting in September, with an ideal kickoff that includes a workshop in collaboration with Automattic, a company that, if you remember, its founder, Matt Mullenweg generously offered to sponsor this endeavor and proposed a workshop between our two teams to aid in the discovery and initial phases.

If you'd like to be part of the conversation or contribute your insights, please don't hesitate to join us in the #pitchburgh-gutenberg channel.

JSON Schemas and data storage in Core

Development on this initiative starts in earnest this month. The project schedule has a completion date of 31 December, which is ambitious but consistent with the strong community interest in these priorities. Even before contracting was completed, important discussions occurred in policy issues regarding minimum database driver versions with an eye toward ensuring these improvements can land in the D11 release cycle - and perhaps even in D10 minor releases. Development work has begun on refining existing merge requests for the JSON data type in the database schema API, and follow-on issues are being scaffolded per the project roadmap.

If you're keen to contribute to this significant effort, we encourage you to participate in the discussion within the #pitchburg-json-ftw channel or the #contribute channel on Drupal Slack. - Brad Jones

What is the Cyber Resilience Act? The Cyber Resilience Act is proposed legislation that will make it mandatory for hardware and software producers to: Ensure that their products meet minimum security standards before they’re released  Keep their products’ security up-to-date after release (e.g. monitoring security threats and releasing security updates) The Act also aims to make it easy for users to compare the security features of different products. The end goal: to make digital products more secure, to protect citizens and businesses.  View the draft Act in multiple languages What effects will it have on software development? A factsheet about the Act outlines the manufacturer’s obligations.

Let me tell you that for peasants like me

Some years ago, after a DrupalCon in the US, I received a direct message. It was kind of shocking and has been on my mind on and off ever since...

Rachel Tue, 08/22/2023 - 18:53

Tags

• Drupal
• OpenSource

• Read more about Let me tell you that for peasants like me
• Log in with GitHub, LinkedIn, or Google to post comments

Rate Limiting an aggressive bot in Nginx John Locke Tue, 08/22/2023 - 10:58

High load isn't necessarily an emergency, but it may be a heads-up before a site noticeably slows down. Sometimes there are weird spikes that just go away, but sometimes this is an indication of a Denial of Service.

Rate Limiting

NGinx has rate limiting that can be used to handle cases where a slow URL is targeted. Today one of our sites had high load alerts. Here's how I handled it:

• Read more about Rate Limiting an aggressive bot in Nginx
• Add new comment