Talking Drupal: Talking Drupal #373 - Performance, Privacy, and the Open Web
Today we are talking about Performance, Privacy, and the Open Web with Andrey Lipattsev & Tim Lehnen.
For show notes visit: www.talkingDrupal.com/373
Topics- What is the Open Web and what does it have to do with performance and privacy
- What do we mean by web performance
- What are the core vitals
- What are the non core vitals
- What is the benefit of performance
- Performance is usually granular for a developer, what is different when looking at industry trends from a browser side
- What does d.o do
- DA and Google at DrupalCon
- What is privacy
- What is the privacy sandbox in chrome
- Web Vitals
- MOTW discussion
- Performance
- https://web.dev/learn-core-web-vitals/
- https://pagespeed.web.dev/
- https://web.dev/vitals-business-impact/
- https://make.wordpress.org/performance/2022/07/26/welcome-to-the-performance-team-blog/
- Example: https://web.dev/measure/?url=https%3A%2F%2Fdrupal.org
- https://github.com/GoogleChrome/lighthouse-stack-packs/blob/master/packs/drupal.js
- https://github.com/tag1consulting/google-drupal/projects/1
- Privacy
- cls - cumulative layout shift
- IMDB pagespeed
- Core vital affect on search ranking blog post
- Direct business impact
- Collaboration on lazy load
- Core vital tech report
- Talks on performance for chrome
- webp
- Webprofiler
Andrey Lipattsev - www.linkedin.com/in/andreylipattsev @AndreyLipattsev Tim Lehnen - @hestenet
HostsNic Laflin - www.nLighteneddevelopment.com @nicxvan John Picozzi - www.epam.com @johnpicozzi Randy Oest - randyoest.com @amazingrando
MOTW CorrespondentMartin Anderson-Clutz - @mandclu Entity Autocomplete Anchor Nominated by matthieu, who has been a guest on the show and is himself a maintainer of a number of useful modules Provides a widget that allows users to add anchors to entity reference links
Five Jars: How to Control the Scope of the Project: Scope Creep Prevention Document
Sooper Drupal Themes: Drupal 10: Top 10 New Features and Improvements
Drupal 10 release is December 14 2022
32 Days07 Hours31 Minutes34 SecondsDrupal 10 countdown1. A new text editing experience with CKEditor 5
With Drupal 10 we're upgrading from CKEditor 4 to CKEditor 5. While the two versions are mostly very similar, version 5 looks more modern and refined. The interface is simplified, and certainly in UX design simplicity is the ultimate sophistication. Version 5 does not rely on right-click menus, making it more touch-device friendly than version 4.
When editing links and images in CKEditor 5 you have fewer options than in version 4. CKEditor 5 is clearly more specialized for the content editors' audience and not for the site-builders audience. If you ask us this is the way forward.
2. A new look for Drupal 10: Meet Olivero theme
The Olivero theme takes on the important job of representing Drupal to new users and giving Drupal a face, and a brand. The theme looks more modern than its predecessor.
As Drupal core sets the standard for contrib it's great to see Olivero has excellent accessibility features. Unlike earlier Drupal core themes it has a clever dropdown menu system that ensures menu items don't disappear beyond the edges of the browser. Check out this Smashing Magazine article to learn more about these features.
The only thing we don't fancy is the RSS link that shows in the "Social bar" by default. Who still uses an RSS reader?
3. A modern admin theme: Claro
Perhaps more importantly, Drupal 10 also comes standard with another new theme that you'll actually be using. Whereas the Olivero theme is merely a placeholder in most projects, the new admin theme is where you'll be spending a lot of time at.
Verdict: Claro is a big upgrade both in user experience and in making Drupal look like a modern product. The "Seven" theme Claro replaces looks dated in 2022 and the new admin theme is a welcome change. Just look at the side-by-side comparison above.
Want to know a little secret? The team that designed claro maintains a sub-theme of Claro called "Gin". The Gin theme features cutting-edge modern features that are too hot for Drupal core. Check out the Gin project and see for yourself.
4. Find modules from inside your website with Project Browser
Another initiative where Drupal is looking out for small-business users is the project browser initiative. The project browser adds an interface to your Drupal website for discovering and installing contributed modules and themes. WordPress has had this functionality for a long time now but Drupal is trying to do a better job by checking for dependencies and incompatibilities automatically.
This would be a great win for Drupal since a major frustration for WordPress developers is the clashing and crashing of WordPress plugins.
5. Drupal 10 is easier out of the box
The chart above is based on a UX research project that showed Drupal has the lowest approval rating among new users but the highest rating among expert users. This prompted Drupal's founder Dries Buytaert to come up with an initiative to address the "beginners" audience.
The initiative bundles the three modules - Media, Layout Builder, Claro - that together, present a more user-friendly software to beginner users. These modules already existed in Drupal 9, but in Drupal 10 they are considered stable enough to be turned on by default.
6. Automatic Updates
Automatic updates have always been a controversial topic for Drupal. Big enterprise sites don't want it, small users don't want to endlessly spend time and money on weekly updates for Drupal core and the many additional modules that get added to a typical Drupal site.
This initiative is not going to be ready in time for the Drupal 10.0.0 release. This is probably the most complex Drupal 10 improvement both in terms of technology and in getting alignment in the Drupal community. Development of these features takes place in a contrib module that you can try today! Unfortunately, it's currently of limited use because it merely promises to update the Drupal core without breaking your site. Updating other modules and themes is still considered experimental.
Given the fact that many websites simply never install (security) updates, having automatic unattended updates could be a big security advantage for small-business Drupal websites.
7. Easier migration from Drupal 9 to Drupal 10 with Drupal Rector
Major Drupal version updates used to be something to be feared. This is no longer the case. Part of the reason is that after the update from Drupal 7 to 8, the Drupal community decided to make updates easier in the future. The architecture in Drupal 8+ is sophisticated enough to make this possible.
The other part is Drupal rector. Drupal Rector was already available when Drupal 9 came out but the reason we include it here is that the tool has matured. Drupal rector is much more effective now and will automatically generate most of the changes needed to migrate your custom modules to Drupal 10. This will save precious developer hours and makes this major update smoother than ever.
8. Better front-end performance by removing jQuery
Finally, Drupal has done away with jQuery. For a long time, jQuery was a front-end developer's best friend. jQuery made Javascript less intimidating and let us do more heavy lifting with fewer lines of code.
Since jQuery's release in 2006, Javascript has improved tremendously, as one would expect after 16 years. jQuery is no longer needed because most of its features are incorporated into Javascript itself. Removing jQuery means that much of the heavy lifting it was doing will be done more efficiently by native functions.
The new starterkit theme we are working on is not going to serve as a base theme to be subthemed, but rather a theme to be copied on a new theme's creation.Lauri Eskola, starterkit initiative lead
9. New workflow for theming Drupal 10
This is a big improvement for Drupal front-end developers, especially for developers who are new to Drupal. The first thing you want to do with a new CMS is to customize it to match your design and branding requirements. Yours truly has about 2 decades of experience customizing the front-end of CMS software and remembers when theming was a truly painful experience.
The change with Drupal 10 is that rather than sub-theming the Classy theme in Drupal 10 you are copying the starterkit theme and customizing it. That advantage is that you don't rely on a stuffy old base theme anymore that's frozen in Drupal core. I think it's also great for themers because they don't have to search the classy theme and copy files into their own theme, instead all the files are in their custom theme folder, in one convenient place.
Symfony 6 is just Symfony 5, without all the code that has been deprecated for two years. There is no new feature, no main change specific to version 6 because everything is already in Symfony 5.Nicolas Grekas, Symfony core team
10. Better backend performance with PHP 8.2 and Symfony 6
Considering that the upgrade of the Symfony PHP platform is the reason we need to increment the major version number of Drupal 9 to Drupal 10 you'd expect some new features. Alas, Symfony 6 is just a spring cleanup.
Symfony 6 tosses out a lot of old deprecated code. This means we have to update our modules and themes that use dated Symfony APIs. Once we're all done with our spring cleaning our sites are taking advantage of the latest PHP and Symfony improvements and can be expected to perform better, providing a snappier experience to our end users.
Drupal Association blog: One Month until Drupal 10 is expected to be released
We’re getting even closer to the launch of Drupal 10 as we hit the one month countdown mark! With Drupal 10 expected to be released on 14 December, we’re gearing up to update in order to take advantage of all of the benefits such as Olivero theme, CK Editor 5, and new Claro backend.
Prepare now with Upgrade Status
-
If you’re on Drupal 9, use Upgrade Status on your Drupal 9 site to get an estimate of the work needed. Install the Upgrade Status module, check your environment readiness and start by making that compatible. Then, proceed with the to-do list shown on the UI.
Fix Code with Drupal Rector
-
Automatic code fixes are available in Drupal Rector. The plan provided by Upgrade Status will tell you where to use drupal-rector to update your own custom code. Upgrade Status will also help you find the best way to contribute to drupal.org projects that are not yet compatible!
Hosting a Drupal 10 party? Share it with the community!
-
If you’re hosting a Drupal 10 celebration or party of your own, add it to the community events page on Drupal.org under Local Meetups for more visibility! Then, tag us with your celebration photos on Twitter @DrupalAssoc.
Be sure to check out the all of the new features now on the Drupal 10 page.
Thank you again to everyone who has been working to complete the requirements and strategic initiatives needed for Drupal 10! Do you have questions about the upcoming launch of Drupal 10? The Drupal 10 readiness team holds meetings every Monday at 18:00 UTC in #d10readiness on #Drupal Slack. Join the channel to discuss other questions about Drupal 10 anytime!
Stay tuned throughout the months leading up to the 14 December release target for more updates and information on Drupal 10, and make sure to follow us on Twitter @DrupalAssoc and @Drupal to stay up-to-date on all of the latest Drupal 10 news.
Pixelite: Drupal and the Open Web in the Australian Government - 2022 edition
This is the complementary blog post for my DrupalSouth Brisbane 2022 session.
Have you ever wondered how popular Drupal is in your local state and at the Australian Federal Government level? This blog post will help to answer that question, using open source tooling. The hope is that you gain some insight to the relative popularity of Drupal and appreciate more the impact you and Drupal have in Australia.
Existing solutions
There are a number of websites that will claim to be able to give you this information. However they all will likely want you at some point to pay them money.
- wappalyzer.com
- semrush.com
- builtwith.com
- whatcms.com
- similartech.com
- larger.io
“I wanted an open way to do this”
As it turns out, you can plug a few things together to scrape the technologies in use
- Wappalyzer library (open source)
- Puppeteer (open source)
Problem #1: How to get a list of all Australia Government domains
If anything, there are too many sources of this information:
A crawling method could also be done, loads of suitable seed sites, e.g.:
The main issue is that just having a list of sites, does not convey the importance of the site relative to another site.
Enter DomCop to which publishes a list of the top 10 million domains on the internet, including a rank and Open PageRank.
.gov.au
applied.On top of suppling 5,795 Australian Government domains, there also is an "Open Page Rank" field. The PageRanks are calculated based on the Open data provided by Common Crawl and Common Search.
Problem #2: What is PageRank?
PageRank is a system for ranking web pages that Google's founders developed in 1996. A PageRank score of 0 is typically a low-quality website, whereas, a score of 10 would represent only the most authoritative sites on the web. It is logarithmic (with a base of 5).
A site with PageRank 3 is 5 times more authoritative than a site with PageRank 2.
Problem #3: Machinery of Government (MoG) and link rot
Australian Governments sites are never static, they are constantly evolving. Sometimes several sites merge into 1, or sometimes 1 site splits into move sites.
Just show me the graphs
Disclaimer:
- This is based on Sept 22, 2022 data
- The scoring is based off PageRank data, so the percentages are not raw counts of websites, but an approximation of how important the respective sites are compared to others (assumes a logarithmic base of 5).
- Wappalyzer detection is not perfect (see the end of this blog post for upstreamed PRs), and there is still a fairly large portion of sites where the CMS cannot be identified
- MoGs make this tricky (PageRank relies on incoming links, which break due to MoGs)
- Only
*.gov.au
domains considered (some Government sites use other TLDs) - Unlikely newly created websites are in the top 10 million just yet (due to how PageRank works)
All sites (*.gov.au)
Federal sites (not state based domains)
Programmes like GovCMS are having an impact here.
Victoria *.vic.gov.au
The Single Digital Presence (SDP) programme makes a mark in Victoria.
New South Wales *.nsw.gov.au
Large Drupal sites like https://www.nsw.gov.au/ and https://www.service.nsw.gov.au/ help to make Drupal dominant in NSW.
South Australia *.sa.gov.au
Western Australia *.wa.gov.au
A lot of unknown CMSs in WA, including sites like https://ww2.health.wa.gov.au/ which I still have no idea what the CMS used is.
Tasmania *.tas.gov.au
The lowest usage of Drupal for any Australian state or territory and the highest percentage of Wordpress.
Queensland *.qld.gov.au
Australian Capital Territory *.act.gov.au
The highest percentage of Squiz compared to any other Australia state or territory.
Northern Territory *.nt.gov.au
Open Source Software (OSS) CMS vs Proprietary CMS
For the CMS' that can be identified, splitting them into 2 categories, OSS and Proprietary.
Drupal sites by major version
For sites reporting as Drupal, Drupal 9 and 7 are the most popular.
Observations and other unusual findings
#1 - Drupal usage
“Drupal powers roughly 27% of all digital experiences that you use in the Australian government”
#2 - Top contender
“Squiz Matrix is the top contender with 15%, and has a clear state led mandate in certain states/territories”
#3 - TLS coverage
TLS coverage is not 100% - 129 domains found with no TLS
Domain
CMS
Page Rank
Score
unknown
5.51
7,101
unknown
4.47
1,332
http://www.mbsonline.gov.au/internet/mbsonline/publishing.nsf/Content/Home
hcl-notes
4.45
1,289
http://onesearch.slq.qld.gov.au/primo-explore/search?vid=SLQ
unknown
4.41
1,209
http://www.majorprojects.planning.nsw.gov.au/
unknown
4.41
1,209
#4 - If in doubt, add a number
19 domains found with ww[number]
as a subdomain.
Domain
CMS
Page Rank
Score
drupal
4.85
2,455
unknown
4.62
1,695
unknown
4.49
1,375
unknown
4.3
1,013
https://www0.landgate.wa.gov.au/
squiz-matrix
4.25
935
#5 - You cannot kill Dreamweaver
15 sites found in 2022.
Extending this for the future
- Crawl other domain spaces, e.g. the New Zealand government domain space
*.govt.nz
- Make a website and publish this data quarterly (DomCop's data updates around this frequency)
- Measure trends over time
Upstreamed enhancements
These are all to make the detection of CMS' and Javascript frameworks more accurate for Australia Government sites.
- Ripple #6827
- Dreamweaver #6828
- HCL DX #6829
- MODX #6832
- Squiz Matrix #6833
- Optimizely CMS #6834
- Umbraco #6835
- Elcom #6836
- Lagoon #6861
- OSS flags and links #6902
- Squiz Matrix #2 #6925
- Ektron CMS #6924
- SiteCore #6926
Raw data
If you want to do your own analysis, here is a link to a full CSV dump.
Comments
I am keen to hear feedback on this data, and what can be done to improve the scoring. Also, if you can help fill in some of the 'unknown' data, let me know, I am happy to craft another PR into Wappalyzer.
#! code: Drupal 9: Using PHP_CodeSniffer To Inspect Custom Code
Drupal has a number of coding standards and best practices that govern the way code should be written. This has many benefits but can allow for a consistent and maintainable code to be created.
All Drupal modules and themes are written with coding standards in mind and that allows Drupal developers to look at any project and see a similar style of code. If you ever submit code to Drupal or a contributed project you will be required to adhere to these coding standards, so it makes sense to get used to knowing about them.
The main tool that is used to test coding standards in Drupal is PHP_CodeSniffer, which is a widely used static analysis tool for testing PHP coding standards. It comes with a set of coding standards built in, but it is possible to add the Drupal coding standards for the tool to use.
In this article we will look at installing PHP_CodeSniffer and adding the Drupal coding standards to it. We will also looking at some of the more common coding standards violations that might be encountered and how to solve them.
Installing PHP_CodeSniffer
There are two ways in which to install PHP_CodeSniffer, globally and as a dependency on the project you are working on. Each of these approaches has different pros and cons, so we will address those whilst looking at how to install each.
Evolving Web: Planning for Your Website’s Future
How long should a website last for? The average lifecycle of a website is about two and a half years. At Evolving Web we aim to develop products that will last between five and ten years. However, this sort of longevity can only be achieved through ongoing improvements.
In March 2022 we published a blog post on the 10 signs your website is outdated. This post is worth reading in full, but to briefly recap, these are 10 signs that you might be due for some investment and serious improvement:
-
Users find it hard to navigate.
-
It’s content-heavy and hard for users to understand what you do.
-
It lacks clear calls to action.
-
You have outdated or incorrect content.
-
Search results aren't relevant.
-
Load times are too slow.
-
You need developer help anytime you want to make a change.
-
There are too many PDFs or content that's not accessible.
-
It doesn't allow you to implement SEO best practices.
-
It’s not optimized for mobile devices.
If one or more of these issues apply to your site, you definitely need to consider an overhaul. How extensive an overhaul is required depends on a number of factors, which can be addressed in a website transition plan.
Looking For the Signs
A website is a lot like a house. Fixtures and appliances age. Paint jobs, roofs and other elements occasionally require touch-ups or complete replacement. Clutter builds up and needs to be dealt with. Families expand, creating new demands.
The main difference between a website and a house is that your house is a private space, whereas your website is your company or organization’s gateway to the world, making it all the more pressing that it be up to date. Your house will survive without the latest developments in interior design. Your website, not so much.
Just as homeowners need to keep an eye out for signs that their houses might need minor or major repairs or remodelling, website owners need to do the same with their sites. That two-and-a-half-year lifecycle is not a long period of time. In fact, there are steps that are well worth taking as soon as your website first goes live.
Websites benefit from periodic, scheduled benchmarking. This means looking at competitors’ websites and seeing what they may be doing better (or at least differently) than you. Are your competitors using new keywords? Are they offering online features that you aren’t? Are they doing a better job at directing traffic to their sites through digital channels? Consider doing a benchmarking analysis every six months or so.
User testing is an important element of any website development project, but it is also a useful tactic to re-employ after your site is operational. This is especially useful for budget-conscious companies or organizations, as virtually anyone can find a handful of external stakeholders from a target audience willing to offer feedback on a website. Chances are that said stakeholders have also spent time on competitors’ sites and can offer comparative feedback.
Employing these and other tactics should give you ample warning that your site is perhaps not performing as well as it could be and that improvements are overdue.
For more on testing, read our recent blog post Design, Test, Design Again: Our Approach to Usability Testing.
10 Steps in Planning Your Website Improvement Project
Once you’ve identified that your site has shortcomings, it’s time to drill down to the heart of the problem. Is navigability a problem? Does your content not pass muster? Is your site falling short of accessibility standards? Here are ten steps you can take to identify the problems with your site that will enable you to take action.
1. Do a Content Inventory.
The first step to take in any website transition planning process is to map out your website’s content from top to bottom. Once you have a complete sitemap, you should be able to identify any problems your users might be having in navigating your site.
2. Conduct a Content Audit.
The next step is to do a full content audit. The purpose of this is to identify what content is serving you well (i.e. drawing traffic), what content is gathering dust and how specific pages and types of content have fared over time. Metrics to examine include page rankings, user behaviour (page views, bounce rate etc.), engagement and sales.
While companies or organizations may opt to undertake this internally, Evolving Web has extensive experience conducting content audits and would be happy to help you arrange this process.
3. Assess Your Tone of Voice.
Does your organization have guidelines around brand voice and tone? If so, is the voice and tone of your website consistent with those guidelines? A site that may have started off with a consistent tone of voice may start to lose this over time. As such, a check-in on brand identity is well worth the effort.
4. Review Your Company or Organization’s Mission Statement.
A website is a tool that should be embodying your company or organization’s mission statement, and if it no longer is there’s a problem. It’s worth reexamining if your mission statement comes across on your website – more than just a brief mention in your “About Us” section.
5. Review your Keywords.
While juggling keywords is no longer a valid search engine optimization strategy in itself, it still matters that you use the right keywords. Use Google Analytics to find out what terms people are using to find your site. Research related search terms by searching your current keyword in Google and see what the search engine suggests.
You can also use a keyword search platform like Google Keyword Planner, SEMrush or KWFinder to get the latest on what keywords people interested in your industry are using. You can also reach out to us directly for help with your SEO strategy.
6. Research Alternative CMS Options.
If there are mechanical issues with your website, like overly slow loading times, it could be that your CMS is no longer the best fit for the type of content you’re seeking to provide. It could be time to start looking into alternative CMS options.
7. Do an Accessibility Scan.
As with Step 2, conducting an accessibility audit should be achievable internally, through your own stakeholder network. At the outset of a website development project, it is wise to develop a network of people who either have disabilities or work with assistive technology who can assist with an initial accessibility audit and follow-up checks to ensure that your site continues to be accessible to all.
8. Do QA Testing for Mobile Devices.
This test can easily be done internally within a team – provided the people on that team operate mobile devices. Just as it’s crucial to test your site on mobile devices at the outset, it is equally critical to continue doing so as the site ages.
9. Keep an Eye on Analytics and Digital Trends.
How are people reaching your site? Has this picture changed over time? What digital channels are your competitors using? Keeping your eye on your analytics picture as well as on trends in digital media will alert you to dynamics that could affect how people interact with your website.
10. Get Help.
Much of the work involved in these steps, and the actions needed to remedy the situations, can be done in-house by most teams. However, if you’ve been through all these steps and found multiple issues requiring extensive overhaul, it’s probably time to call in the professionals. At Evolving Web we are well-versed in all these aspects of website transitions and would love to assist you with your site revamp.
The Role of an Agency
Ever wonder why we chose the name “Evolving Web”? Simply put, we chose it because it reflects our view of websites as continuously evolving entities that need to be nurtured and guided over time. Websites have changed significantly since we first put out our shingle in 2007, and the work we do has evolved with them – as have our relationships with our clients. Constant evolution has been the theme of our work since the beginning.
The circumstances in which we enter into relationships with our clients are as diverse as the clients themselves. In some cases, our role is a simple migration from WordPress to Drupal, or from an older version of Drupal to the latest version. In other cases, a web project may involve a migration plus a significant content or UX overhaul – changes akin to a major house remodelling. And in some cases, we’re building a house completely from scratch.
At times we’re faced with a site with particular issues. In the case of Royal Roads University’s Our People site, we were tasked with fixing a site that was failing to deliver content to its intended audiences due to problems with information architecture. Through extensive interviews and surveys and a thorough content audit, we were able to reorganize the site in a way that satisfied all the university’s internal stakeholder groups.
Not sure where to start? We offer a wide range of different website audit services, which can help determine the state of your existing site and what it might need in terms of transition. These include a general-purpose audit (the most thorough version we provide), a strategic UX and content audit, a high-level in-depth technical audit, an accessibility audit or a pre-migration audit. An audit can help determine whether a straightforward migration is all that’s needed or if a more thorough redesign is necessary at this stage.
We can help you craft a website transition plan based on the audit's findings. A good transition plan should reflect the unique needs of your organization and map out a plan to get you to the website you need. Wherever you are in your “evolving website”, we can meet you there.
//--> + more awesome articles by Evolving Web
Drupal Core News: New provisional Drupal 7 maintainer: poker10
I am pleased to announce that Juraj Nemec (poker10) has accepted our invitation to become a provisional Drupal 7 core maintainer!
Juraj is based in Slovakia and has been working with Drupal for more than 12 years. He works at ActivIT as the Tech Lead and Senior Drupal developer. As a backend specialist, he most likes optimizing large-scale systems, performance tuning and digging into what can be improved to bring the best user experience.
He recently contributed to fixing Drupal 7's PostgreSQL tests, helped to improve PostgreSQL performance with several important backports, and is actively working on improving Drupal 7's compatibility with PHP 8.1 and 8.2. He is also helping to get the jQuery Update module ready for its first big update in several years.
Juraj is excited about the opportunity to contribute to the Drupal community. He will work with the Drupal 7 maintainer team (Fabianx and mcdruid).
Please join me in welcoming Juraj to the committer team!