Akansha Tech Journal: Build Smart Drupal Chatbots with RAG Integration and Ollama

Drupal RAG Integration: Diving into the world of Large Language Models (LLMs) and their potential in the Drupal ecosystem, I've crafted a solution that combines a Drupal module with a RAG (Retrieve, Augment, Generate) backend, enabled by FastAPI. The result? A smart chatbot that knows your website content like the back of its hand, ready to answer visitor queries with updated site knowledge every time you publish new content.

Spinning Code: Writing for Developers and Consultants: Listening

My first few articles in this series have been focused on the messages we send to others – mostly our written messages. This time I am focusing on Listening as key skills when communicating with others.

Listening is a skill. In general in our lives we are expected to know how to listen, even though most teaching is informal.

If you aren’t listening to your colleagues and clients (friends, family, etc.), which can include reading carefully the materials they write, how can you communicate back to them clearly? And when they can tell you didn’t listen closely, why should they listen to you carefully?

The Importance of Listening

Listening when other people speak, or reading their words when they send you an email or chat message, is how we learn about other people’s needs. As developers and consultants we need to understand the needs of our users and clients. If we don’t understand what they need, we will provide the wrong thing.

Active Listening

Listening is not a passive activity. If you are simply allowing someone’s words to wash over you, but not engaged with their meaning you are missing critical details of what’s being communicated.

Active Listening is a skill you can teach yourself and improve on over time.

When we speak we communicate with more than words. Our body language, tone, pace of speaking, and a host of other details go into how we communicate our message. When we actively listen, we are absorbing all those details to make sure we’re getting the whole message.

Active Listening involves also reflecting back that you’re paying attention to the speaker. You can use your own body language to send none verbal cues that you’re paying attention. You can also make affirming noises, and other auditory markers that you’re following along. And when you start to speak you can paraphrase their comments to demonstrate you understood the previous person’s contributions.

Active Listening in an Age of Video Meetings

Most materials you’ll find on Active Listening focus on in-person discussions. That’s in part because people have been talking about Active Listening for decades and the technology is still playing catchup.

But you can bring Active Listening skills to video meetings — most easily if you have your camera on.

While affirming noises and other auditory responses can cause audio filters to struggle – clipping one speaker’s audio to provide your supportive sounds – there are still ways to make sure people know you’re listening. When you camera is on, looking at the screen closest to your camera, nodding or shaking your head, making (work place appropriate) hand gestures, and other visual cues can be helpful. Using the reactions features of most systems to give thumbs up, applause, and other indicators can also help send the message that you’re paying attention.

The hardest talk I ever had to give was an online conference early in the Covid Pandemic. I had no real-time feedback from the audience at all. I was not told how many people were listening; I could not see any chat messages: no input at all. It was just me, staring at my slides, trying to maintain good energy. Eventually I got questions read to me at the end that suggested at least someone was watching – but for most of my talk I felt like I was talking to an empty room.

Give people input whenever you can without being distracting. Helping them understand they are being heard will make it easier for them to communicate with you.

Listening as a Form of Respect

Truly listening to another person is a marker of respect. You are demonstrating that the other person is worth your time and energy. If that’s not obvious to you already, think about the difference between a friend who is looking at you while you’re talking vs that same friend looking at their phone; which makes it clear your friend cares about you?

At work the same is true with colleagues. If you are looking at your phone, reading email, looking at social media, or any number of other activities that pull your attention away you are communicating the person isn’t as important to you as all those distractions.

We all are guilty of this from time to time. I have been pushing myself recently to admit it to other people because it gets me to stop.

For example, the other day just as I as starting a call, I got a message from someone else that I found very frustrating – and was time sensitive. I started to reply while also starting the call. I wasn’t really respecting my colleague’s time. So I apologized to my colleague, asked for a moment to reply to the message explaining it was both time sensitive and distracting, and then I focused on our call. I was both more focused on our conversation two minutes later, and I avoided annoying her by constantly looking away at the other message. Because she listened to me, it also meant we could restart our conversation by commiserating about distracting messages that pull our attention away from meetings.

Communicating well requires full engagement in the work, but in the messages you send and in making sure you receive messages as well.

As I said in my first post on this topic, communications skills for developers and consultants is an enormous topic. The plan for this series is evolving as I go. If you have suggestions or requests feel free to leave me a message.

The post Writing for Developers and Consultants: Listening appeared first on Spinning Code.

Wim Leers: XB week 9: front-end locomotive gathering steam

Experience Builder (XB) allows crafting an experience using components. A tree of components. Except that until now, the XB field type does not yet support storing a tree — we had to start somewhere! Ted “tedbow” Bowman landed #3455728: FieldType: Support storing component trees instead of lists. Next up on this front: before we start actually storing trees, we need thorough validation — Ted’s tackling that next. This will allow us to confidently store trees of components (by putting components in other components’ slots), and then finally put to good use the oldest MR the XB project currently has: Kyle “ctrlADel” Einecker’s MR that adds representative set of Single-Directory Components.

An important XB config management piece landed this week: #3452397: Allow specifying default props values when opting an SDC in for XB, started by Feliksas “f.mazeikis” Mazeikis, finished by me because it blocked next steps and Felix was temporarily reallocated to work on something other than XB#3460232: Support props defaults that have content dependencies: avoid File content entity dependencies is the next step and is the first thing Felix will work on upon his return.

Thanks to Lee “larowlan” Rowlands finding the time to review #3453152: Centralize & standardize logic for constructing PropSource objects + kernel test coverage, now that landed at last, which doesn’t set everything straight, but is a massive improvement compared to the PoC-y origins of PropSource. In the short term, the most urgent next step is #3461490: Document the current JSON-based data model, and describe in an ADR.

On the front end side, Jesse “jessebaker” Baker contributed a diagram to visualize how he envisions the UI will work:

Image removed. Initial version of a diagram of the UI’s architecture as envisioned by Jesse Baker. We’ll update this along the way.

And Harumi “hooroomoo” Jang landed the initial implementation of the top bar, a week after the design was available:

Image removed. The top bar’s mid-fidelity design. Going forward, every issue changing the UI will have to include a screenshot of the updated UI — that will make it easier for anybody to follow along!

To make UI work like Harumi’s go faster, I made CI faster for people working only on the UI: no need to run PHP-related CI jobs then, and after Ben “bnjmnm” Mullins pointed it out, I added an explicit CI job to flag when npm run build fails — because until now only the Cypress CI job would fail, which seemed to suggest that is where the problem was — oops!

In progress

That was all that landed this week, but there’s a lot of things in progress. Here are the most notable ones:

  • Superficially boring, but very important for velocity: Lee filed #3452585: CI: Cypress test infrastructure clean-up: split in cypress E2E + cypress unit CI jobs waaaaay back, on June 5! I started pushing that forward to help the people focused on the front end (Jesse, Ben & Harumi) go faster — didn’t land yet, but was getting close. It’s now tantalizingly close.
  • @fazilitehreem started working on #3459249: Allow opening the contextual menu by right clicking a component!
  • Ben is making good progress on a truly mind-blowing MR — where he’s using a subset of the JSX theme engine 1 to render existing Field Widgets using the existing Twig templates … but using JSX equivalents for a select subset of those templates, to achieve native integration with React for existing Field Widgets, which in turn will enable real-time updates of the preview.
    This approach allows mixing of JSX and Twig templates, and functionality such as vanilla Drupal JS behaviors and form alters continue to work for both Twig- and JSX- rendered markup.
    If that tickled your brain or blew your mind, there’s more to come! There’s a lot of moving parts, but the intent of XB is to bring the entire existing ecosystem along for the superior ride, and this is an important part of that.

Missed a prior week? See all posts tagged Experience Builder.

Goal: make it possible to follow high-level progress by reading ~5 minutes/week. I hope this empowers more people to contribute when their unique skills can best be put to use!

For more detail, join the #experience-builder Slack channel. Check out the pinned items at the top!

Finally, the design locomotive is now going full steam ahead:

In closing, catch asked a few very good fundamental questions — these are the kinds of questions that we all need to try to articulate: not about implementation details, but about potential gaps/oversights in the product requirements.

Thanks to Ben and Lauri for reviewing this!

  1. This theme engine was built by Ben, Harumi and Alex “effulgentsia” Bronstein long before XB even was in the picture! ↩︎

Sitback Solutions: Making the polyfill.io vulnerability a thing of the past for Drupal

Image removed.Recently, there has been a flurry of activity as the Drupal community has scrambled to mitigate a vulnerability caused when the company that bought polyfill(.io) started injecting malware into the scripts returned by the service, referred to as a supply-chain attack. If you own or manage a Drupal site and you haven’t heard about this, ... Making the polyfill.io vulnerability a thing of the past for Drupal

Promet Source: Provus®Gov vs CivicPlus for Local Government

Takeaway: Provus®Gov emerges as the superior choice for forward-thinking government entities. Powered by Drupal, Provus®Gov as an open-source, plug-and-play platform offers unmatched flexibility, scalability, and advanced features that grow with your needs. Its user-friendly interface empowers non-technical staff, while robust security measures and built-in accessibility compliance ensure your site meets the highest standards.

The Drop Times: Brad Jones on Modernizing Drupal's Data Management with JSON Integration

Brad Jones, a digital nomad and seasoned Drupal developer, merges his technical expertise and unique experiences as a part-time paramedic. His proposal, "JSON Data and Schemas FTW!", aims to revolutionize Drupal's data management by integrating JSON data types. Recognized at DrupalCon Pittsburgh's Pitch-burgh Innovation Contest, this initiative seeks to enhance flexibility and interoperability within Drupal.

eiriksm.dev: Getting rid of cronner.module

Background: This blog post is part 2 of musings around legacy code on violinist.io. Violinist.io is an automated dependency updater for PHP/Composer. It's a SaaS built on Drupal 8, now running on Drupal 10, in its eigth year. In this second post I am looking at one way to approach legacy code, and how to use static analysis and test driven development to safely refactor and remove legacy code.

Some months ago I wrote about the combined feeling of shame and respect that surrounds legacy code. In an attempt to get rid of the legacy code in what is called cronner.module, I will start from the top of the .module file, and work my way towards removing it completely. From time to time, that can also result in a good blog post, I thought. Here's a blog post about that.

The first lines of the module file is this:

<?php /** * @file * Cronner module. * * Bad name, but what are you going to do, right? */

The code here sounds a bit resigned when it asks about what are you going to do. But in this blog post, what are we going to do? We will start the process of getting rid of all of it, that's what we will do!

The first few lines of actual code are constant definitions. Here's the first one:

define('CRONNER_STATE_KEY_PREFIX', 'cronner_state.node.');

So far we can speculate, but it seems to be a prefix. And it's used for storing some state about nodes. Defining constants like this might seem strange if you are a bit new to Drupal, but this way of defining constants was canonical in Drupal 7 and below, but also followed a lot of codebases into the Drupal 8 era. Drupal core as well actually. One such example is the constant FILE_EXISTS_RENAME which lived on until Drupal 9 (deprecated in 8.7.0). Anyway I digress.

What we want then, is to remove this one constant. Let's just try that and see what breaks? Here's the output from phpunit:

There were 19 errors: … Error: Undefined constant "CRONNER_STATE_KEY_PREFIX”

The unit tests are failing. 19 of them. Good start, we have decent unit test coverage. Let's see if our integration tests fail?

xxx scenarios (xx passed, 112 failed) xxxx steps (xxxx passed, 49 failed, xxx skipped) xxmxx.xxs (90.59Mb)

112 scenarios failed in our behat test suite. Great indication that we can remove things safely! Lastly let's do some static analysis with PHPStan:

Line web/modules/custom/cronner/cronner.module ------ --------------------------------------------------------------------- xxx Constant CRONNER_STATE_KEY_PREFIX not found.

As expected, PHPStan also informs me about this change being a problem. Thanks PHPStan! It also indicates that among the scanned files, the constant is only used once. It's used like this:

/** * Gets the state key of a node. */ function _cronner_get_state_key(NodeInterface $node) { return CRONNER_STATE_KEY_PREFIX . $node->id(); }

In addition to the comment being very little helpful, this tells me I have just encountered another part of the cronner module to remove. Let's remove this entire function as well since we already know the function must be covered by both unit tests and integration tests. Then let’s go ahead and use a combination of TDD and static analysis (with PHPStan) to make sure our refactoring is successful. I remove the function and re-run PHPStan:

Function _cronner_get_state_key not found. … [ERROR] Found 7 errors

PHPStan is helpfully pointing out all of the 7 places I should start with the refactoring. I look through some of them and see the usages are quite connected to some of the other constants in the beginning of the file:

define('CRONNER_PROJECT_NEW', 'new'); define('CRONNER_PROJECT_QUEUED', 'queued'); define('CRONNER_PROJECT_RUNNING', 'running'); define('CRONNER_PROJECT_PROCESSED', 'processed'); define('CRONNER_PROJECT_ERRORED', 'errored'); define('CRONNER_PROJECT_UNKNOWN', 'unknown');

My mother always used to say. When you are tidying up your room it’s best to keep going while you are somewhat effective in deciding what to get rid of. If you find some of your old toys and start to play with them, it’s an indication that the tidying session is heading towards an unproductive state. And right now I feel effective and decide right away I am removing those constants as well. I guess that also makes the analogy to me tidying my room as a kid kind of weird, since the opposite would mean these constants were my toys, and I end up playing with them? I mean, I do end up playing with old code from time to time, but constants? No fun playing with.

But looking at these old toys, it also becomes obvious what we use these constants for. Node state could mean all kinds of things, right? This is used to store and update the job status of the projects. Like if they are currently queued, if they are currently running and so on.

So it turns out that while removing this, this actually seems like a good opportunity to clean up some of that code and make it a bit more modern. What I usually do when cleaning up custom code like this is to put as much as possible on drupal.org as open source code. This is the case now as well. So I open an issue to create a project run status service, and start to refactor the custom code and logic surrounding the removals of constants and functions in cronner.module. It can be found here: https://www.drupal.org/project/violinist_projects/issues/3453459.

Now I go ahead and start using this service. It quickly becomes obvious that some of the constants are also used in a map to display a human readable status to the user:

/** * Gets a human readable version of a status constant. * * @param string $status * A status constant. * * @return string * Something more sensible. */ function cronner_get_human_status($status) { $map = [ CRONNER_PROJECT_UNKNOWN => t('Unknown'), CRONNER_PROJECT_ERRORED => t('Errored'), CRONNER_PROJECT_PROCESSED => t('Processed'), CRONNER_PROJECT_RUNNING => t('Running'), CRONNER_PROJECT_NEW => t('New'), CRONNER_PROJECT_QUEUED => t('Queued'), ]; return !empty($map[$status]) ? $map[$status] : $status; }

There’s also methods for getting a state from a node, and setting it. Which I am also removing, and finding usages of in static analysis:

xxx Function cronner_set_state not found. 💡 Learn more at https://phpstan.org/user-guide/discovering-symbols xxx Function _cronner_get_state_key not found. 💡 Learn more at https://phpstan.org/user-guide/discovering-symbols xxx Function cronner_get_human_status not found. 💡 Learn more at https://phpstan.org/user-guide/discovering-symbols

Or in unit tests like so:

ReplaceTokensTest::testClaimJobAndTokenReplaced Error: Call to undefined function cronner_set_state()

Instead of these calls to getting the state, setting the state, and getting the human readable state, I am now using the newly created service. When I started writing this blog post I was very much looking forward to summarizing the numbers of deleted lines and how great that was, but in practice some of the changes are actually adding lines to the module:

- cronner_set_state($node, CRONNER_PROJECT_PROCESSED); + /** @var \Drupal\violinist_projects\ProjectRunStatus $run_status_service */ + $run_status_service = \Drupal::service('violinist_projects.run_status'); + $run_status_service->setRunStatusForProject($node, ProjectRunStatusValue::STATUS_PROCESSED);

Jokes aside. After some successful deletions and refactorings into using a service I am looking at 99 deletions and 55 additions. A net positive result I would say. Unfortunately there is still quite a way to go before I can go ahead and delete the entire folder called “cronner”. But at least I managed to refactor the codebase and delete the first 7 lines of constants in the file.

To celebrate this tiny step towards getting rid of cronner.module I tried to search for cronner on giphy. No hits, at this point. Oh well, here is an animated gif that supposedly illustrates “cronn”