If you have some data in a Drupal site, say a list of contact form submissions, or a list of content, you might want to provide a way for visitors to download that list as a CSV or Excel file: there's a module for that.
Views Data Export - The story
This module has its roots firmly in the Drupal 6 glory days where basically every list in Drupal was a view. Had some blog posts: view; list of comments: view; images needing copyright review: view.
That made a simple extension module that could export that content to CSV or Excel really appealing, and the Views Bonus Pack module provided this functionality however, there were a number of issues. Top of our list of problems with it was that it tried to do the entire export in one big go, if you had a lot of data, then you were out of luck, you simply couldn't export it all. You'd have to do it page by page or something like that.
In our old office, we dreamed up a way to have Drupal's batch API iterate over the result pages and collect them all up into a single file for download.
Additionally, at the time we had a client who wanted to export data that was coming in pretty rapidly: they were getting submissions at the rate of several a second and needed to be able to accurately export all the submissions in one go, not getting any duplicates, etc.
We really wanted to be able to get this working nicely with the SQL backend for Views and found a way in MySQL to query and populate a table with the resultset in a single transaction. In this way, we would make an initial point-in-time copy of the data required to render the view, and then we could take our time to process through that data. As long as MySQL could query and write the data fast enough we could then use Drupal's Batch API to loop through the data at our leisure.
This approach worked nicely but needed to fork the Views Bonus module and so the Views Data Export module was born. In the years that followed: the module gained popularity, a Drupal 7 port, and something like 75,000 active sites using it.
Decline and Drupal 8
ComputerMinds had never put in place any sort of proper amount of time to do maintenance for Views Data Export and so after many years and unattended bug reports, Drupal 8 rolled around and we didn't invest time in doing a Drupal 8 port either.
However, Drupal has a great community and someone else stepped up and offered to port and maintain Views Data Export.
However, the bugs and feature requests continued to pile up, and the port was more of a re-imagining than a direct port of the code. This was completely understandable since the underlying systems of Drupal and Views had massively changed in those years. And basically, we weren't involved in explaining what the Drupal 7 version was doing and why.
The module entered a period of very minimal maintenance and essentially only got further updates for critical bug fixes and new Drupal core versions.
Let me be clear that this is no slight against anyone who has done brilliant work on the Drupal 8, 9 or 10 versions of Views Data Export, but proper amounts of time and resources were never put into the module...until now.
Re-maintainership
Recently we've changed some internal priorities around so that we can spend more time giving back to the Drupal community, and this coincided with someone asking me to be a co-maintainer for Views Data Export so that they could add Drupal 11 support.
I thought about it a lot and waited for the other maintainers to weigh in, but ultimately decided that I wanted to get back involved in the maintainership of Views Data Export and thus declined the generous offer of help. I did then work to get a Drupal 11 version out.
Going forward
I aim to maintain Views Data Export.
I'm working on exactly how this works, but it starts by regularly spending time working on the project.
Now...the project pages and issue tracker on Drupal.org are in a bit of a dire state:
- The project page documentation is almost entirely the original text I wrote for Views Data Export for Drupal 6, so some of the information there is completely out of data.
- The issue queues have been left relatively unattended for a decent while, so we have (at the time of writing):
- 204 open bugs
- 276 other open issues.
- Several issues that are marked as RTBC that haven't been given any attention by the maintainers.
- Plenty of duplicate issues.
- Drupal.org issue queues are hard to read and work with, and slow :(
- The documentation is for Drupal 6/7 and is pretty bare-bones.
The code...well I've not read every line of code in the latest version of Views Data Export, I'm going to be doing that today to break up some of the issue queue management I'm going to be doing.
Drupal 7 version
According to the stats on Drupal.org there are still around 41,000 active Drupal 7 installs, and while those will be community EOLd in a month, I don't want to prematurely cut them off, so while I plan to close almost all of the Drupal 7 issues that I find, I won't mark the module unsupported as such. But that will happen in January 2025 anyway, when Drupal.org does that automatically.
Progress check
I intend to write an article like this one every 2 weeks detailing what has been done and what the plans for the future are. Mostly to keep me honest and focused! It's easy when starting on a bit of a big project like this to get lost and despondent about the size of the problem, so by trying to write about it, I'm hoping that I'll be able to look back and see progress even if won't always feel like any progress has been made!