As you may know, Drupal 6 has reached End-of-Life (EOL) which means the Drupal Security Team is no longer doing Security Advisories or working on security patches for Drupal 6 core or contrib modules - but the Drupal 6 LTS vendors are and we're one of them!
Today, there is a Moderately Critical security release for the Views module to fix an Access Bypass vulnerability.
The Views module allows site builders to create listings of various data in the Drupal database.
The Views module fails to call db_rewrite_sql() on queries that list Taxonomy Terms, which could cause private data stored on Taxonomy Terms to be leaked to users without permision to view it.
This is mitigated by the fact that a View must exist that lists Taxonomy Terms which contain private data. If all the data on Taxonomy Terms is public or there are no applicable Views, then your site is unaffected.
See the security advisory for Drupal 7 for more information.
Here you can download the Drupal 6 patch.
If you have a Drupal 6 site using the Views module, we recommend you update immediately! We have already deployed the patch for all of our Drupal 6 Long-Term Support clients. :-)
If you'd like all your Drupal 6 modules to receive security updates and have the fixes deployed the same day they're released, please check out our D6LTS plans.
Note: if you use the myDropWizard module (totally free!), you'll be alerted to these and any future security updates, and will be able to use drush to install them (even though they won't necessarily have a release on Drupal.org).
Continuing along with my series of reviews of Acquia Developer Certification exams (see the previous one: Drupal 8 Site Builder Exam, I recently took the Back End Specialist – Drupal 8 Exam, so I'll post some brief thoughts on the exam below.
I didn't get a badge with this exam, just a cert... so here's the previous exam's badge!
I’ve written previously about git workflow for working on drupal.org patches, and about how we don’t necessarily need to move to a github-style system on drupal.org, we just maybe need better tools for our existing workflow. It’s true that much of it is repetitive, but then repetitive tasks are ripe for automation. In the two years since I released Dorgpatch, a shell script that handles the making of patches for drupal.org issues, I’ve been thinking of how much more of the drupal.org patch workflow could be automated.
Now, I have released a new script, Dorgflow, and the answer is just about everything. The only thing that Dorgflow doesn’t automate is uploading the patch to drupal.org (and that’s because drupal.org’s REST API is read-only). Oh, and writing the code to actually fix bugs or create new features. You still have to do that yourself, along with your cup of coffee.
So assuming you’ve made your own hot beverage of choice, how does Dorgflow work?
Simply! To start with, you need to have an up to date git clone of the project you want to work on, be it Drupal core or a contrib project.
To start work on an issue, just do:$ dorgflow https://www.drupal.org/node/123456
You can copy and paste the URL from your browser. It doesn’t matter if it has an anchor link on the end, so if you followed a link from your issue tracker and it has ‘#new’ at the end, or clicked down to a comment and it has ‘#comment-1234’ that’s all fine.
The first thing this comment does it make a new git branch for you, using the issue number and the name. It then also downloads and applies all the patch files from the issue node, and makes a commit for each one. Your local git now shows you the history of the work on the issue. (Note though that if a patch no longer applies against the main branch, then it’s skipped, and if a patch has been set to not be displayed on the issue’s file list, then it’s skipped too.)
Let’s see how this works with an actual issue. Today I wanted to review the patch on an issue for Token module. The issue URL is https://www.drupal.org/node/2782605. So I did:$ dorgflow https://www.drupal.org/node/2782605
That got me a git history like this:* 6d07524 (2782605-Move-list-of-available-tokens-from-Help-to-Reports) Patch from Drupal.org. Comment: 35; URL: https://www.drupal.org/node/2782605#comment-11934790; file: token-move-list-of-available-tokens-2782605-34.patch; fid 5784728. Automatic commit by dorgflow. * 6f8f6e0 Patch from Drupal.org. Comment: 15; URL: https://www.drupal.org/node/2782605#comment-11666939; file: 2782605-13.patch; fid 5710235. Automatic commit by dorgflow. / * a3b68cc (8.x-1.x) Issue #2833328 by Berdir: Handle bubbleable metadata for block title token replacements * [older commits…]
What we can see here is:
- Git is now on a feature branch, called ‘2782605-Move-list-of-available-tokens-from-Help-to-Reports’. The first part is the issue number, and the rest is from the title of the issue node on drupal.org.
- Two patches were found on the issue, and a commit was made for each one. Each patch’s commit message gives the comment index where the patch was posted, the URL to the comment, the patch filename, and the patch file entity ID (these last two are less interesting, but are used by Dorgflow when you update a feature branch with newer patches from an issue).
The commit for patch 35 will obviously only show the difference between it and patch 15, an interdiff effectively. To see what the patch actually contains, take a diff from the master branch, 8.x-1.x.
(As an aside, the trick to applying a patch that’s against 8.x-1.x to a feature branch that already has commit for a patch is that there is a way to check out files from any git commit while still keeping git’s HEAD on the current branch. So the patch applies, because the files look like 8.x-1.x, but when you make a commit, you’re on the feature branch. Details are on this Stack Overflow question.)
At this point, the feature branch is ready for work. You can make as many commits as you want. (You can rename the branch if you like, provided the ‘2782605-’ part stays at the beginning.) To make your own patch with your work, just run the Dorgflow script without any argument:$ dorgflow
The script detects the current branch, and from that, the issue number, and then fetches the issue node from drupal.org to get the number of the next comment to use in the patch filename. All you now have to do is upload the patch, and post a comment explaining your changes.
Alternatively, if you’re a maintainer for the project, and the latest patch is ready to be committed, you can do the following to put git into a state where the patch is applied to the main development branch:$ dorgflow commit
At that point, you just need to obtain the git commit command from the issue node. (Remember the drupal standard git message format, and to check the attribution for the work on the issue is correct!)
What if you’ve previously reviewed a patch, and now there’s a new one? Dorgflow can download new patches with this command:$ dorgflow update
This compares your feature branch to the issue node’s patches, and any patches you don’t yet have get new commits.
If you’ve made commits for your own work as well, then effectively there’s a fork in play, as your development in your commits and the other person’s patch are divergent lines of development. Appropriately, Dorgflow creates a separate branch. Your commits are moved onto this branch, while the feature branch is rewound to the last patch that was already there, and then has the new patches applied to it, so that it now reflects work on the issue. It’s then up to you to do a git merge of these two branches in order to combine the two lines of development back into one.
Dorgflow is still being developed. There are a few ideas for further features in the issue queue on github (not to mention a couple of bugs for some of the various possible cases the update command can encounter). I’m also pondering whether it’s worth the effort to convert the script to use Symfony Console; feel free to chime in with any opinions on the issue for that.
There are tests too, as it’s pretty important that a script that does things to your git repository does what it’s supposed to (though the only command that does anything destructive is ‘dorgflow cleanup’, which of course asks for confirmation). Having now written this, I’m obviously embarking upon cleaning it up and to some extent rewriting it, though I do have the excuse that the early weeks of working on this were the days after the late nights awake with my newborn daughter, and so the early versions of the code were written in a haze of sleep deprivation. If you’d like to submit a pull request, please do check in with me first on an issue to ensure it’s not going to clash with something I’m partway through changing.
Finally, if you find this as useful as I do (this was definitely an itch I’ve been wanting to scratch for a long time, as well as being a prime case of condiment-passing), please tell other Drupal developers about it. Let’s all spend less time downloading, applying, and rolling patches, and more time writing Drupal code!
The Drupal Association Engineering Team delivers value to all who are using, building, and developing Drupal. The team is tasked with keeping Drupal.org and all of the 20 subsites and services up and running. Their work would not be possible without the community and the project would not thrive without close collaboration. This is why we are running a membership campaign all about the engineering team. These are a few of the recent projects where engineering team + community = win!
We now have a new system for Documentation. These are guides Drupal developers and users need to effectively build and use Drupal. The new system replaces the book outline structure with a guides system, where a collection of pages with their own menu are maintained by the people who volunteer to keep the guides updated, focused, and relevant. Three years of work from the engineering team and community collaborators paid off. Content strategy, design, user research, implementation, usability testing and migration have brought this project to life.
Pages include code 'call-outs' for point-version specific information or warnings.
Thanks to the collaborators: 46 have signed up to be guide maintainers, the Documentation Working Group members (batigolix, LeeHunter, ifrik, eojthebrave), to tvn, and the many community members who write the docs!Enabling Drupal contribution everywhere
Helping contributors is what we do best. Here are some recent highlights from the work we're doing to help the community:
Users now have better profiles to make into Drupal résumés.
Months of content strategy work resulted in a better communication plan, including improved user experience for newly registered users.
Organization pages have expanded in scope to encourage more participants than just Drupal service providers. The organization list, new layout making case studies and contributions stand out, and a more robust contribution credit system are all helping to bring more contributors to the Drupal ecosystem.
We're expanding the contribution credit system to include more types of contribution, and we'll keep working on improving the system with check-ins every 6 months.
We have a process for prioritizing community initiatives so we can still help contributors. Thanks to volunteers who have proposed and helped work on initiatives recently, we've supported the launch of the Drupal 8 User guide and the ongoing effort to bring Dreditor features into Drupal.org itself.
You can help sustain the work of the Drupal Association by joining as a member. Thank you!
Our community is made up of incredible members from across the globe who continue to grow the Drupal project and create communities. DrupalCon is a place where community leaders and key contributors come together to meet, learn, and collaborate. The Drupal Association's Grant and Scholarship Program makes attending possible for many community members who may not have been otherwise able to join us.
Thirsty for Drupal knowledge? Want to dive deep into a topic and learn from the best in the field? Like to get hands-on with your learning material? We are excited to offer 10 full-day training classes at DrupalCon Baltimore that will turn you into a Drupal superhero. No matter if you are an absolute beginner or Drupal expert, our classes cover all experience levels.
When thinking about ways to measure your website’s effectiveness, you may also want to think about the metrics you use to gauge the success of the website in accomplishing your business goals. How else do you measure success?
Drupal 8 integrating with Traditional ECMs to enhance Enterprise Content Management Capabilities. Drupal 8 is a strong platform with it’s Strong Web Content Management System, has a role to Play in integrating with the existing Challengers/Traditional ECMs to enhance their Enterprise Content Management Capabilities.
If you want to create a web resource, or to implement certain improvements on an existing one, then you must find specialists who will make your ideas come true. It’s quite a challenging task. We offered our 10 tips for hiring a top web developer by mentioning qualities that must be possessed. However, in addition, there are different types of web developers, with different skills, duties and types of work. The objectives and the scale of your project require appropriate experts.Read more
In this blog I want to explain the round up we have done around the refactoring of the acl_contact_cache. In the previous sprints we discovered that a lot of the performance was slowed down by the way the acl_contact_cache was used (or rather not used at all). See also the previous blog post: https://civicrm.org/blog/jaapjansma/the-quest-for-performance-improvements-5th-sprint
At the socialist party they have 350.000 contacts and around 300 users who can access civicrm. Most of the users are only allowed to see only the members in their local chapter.
In the previous blog we explained the proof of concept. We now have implemented this proof of concept and the average performance increase was 60%.
We created a table which holds which user has access to which contacts. We then fill this table once in a few hours. See also issue CRM-19934 for the technical implementation of this proof of concept.Performance increase in the search query
In the next examples we are logged in as a local member who can only see members in the chapter Amersfoort. We then search for persons with the name 'Jan'. And we measure how long the query for searching takes.
The query for presenting the list with letters in the search result looks likeSELECT count(DISTINCT contact_a.id) as rowCount FROM civicrm_contact contact_a LEFT JOIN civicrm_value_geostelsel geostelsel ON contact_a.id = geostelsel.entity_id LEFT JOIN civicrm_membership membership_access ON contact_a.id = membership_access.contact_id WHERE ((((contact_a.sort_name LIKE '%jan%')))) AND (contact_a.id = 803832 OR (((( ( geostelsel.`afdeling` = 806816 OR geostelsel.`regio` = 806816 OR geostelsel.`provincie` = 806816 ) AND ( membership_access.membership_type_id IN (1, 2, 3) AND ( membership_access.status_id IN (1, 2, 3) OR (membership_access.status_id = '7' AND (membership_access.end_date >= NOW() - INTERVAL 3 MONTH)) ) ) ) OR contact_a.id = 806816 )) AND (contact_a.is_deleted = 0) )) ORDER BY UPPER(LEFT(contact_a.sort_name, 1)) asc;
As you can see that is quite a complicated query and includes details about which members the user is allowed to see. Only executing this query takes around 0.435 seconds and the reason is that mysql has to check each record in civicrm_contact (which in this case is around 350.000 and growing)
After refactoring the acl cache functionality in CiviCRM Core the query looks different:SELECT DISTINCT UPPER(LEFT(contact_a.sort_name, 1)) as sort_name FROM civicrm_contact contact_a INNER JOIN `civicrm_acl_contacts` `civicrm_acl_contacts` ON `civicrm_acl_contacts`.`contact_id` = `contact_a`.`id` WHERE (((( contact_a.sort_name LIKE '%jan%' )))) AND `civicrm_acl_contacts`.`operation_type` = '2' AND `civicrm_acl_contacts`.`user_id` = '803832' AND `civicrm_acl_contacts`.`domain_id` = '1' AND (contact_a.is_deleted = 0) ORDER BY UPPER(LEFT(contact_a.sort_name, 1)) asc
The query now takes around 0,022 seconds to run (20 times faster).Explanation
How does this new functionality works:
1. Every time an ACL restriction is needed in a query civicrm core only does an inner join on the civicrm_acl_contacts table and that is all
2. The inner join is generated in the service 'acl_contact_cache' that service also checks whether the the civicrm_acl_contacts table need to be updated or not.
3. When an update of civicrm_acl_contacts table is needed depends on the settings under administer --> System Settings --> Misc --> ACL Contact Cache Validity (in minutes)
So how does this look like in code?
Below an example of how you could use the acl_contact_cache service to inject acl logic into your query:// First get the service from the Civi Container $aclContactCache = \Civi::service('acl_contact_cache'); // The $aclContactCache is a class based on \Civi\ACL\ContactCacheInterface // Now get the aclWhere and aclFrom part for our query $aclWhere = $aclContactCache->getAclWhereClause(CRM_Core_Permission::VIEW, 'contact_a'); $aclFrom = $aclContactCache->getAclJoin(CRM_Core_Permission::VIEW, 'contact_a'); // Now build our query $sql = "SELECT contact_a.* FROM civicrm_contact contact_a ".$aclFrom." WHERE 1 AND ".$aclWhere; // That is it now execute our query and handle the output...
The reason we use a service in the Civi Container class is that it is now also quite easy to override this part of core in your own extension.
The \Civi\ACL\ContactCache class has all the logic to for building the ACL queries. Meaning that this class contains the logic to interact with the ACL settings in CiviCRM, with the permissioned relationship etc.. All those settings are taken into account when filling civicrm_acl_contacts table which is per user and per operation once in the three hours.
The release candidate phase for the 8.3.0 minor release begins the week of February 27. Starting that week, the 8.3.x branch will be subject to release candidate restrictions, with only critical fixes and certain other limited changes allowed.
8.3.x includes new experimental modules for workflows, layout discovery and field layouts; raises stability of the BigPipe module to stable and the Migrate module to beta; and includes several REST, content moderation, authoring experience, performance, and testing improvements among other things. You can read a detailed list of improvements in the announcements of alpha1 and beta1.
Minor versions may include changes to user interfaces, translatable strings, themes, internal APIs like render arrays and controllers, etc. (See the Drupal 8 backwards compatibility and internal API policy for details.) Developers and site owners should test the release candidate to prepare for these changes.
8.4.x will remain open for new development during the 8.3.x release candidate phase.
Drupal 8.3.0 will be released on April 5th, 2017.No Drupal 8.2.x or 7.x releases planned
March 1 is also a monthly core patch (bug fix) release window for Drupal 8 and 7, but no patch release is planned. This is also the final bug fix release window for 8.2.x (meaning 8.2.x will not receive further development or support aside from its final security release window on March 15). Sites should plan to update to Drupal 8.3.0 on April 5.
One of the problems with Drupal distributions is that they, by nature, contain an installation profile — and Drupal sites can only have one profile. That means that consumers of a distribution give up the ability to easily customize the out of the box experience.
This was fine when profiles were first conceived. The original goal was to provide “ready-made downloadable packages with their own focus and vision”. The out of the box experience was customized by the profile, and then the app was built on top of that starting point. But customizing the out of the box experience is no longer reserved for those of us that create distributions for others to use as a starting point. It’s become a critical part of testing and continuous integration. Everyone involved in a project, including the CI server, needs a way to reliably and quickly build the application from a single command. Predictably, developers have looked to the installation profile to handle this.
This practice has become so ubiquitous, that I recently saw a senior architect refer to it as “the normal Drupal paradigm of each project having their own install profile”. Clearly, if distributions want to be a part of the modern Drupal landscape, they need to solve the problem of profiles.Old Approach
In July 2016, Lightning introduced lightning.extend.yml which enabled site builders to:
- Install additional modules after Lightning had finished its installation
- Exclude certain Lightning components
- Redirect users to a custom URL upon completion
This worked quite well. It gave site builders the ability to fully customize the out of the box experience via contrib modules, custom code, and configuration. It even allowed them to present users with a custom “Installation Done” page if they chose — giving the illusion of a custom install profile.
But it didn’t allow developers to take full control over the install process and screens. It didn’t allow them to organize their code they way they would like. And it didn’t follow the “normal Drupal paradigm” of having an installation profile for each project.New Approach
After much debate, the Lightning team has decided to embrace the concept of “inheriting” profiles. AKA sub-profiles. (/throws confetti)
This is not a new idea and we owe a huge thanks to those that have contributed to the current patch and kept the issue alive for over five years. Nor is it a done deal. It still needs to get committed which, at this point, means Drupal 8.4.x.
On a technical level, this means that — similar to sub-themes — you can place the following in your own installation profile’s *.info.yml file and immediately start building a distribution (or simply a profile) on top of Lightning:base profile: name: lightning
To encourage developers to use this method, we will also be including a DrupalConsole command that interactively helps you construct a sub-profile and a script which will convert your old lightning.extend.yml file to the equivalent sub-profile.
This change will require some rearchitecting of Lightning itself. Mainly to remove the custom extension selection logic we had implements and replace it with standard dependencies.
This is all currently planned for the 2.0.5 release of lightning which is due out in mid March. Stay tuned for updates.
I have been working with Drupal for over six years now, for much of that period exclusively working with Drupal as an employee or in a freelance/contract role.
Prior to the start of Drupal 8, the nearest thing I manged to getting off of the island was periods of doing something completely different (Expression Engine, Django, a custom PHP framework, attending a Silverstripe conference), usually because of an existing or already started/inherited project connected with the Drupally people I was working with.
These sabbaticals were reasonably short, however often instructive and enjoyable, in some cases (not all) the feeling was 'could have been better done in Drupal', they were a complete break from Drupal.
I have just finished ten months of working on a Symfony based project and things are very different.Bursting the Drupal Bubble
For almost a year, I have not been doing Drupal, but also so busy and involved learning more about Symfony, Angular 2 and other technologies that I have not had time or capacity to attend Drupal events, meetups etc.
Although working with a group of people that had all used Drupal a lot before (amongst other things), Drupal was very rarely mentioned, not one person said 'this could have been done better in Drupal' in fact the only time that really came up at all was in reference to other client systems that were integrating with what we were building.
I needed to burst the bubble, forget about Drupal and whilst working on something different spend my time thinking about that.
The bubble I am referring to is thinking about everything in the context of Drupal, even the proudly invented elsewhere bits in Drupal 8. This kind of thinking can perhaps have the same bad effects as the echo chambers on social networks where everyone you are connected to has essentially the same or similar references and experiences.
There is a possibility that the next contract I do will not be Drupal either.Drupal is still there
Using Symfony and anything using Symfony components (Laravel may be next) still increases knowledge and skills related to Drupal (before 8 anything elsewhere just meant Drupal relevant knowledge was fading over time). Having used the Symfony console a lot and written commands for it, I will be pouncing on the Drupal Console when I do my next piece of Drupal work for example (which is more than I can say for Drush).
I am clearer about what I think Drupal strengths are though. I current have two personal projects I want to start/finish, one is a match for Drupal one for Symfony.Conclusion
Mileage may differ for other people, I found I had to put Drupal to one side for a while, it is all very well getting off the Island but that is not quite so effective if you remain in an Island culture ghetto somewhere on the mainland and don't fully integrate for a while. Some people will have no need to ever get off at all, we all have different stories to tell.
However if you do come across some work that would be better done in something else, will you even know? If you do know will you have the courage to burst the bubble?Add new comment