Improving Drupal Performance with PHP Fibers

Fibers was introduced to PHP core with the 8.1 release, and some initial support for rendering Big Pipe and other render placeholders within Fibers was added to Drupal core last year.

Fibers allow suspension of code at a certain point, which can then be resumed later. While it does not allow PHP execution to happen in parallel, it allows execution while waiting for async i/o operations (http requests, database queries) to complete.

We’re now getting to the point where actually using Fibers in core to improve rendering performance is feasible.

Asynchronous database queries

The biggest improvements will come from support for asynchronous database queries. This has required the creation of a mysqli database driver which is currently RTBC. mysqli supports asynchronous database queries, which unfortunately the PDO driver for MySQL doesn’t. Once this is available, it will be possible to add support to Views and entity queries.

When a page contains a views block, the query can be sent asynchronously, Views can suspend the Fiber back to the event loop, and then the renderer can get on with rendering other blocks (which might also have their own async queries to execute). Once the loop gets back to the first block, Views will check if the query has returned, continue rendering that block, or if not suspend back to the loop to see if something else is ready to continue.

There are two ways that this results in a perceived performance improvement:

1. While waiting for async queries to return, simpler placeholders can be rendered in the meantime. This means that even a single async query on a page can result in an improvement.
2. When there are multiple views listings on a page, such as a large landing page or dashboard, each listing query can be executed asynchronously and then processed when it’s ready.

Async execution doesn’t reduce the amount of work that the page needs to do, it just allows some of it to happen while the process would otherwise be waiting. However the total request will complete quicker, and with Big Pipe it will result in more of the page rendering earlier because none of the parts will be dependent on any of the others to complete.

There are still a few steps to get to this point, but it’s getting closer.

Lazy preloading to reduce database queries altogether

While Fibers was designed to formalize async execution, I recently discovered a way to take advantage of its suspend-and-resume pattern to actually reduce the amount database queries executed on a page, without any ‘real’ asynchronous processing at all.

Drupal includes a path aliasing system, which allows aliasing of any internal path with a human-friendly version. A complex Drupal page can have dozens of links, particularly when listing entities such as content or users. Because individually looking up each alias from the database would in turn require dozens of database queries, Drupal already includes a path alias pre-load cache to try to reduce this. It records the internal paths on a page for 24 hours at a time, so that following requests can load all the aliases for these paths at once instead of one at a time, if content on the page changes during the next 24 hours, any additional paths get picked up individually until the next cache clear. When Drupal used to render every page every request, this was a huge improvement, but now with render caching and Dynamic Page Cache in Drupal core, aliases don’t need to be looked up for render cached content, so the hit rate is quite low. It also doesn’t help at all when caches are completely cold.

With Fibers, before we’re going to load a path alias from the database, we can add the path alias to a preload list, and then suspend execution. If other Fibers also reach the point of loading a path alias, they can do the same. By the time we get back to the first Fiber, we might have 3, 10, or 50 path aliases that we now know we need to load, and can load them in a single database query into memory. Those aliases are now cached in memory, ready for the subsequent Fibers to access without having to be individually looked up.

If we take a views listing with 50 items in it, each with a link to their canonical URL, by rendering each Views row within a Fiber, it’s possible to suspend the Fiber 50 times for 50 path aliases, and preload them with a single database query. If the listing contains cards with both a title and author link to 50 individual authors, this would happen twice for 50 + 50 queries, becoming 1 + 1 query instead.

If five block placeholders have one URL alias to lookup each, this will also be consolidated into a single database query, so it’s not restricted to listings, but any parts of the page that can be rendered independently and need to render links.

This is not actually doing anything asynchronously at all, instead it takes advantage of Fibers to consolidate information about multiple similar operations before they actually happen, then intercepts that process via multiple loading so that the individual operations never need to take place.

Crucially, this can happen on a completely cold cache. On the front page of the Umami demo install profile, it results in a reduction of 15 queries when caches are cold, with scope to improve further. This allows us to completely remove the path alias preload cache, conserving cache storage without losing the advantages of preloading in situations it helps with, while also reducing the number of individual path lookups on cold caches which was never possible before.

Drupal has a mature caching layer which prevents most of this work happening when caches are warm, but Fibers provides an opportunity to significantly improve performance when caches are cold too.

We can apply this concept to entity loading as well as path aliases, and there are other not-quite-async uses of Fibers in progress in Drupal core too, such as for cache warming during potential stampede situations.

While Drupal Core is making slow but steady progress to embrace this new capability in PHP, it may be that PHP gets more capabilities itself too, depending on the outcome of the True async RFC that’s currently under discussion.