Third & GroveThird & Grove
Jan 17, 2014 - Justin Emond

The trick to making the Acquia Purge module scale

The Acquia Purge module is a port of the Purge module designed to be more compatible on the Acquia cloud environment.

The Purge module works by issuing targeted cache clear commands to Varnish for specific pages. In conjunction with the Expire module (which tries to identify every page that needs a cache clear when various CRUD events occur like a user or node being saved), the Purge module lets you set a high cache limetime in Varnish for the site overall as Purge takes care of flushing your Varnish cache for the specific pages that need to be flushed when your data changes.

The Purge module works by queuing up cache clear requests as they happen and then working down that queue. The Acquia Purge module works great, but for a large site with loads of content the queue for purging can get very long, very quickly.

By default, the module tries to process the queue via AJAX requests from the client browser of logged in editors. This is not ideal for three reasons:

  1. The user experience is poor (as the information presented to the user is technical and confusing);
  2. The browser experience is slowed down (as the client browser is making requests); and,
  3. It’s unlikely that the user will stay on your site long enough with the browser open to make a dent in your queue.

To improve this experience and make the module scale to large sites, you need to do two things:

Step 1: Disable user cache purge reporting

Disabling “Report cache purges” will suppress the confusing messages the module outputs to the user. You will find this configuration setting in the core Drupal Performance settings page (admin/config/development/performance in Drupal 7).

Step 2: Set up a background cron process to trigger purge queue processing

The purge queue can easily grow too long for the AJAX client browser process to manage. If the queue never gets smaller than you lose all of the benefits of the module: pages you need cleared from Varnish will persist until the general Varnish cache expires.

Luckily, the developers of the Acquia Purge module provide excellent integration with Drush (yah!) and provide a command to process the queue. We typically schedule this command to run every 15 minutes via cron. The command looks something like this for an Acquia site:

 

/usr/local/bin/drush -dv -l http://www.example.com @example.prod ap-process

 

And you are done!

If your initial queue is very long, you might consider just flushing the queue via:

 

drush ap-forget