Drupal 9 Configuration Management Strategies

This blog post was originally published on mikemadison.net and has been republished here with permission. 

I recently posted an Introduction to Drupal 9 Configuration. If you aren’t up on the topic, I’d read that post before diving into this one!

There are many ways to manage Drupal 9’s configuration. You’ve got some options here, and while this may make your life a little more challenging—especially if you haven’t yet had time to wrap your head around configuration management at large—it also allows you to tailor your configuration management strategy to your needs.

What is a Configuration Management Strategy? 

Simply put, a configuration management strategy is the plan for your project in how you will utilize the configuration management system. Typically, the requirements of a project (single site vs. multisite, common features vs. bespoke features, etc.) will dictate which tools to bring to bear on that project. Once you’ve established the requirements and figured out which tools are needed to facilitate those requirements, you’ll need two things:

• The strategy of how you will use these tools to achieve your goals
• The workflow your developers will use to implement your strategy

Variables to Consider When Formulating a Strategy

One thing that makes Drupal architecture so complex is that often there is more than one way to do a thing. Configuration is, sadly, no different. So I can’t just tell you “always do it this way.” Because while I very commonly do it “a particular way,” I don’t always do it that way myself. So instead, I want to arm you with the decision making protocols I bring to the table when I’m designing a Configuration Management Strategy for my customers. Here are some variables to consider when gathering information.

Single Site vs. Multisite vs. Many Site 

The longer you hang around the Drupal community the more expected this discussion will become. So many of my architectural considerations and decisions these days are influenced by this question.

Single sites are just that, they are one site running a single codebase, database, and file system.

Multisites are a single codebase but separate databases and file systems (one for each “site” running in the multisite array).

Many sites are typically separate codebases, databases, and file systems that still somehow derive from a templated / distributed codebase.

As a general rule, single sites consume the configuration stored in their codebases entirely (with, perhaps, some caveats for environmental logic). Multisites and Many Sites however, are much less straightforward and by their very definition must have at least some varied config. 

Why? Because if you’re using the Core configuration management system provided by Drupal 9 and you’re doing a full configuration sync from your codebase all of your sites will end up being identical (even down to the site’s name). This is because Drupal Core provides no mechanism for ignoring or conditionally altering the configuration management in a multisite configuration. While technically it attempts to store configuration in a separate “files” directory per site, it is strongly recommended not to utilize this feature (and instead rewrite your configuration directory to be outside the docroot).

Feature Variation 

No, I’m not talking about the “Drupal Features Module” in this context. I’m talking about the commonality of features on your platform.

Here’s a quick example:

We are building a multisite codebase. We have customers that want to have a hero carousel on their sites. However, the customers do not agree on the type of hero banner. One of the customers wants a horizontally scrolling carousel with a background image and call to action. Another customer wants a vertically scrolling carousel with a background video (with no CTA).

Here we have a common feature (carousel) that is vastly different in implementation detail. We want to devise a configuration management strategy that supports this use case.

I always suggest doing some discovery to identify the features for your platform so you can analyze these types of variance. The best case scenario is to design a solution that simplifies the difference between your sites. I see it two ways (for this example):

• We build two hero carousels that do very similar things (with subtle variation).

• We build one hero carousel that has a few extra fields that allow for changing the scroll direction, background type, etc.

That second example is a more complex carousel to build, no doubt. But the end result is a carousel that can be used on both sites. And the resulting config is the same for both sites.

I’m not saying that this will work for every feature. However, when thinking about multisite / many site configuration there will always be feature variation. It’s a natural part of the game. The more you can reduce that feature variation the better!

Feature Grouping

Once you’ve identified the features, it’s time to think about how you can group them. This is a significant part of the strategy, so make sure you spend some time on this!

Typically, you’ll fall into one of four categories here:

• All sites have “basically” the same config (90-95% shared)
• Sites are grouped into profiles / distributions and sites within those groups have the same config (again, 90-95% shared)
• Each site requires individual configuration, but they still generally share the majority of their config (again, 90-9% shared)
• Each site has bespoke configuration and is entirely unique.
   

As a general rule of thumb, if you will use options 2, 3, or 4 (profile splits, site splits, or site splits without default config) I strongly urge you to only do so up to five splits. That means five sites, five profiles, etc. The reason for this is that even with a robust configuration management strategy you will still run into economy of scale problems.

One of my customers this year has a platform with roughly 100 sites, each with its own bespoke configuration (so, option 4 above). In this situation the platform suffers from unintended consequences, meaning that when one site updates something, that update frequently impacts other sites on the platform (even sometimes bringing them down).

As you scale and grow the number of splits on your platform. So, you have to scale the amount of testing anytime you change anything. Anytime you apply a security update. Anytime you add a new feature or update a module. Keeping the list relatively small (e.g. fewer than 5 sets of config) means that you can readily test these changes. Can you have 10? or 20? Sure. Can you have 100 or 1000? Absolutely. Just remember that the more you have, the harder it is to manage, test and deploy.

Tools for Implementing Configuration Management Strategies 

Putting together the details can be a lengthy and painful process. While it sounds fairly simplistic in this context you truly need to have a good understanding of the scale and complexity of your Drupal platform before you can truly lock in your configuration management strategy. At a high level, you’ll typically form a strategy around these key modules / approaches:

• Config Split
  The config split module allows you to define conditions and activate versions of configuration when those conditions are met. Typical examples are environment, profile or site splits (but you can split on anything). Config Splits are usually a part of a complete sync of config.
• Config Ignore
  The config ignore module allows you to exclude certain configuration from your syncs. This is highly valuable for certain config keys such as the site name. It is highly dangerous to “overuse” this module (e.g. ignoring the config for a view, or an entity). Used in moderation on “simple” configuration is usually safe though. Config Ignore can be used as part of either a complete or partial sync, but is usually redundant outside of a complete sync.
• “Installed” Config
  This config is bundled with a module / theme / profile and installed when that thing is installed. This approach is similar to the one used by the Features module and I commonly refer to it as a partial workflow.
   

Note: the Features module was widely used in Drupal 7 and the early days of Drupal 8. However, it's less commonly used in D8 and D9.

Examples of Configuration Management Strategies 

Complete Config Sync:

The key benefits of using a complete config sync are that:

• all environments (local, dev, ci, stage, prod, etc.) have the same config (with minimal differences)
• all config is managed via git (meaning that changes to config are tracked as code changes)
• failure in one environment typically telegraphs failure in other environments (e.g. a failure locally almost always means the same failure would occur in production)
• there is a clear, testable, upgrade path for things developed with config (e.g. views, content bundles, etc.)
   

The cons of this approach are:

• it's monolithic (meaning there's a TON of config files to manage, easily over a thousand on many platforms)
• if you need disparate config for multisite it can get out of hand rapidly
• hotfixes can become very challenging (if you change it on prod, but not in code, the next deployment will blow away your change)
   

Overview:

• all config is exported into a config sync directory (usually config/default)
• config split and config ignore are used to enhance the core config system
• drush is used to manage config imports / exports
• ideally you have automated config imports during deployments
• config imports should occur after database updates
   

Note that this strategy works for both site and profile config splits. However, it works “best” when you can have a common set of configuration (90% or greater) that is shared across “all” splits and utilize the split and/or config ignore to handle specific cases where necessary. This method breaks down rapidly if you have a large platform and each profile / site has a large quantity of unique / bespoke configuration.

Partial Config Syncs:

The key benefits of using a partial config sync are that:

• you have greater flexibility with how you "use" config (this method is best used for distributions / profiles / modules AND bespoke multisite / many site platforms
• only the config you are explicitly managing is managed (this is also a con)
• hotfixes / tweaks can be made without the full config workflow
   

The cons of this approach are:

• since much of the config is db only (and not in the file system you lose the benefits of version control and environment parity
• much more likely to have "config only in the db" (meaning it's harder to test and be certain that a local environment will behave in the same way as a production environment)
• updates are much more challenging (especially removing things) and frequently require both config changes (in bundled yaml files) AND database updates (in a module's .install file)
• despite having a smaller footprint, this is a less centralized method of config management and (I personally believe) harder to manage over the long-haul.
   

Overview:

• config is stored in module / profile / theme config directories and installed when its parent container is installed
• updates to config are made via database updates and drush
• contrib modules like config split / config ignore are usually not required (given that only the config you are explicitly "managing" is changed)
   

Note that while this strategy provides a lot of flexibility, it can frequently devolve over time into an unmanaged mess. I have seen many organizations attempt this strategy, but wind up with multisite platforms that are almost impossible to update because of the unintended consequences  of changes and development. This is why I prefer the monolithic approach of the complete configuration sync. It does force you into certain key paradigms, but those paradigms keep you out of trouble. The partials approach has no such opinionation and as a result, you can much more easily wander down a deep dark hole.

In Conclusion 

Configuration Management is a critical part of Drupal 9 (and 8) and requires forethought before diving into development. So many of the tools for your development team, continuous integration process, and deployment mechanism(s) should be tailored specifically around the configuration management strategy and workflow you plan to undertake. Ideally, if you are part of a team that does a lot of Drupal work, you will establish these tools and strategies in such a way that you can reuse them across your suite of projects / customers. Acquia has done this with Build and Launch Tools (BLT).

As long as your strategy is consistent, scales, and is repeatable, that’s the important thing. I’ve seen far too many organizations stumble on this key piece of building a Drupal platform, and I hope that these insights help you architect a future proof solution to config!