Blue/Green deployment is one of the most underused deployment practices. However, it is a valuable process that can help reduce stress and reduce the lead time of deployments. To help you understand the benefits and get started thinking about how to implement them at your organization, we created a list of blue/green deployment best practices you should follow.

What are Blue/Green Deployments?

Blue/green deployments help organizations achieve automation and Continuous Delivery while simultaneously avoiding downtime and mitigating risk. The basic concept of a blue/green deployment is to create two identical production environments (blue and green) that work in parallel.

At any given time, only one of these production environments is live. The blue environment, for example, receives all user traffic, while the clone (green) remains idle. When a new version of an application is ready for release, it is deployed to the green environment for testing.

Once the release passes testing, the application traffic is routed from blue to green. Green then becomes the new production environment, and blue goes idle and can become the “new green” environment for the next release.

This process eliminates downtime, lowers risks, and makes it easier to release quickly and reliably.

Blue/Green Deployment Best Practices

Blue/green deployments can help reduce lead time and confusion. Every organization is different, so implementation details will vary. However, there are a few best practices to keep in mind when creating an implementation plan.

1. Seamless Switching Between Environments

Several best practices can optimize seamless switching in Blue/Green deployments:

• Utilize load balancing: Load balancing is the process of distributing workloads across multiple computing resources. Load balancers route traffic to various servers immediately, unlike DNS, which requires changes in the management interface.
• Execute a rolling update: To avoid downtime, do not switch all servers from the old version to the new one at once. Rather, add one server and retire an old server on repeat until all have been switched.
• Make sure code versions can coexist: To ensure that rolling updates are successful, different code versions need to be able to co-exist. To avoid downtime, code can be updated in “mini-releases” to ensure successful side-by-side running.
• Closely monitor your environments. It is important to monitor both the production and non-production environments. You don’t want to rush out of bed thinking there’s an emergency in the production environment when it’s actually a less-critical issue with the non-production environment. Make sure your organization sets up an easy way to toggle the alerting between the two environments.
• Automate! Automating your processes is quicker, easier, and safer than manual actions. If a repetitive task can be automated, make sure it is! This reduces the risk of human error and creates more successful deployments.

2. Follow Database Versioning Best Practices

Keeping track of your application’s database can be a challenging task. The database schema can be mismatched in different environments, or data in one of the databases may be missing a crucial piece of data. Having more than one database instance can cause the instances to fall out of sync. Luckily, the following database versioning best practices can help:

• Small changes, documented in version control: Application database and reference data should be treated as regular code. Both database schema and the corresponding reference data should be stored in source control systems.
• Separate database instances for developers: Every developer should have their own database instance. This helps avoid unnecessary collisions since oftentimes changes made by programmers are incompatible.
• Decouple schema changes from code changes: Decoupling schema changes from code changes allows the relational database to be outside of the environment boundary defined for the blue/green deployment and shared between the blue and green environments. There are two main approaches for performing schema changes:
  • The schema is changed first: the database updates must be backward compatible, so the old version of the application can still interact with the data
  • The schema is changed last: code changes in the new version of the application must be backward compatible with the old schema.

The former approach is usually additions, with the latter approach being deletions. To mitigate risk, organizations must have a strong software development lifecycle with an emphasis on the testing and QA stages.

3. Use Feature Flags

A feature flag allows developers to turn features on and off at a moment’s notice. They reduce the amount of engineering that needs to be done during feature testing, provide greater control over feature access, and facilitate the integration of customer feedback into features.

Feature flags are important to blue/green deployments because they allow developers to test hypotheses about features without disrupting a live environment. This helps organizations receive feedback from users but with the benefit of just turning off an ill-received feature.  

Benefits and Pitfalls of Feature Flags

Feature flags have numerous benefits to the software development lifecycle including:

• Reducing risk with the ability to gradually reveal a feature
• Increase in collaboration through branching in code, allowing teams to work on the code mainline instead of separate feature branches
• Segment users by turning on a feature for a subset of users
• Separating features roll out from code deployment, resulting in spending less time addressing merge conflicts

While there are several benefits to feature flags, there are some consequences to consider:

• Feature flags can add complexity to code, resulting in technical debt
• Feature flags can be difficult to scale as tracking and synchronizing multiple feature flags is challenging
• Utilizing feature flags and blue/green testing presents a cultural shift that can be hard to implement

4. Be Able to Roll Back Effortlessly to a Previous Deployment

Blue/green deployments allow organizations to easily roll back to a previously successful deployment. This is especially helpful because if anything goes wrong during a deployment in green, users can easily switch back to the previously working blue environment.

This gives developers the ability to carefully make fixes to a build without being pressured for time, as a working, live environment will still be available.

5.  Ensure Your Production Environments Remain the Same

A successful Blue/Green deployment only provides value if the production environment remains the same after the actual deployment.

Inedo’s Otter can help organizations provision their servers and manage configuration to ensure production environments remain the same. Otter helps you provision and configure your servers automatically by allowing users to define reusable sets of configurations, or “roles.” Otter also continuously monitors servers for configuration changes and reports when there’s configuration drift.

Otter is able to easily help organizations successfully complete blue/green deployments without experiencing configuration drift.

Blue/Green Deployments Increase Speed and Safety in Releases

Blue/green deployment is a necessity in any organization. With benefits such as no downtime, mitigated risk, and easy rollbacks, organizations should definitely be implementing this deployment strategy.

Inedo DevOps tools maximize developer time, minimize release risk, and empower stakeholders to bring their vision to life faster. All with the people and technology you have right now.