Published July 30, 2023
by Doug Klugh

Bonus Testing

What if I told you that you can test your software in production?  Well, you can with the right testing strategy and correct deployment methods.  A DevOps approach to delivery will enable your team to conduct limited manual testing of new functionality, extensions, and patches in a production environment before releasing to your customers.  Although it should never replace the unit, service, and system-level testing in your lower environments, it can help to ensure that your software will run as expected in production.

Deployment Methods

Following are deployment methods that provide three different ways of mitigating risks associated with releasing new software versions by testing new features, enhancements, and patches in a production environment.

Blue/Green Deploymentlaunch

Blue/Green Deployments enable teams to deploy software to an inactive production (green) environment in order to test on that production platform.  Once testing is complete and the software is ready to be released, your team will simply redirect traffic from your live production (blue) environment over to the green environment.  This can be done during normal business hours with zero downtime and will help to mitigate risks associated with production releases while greatly reducing deployment time.

Ring Deploymentlaunch

Ring Deployments are used to gradually release to select groups of users.  Starting with small groups of low-risk users, then gradually expanding the user base to include higher risk users, your team can limit the blast radius in the event of something going wrong and contain the impact to your most loyal customers.  Be sure to assign users that are most tolerant to failure to the inner most rings, then assign higher-risk users as you move outward.  For example, you could assign innovators to the inner ring, early adopters to the second ring, then all other users to the third, most outer ring.  Of course, if you need to split your users further, you can always add more rings.  This method not only provides an opportunity to test new features in a production environment, but also enables your team to gather telemetry, gauge user response, and attain customer feedback.

Canary Releaselaunch

Canary Releases manage the risk of releasing new software by first releasing to a small user population, then gradually rolling out to the entire user community.  If problems are detected, users can be quickly re-routed back to the old version.  In addition to functional testing, this can be used to facilitate capacity testing in a production environment with a dependable rollback strategy.  If quality is not an issue, canaries can be used for A/B testing of hypotheses of user engagement.  Canary Releases should not be used on critical systems that cannot tolerate failure.

Related Posts

assistant Development Tip

Always Releasable

Promote continuous delivery by always keeping your software in a releasable state.  Gain high confidence that your software works by integrating changes continuously, while leveraging a deployment pipeline to ensure that your software build is always releasable.  Do not isolate new development within feature branches.  A better way is to integrate those changes earlier by applying them directly on the trunk with a mature suite of tests to protect the build.  Employ DevOps patterns to always keep your software releasable in the midst of constant change.

Learn More
assistant Development Tip

Dark Launch

Decouple your deployments from your releases by deploying your software to your production environment while disabling/hiding the functionality using Feature Toggles.  Those new features can then be quickly released (turned on) at any time without any additional deployments or downtime.  This is especially useful when employing Trunk-Based Development (TBD) as some features may depend on others that are not yet fully baked.

Learn More
assistant Development Tip

Feature Toggles

Decouple feature releases from code deployments by encapsulating features and infrastructure changes inside switchable software modules.  Facilitate continuous delivery by deploying code with isolated features that can be released on-demand, enable A/B testing, support operations with automated circuit breakers and kill switches, and provide customized, dynamic feature access to any level of functionality — making Dark Launches, Ring Deployments and Canary Releases a cinch.

Learn More