The Role of Quality Assurance in DevOps
by Marisa Vesel, on Nov 12, 2019 1:15:00 PM
Automated testing is an absolute best practice for increasing organizations’ release cadence and code quality. However, despite its central role in DevOps and the importance of moving releases through the CI/CD pipeline as quickly as possible, automated testing has its limits. In fact, it doesn’t make humans redundant in development or operations processes. The reason is quality assurance testing, which ensures no customer experience issues accidentally ship to production.
Integrating human QA throughout your DevOps processes will ensure that your end-user experience is of the highest quality every time. As we’ll see below, without it, even tech giants like Google end up making embarrassing errors.
Why is QA Integral to DevOps?
DevOps is a cultural shift combining traditionally siloed Development and Operations teams. Additionally, in a DevOps environment, testers and developers have equally important roles and responsibilities. Oftentimes in DevOps, the boundaries between a developer and tester become blurred. Quality assurance (or QA) helps increase the collaboration between Development and Operations teams by encouraging everyone in the organization to take responsibility for quality and stability.
In companies with a DevOps mentality, developers will begin to think more like testers and begin checking their own code for potential errors. And testers, who are responsible for testing applications and software, will begin to fix the code. This overlap helps organizations meet their goals for accelerated speed of delivery.
Overall, integrating QA into the development process helps lead to better-quality releases. Furthermore, continual testing throughout the process ensures high-quality releases even at a high velocity. Having developers and QA collaborate on testing and fixing code removes silos and gives developers visibility into the customer.
Below, we take a deeper look into the how and the why of human quality assurance in DevOps.
Getting Started with QA in a DevOps Scenario
Here are some quick tips for getting started with integrating QA into your DevOps environment:
- QA and testing teams should be part of technical teams that focus their efforts on automation and testing strategies.
- Quality needs to be well defined so that code meets requirements to provide a satisfactory user experience.
- Create metrics to assess quality.
- Development, Operations, and QA teams should be encouraged by upper-level management to communicate, collaborate, and optimize their efforts.
- Continuous Integration is vital to identifying defects early on in the development lifecycle; integrating work throughout each day ensures successful implementation of code.
Why Do You Need QA For Continuous Delivery?
Automated testing is a great DevOps first step to identify and resolve bugs in code before applications are sent to production. However, automated tests can’t catch all the bugs a human user might experience.
Just because a code passes automated testing doesn’t mean its experience does. Even large-scale, well-established companies struggle with quality in Continuous Delivery because the human and machine roles don’t collaborate enough on QA. Google recently had their own “dancing purple pony” visual bug. Their Google Consumer Survey team pushed a build to production that contained an obvious bug, which they missed before going live.
But didn’t it pass the tests? Absolutely! All tests were successful, staging looked correct, and their QA checklist was complete. All these checks and not a single report of the “dancing purple pony” right on the page. The problem? An incorrect CSS class name. Developers had failed to consider how a customer would perceive the application, and unfortunately, missed the obvious. Had human tests been part of the QA process, QA testers couldn’t fail to miss this whimsical bug.
Recently, the trends is to de-emphasize the need for QA within Continuous Delivery. As automation continues to rise in popularity, more and more teams tend to just take those results as truth. However, human QA still serves its purpose in DevOps:
- Automation and Continuous Delivery practices cannot always grasp the malleable aspects of a project. Application components such as UI and mobile experience present several uncontrolled factors, which automated tests may not be able to handle. Human testers can evaluate and provide feedback on these elements when technology cannot.
- Customer experience and goals should be the highest priority, which QA does by ensuring peak software quality performance.
Integrating QA into DevOps keeps customers’ needs a priority, ensuring only high-quality software releases.
How to Add Human QA into DevOps Practices
In an ideal DevOps world, human judgement is combined with automation, leading to better business decisions and the production of higher-quality software. There are three main solutions for integrating human-driven QA and DevOps, explained here using a theatre analogy:
- Auditions: Tests that occur during the “audition” solution are part of a thorough pull request process. This testing looks for any edge cases and ensures proper system coverage by unit and integration testing, and are completed before substantial new features are merged into the master code base.
- Rehearsals: Testing during the “rehearsal” solution often occurs the day before the scheduled release. These tests have a wide range, focusing on covering operating systems, devices, and browsers. This solution is used by release teams following a disciplined and time-driven release cycle.
- Tech Week: The “tech week” solution is utilized by teams that release at-will. This test occurs regularly each week to gain insight into drifts that may have occurred in the code. During this solution, testers find hotspots that automated tests have failed to discover, which can help guide functional testing.
While automation is great, some human intervention is still needed in a DevOps environment to ensure that the software released meets customers’ highest expectations.
Inedo DevOps tools maximize developer time, minimize release risk, and empower stakeholders to bring their vision to life faster. All with the people and technology you have right now. To get help streamlining your CI/CD processes, contact firstname.lastname@example.org.