Is Your QA Team Getting the Full Value from Test Automation?

The 2018-19 edition of the World Quality Report contains comprehensive research on the key trends shaping quality assurance and testing. The report examines the operational benefits realized by QA teams from their use of test automation tools. Based on a survey of 1700 QA professionals across 10 different industry sectors and 32 countries, the study identified the many ways test automation brings value to their organizations.

QA professionals realized several benefits from test automation:

• Better test coverage
• Better control and transparency of test operations
• Better reuse of test cases
• Reduction of test cycle time
• Better detection of defects
• Reduction of test costs

As you can see from the research results in the chart below, the distribution of benefits reported by survey respondents is fairly even. And the trends from 2016 through 2108 are all going in a positive direction. Test automation is a clear path to a more efficient QA environment.

The results identified “better test coverage” as the leading benefit realized by QA teams. However, this raises an interesting question: How much did test coverage actually improve as a result of automation. Did test coverage improve only slightly and deliver a marginal benefit, or did test automation enable full coverage and deliver maximum benefit? The research doesn’t provide a clear answer to this question, but when asked about the benefits they failed to realize, 36% of respondents identified “better detection of defects”. If QA teams are struggling to improve the detection of defects by their automated tests it’s hard to imagine that they are getting full value from their use of automation.

Full coverage is an elusive goal for many QA organizations and achieving that goal requires the right combination of test tools, comprehensive test data, and a testing strategy that optimizes the use of both during test operations. The challenge of test automation is how to fully and successfully deploy the technology to realize its full potential.

The World Quality Report breaks down the penetration of test automation into various test categories. In a comparison of automation levels between 2018 and 2017, the report shows that no testing category had a proportion of test cases automated to a level greater than 18%, with very little improvement from 2017 to 2018.

The message from the market research is clear, QA leaders are fully committed to test automation as the best long-term strategy for improving test coverage and maximizing the quality of code released to production. However, they are struggling to transition from early adoption of the technology to its full deployment.

Realizing the Full Value of Test Automation

Optimizing the value of automation requires the successful integration of CI/CD pipelines to manage accelerated release frequencies, test automation tools to keep up with continuous delivery and Test Data Generation (TDG) technology to accelerate the provisioning of test data. The secret to quality at speed lies at the intersection of these three technologies. If one is omitted, the level of quality and operational efficiency will fall short of expectations.

When CI/CD pipelines are combined with test automation tools and TDG technology, the full synergy of test automation is achieved.

GenRocket is a technology leader and thought-leader in this space. Its TDG platform is seamlessly integrated with CI/CD pipelines and test automation tools enabling seamless automation of software testing and test data generation.

These complementary technologies help organizations to realize the full potential of automation with fully integrated platforms that accelerate software testing to keep pace with the shorter release cycles of software development.

The following six steps summarize the keys for creating synergy between CI/CD pipelines, test automation and test data generation:

1. Integrate TDG with CI/CD and Test Automation
   Establish a fully integrated test automation environment that includes your CI/CD server (e.g., Jenkins), your testing framework and tool set (e.g., Selenium), and your test data generation platform (e.g., GenRocket). Start with a Proof of Concept, select appropriate use cases that will create quick wins, and map out your test automation deployment strategy. GenRocket has a structured POC program that will assist you in getting started and ensuring a successful experience with this new and powerful technology combination.
2. Enable Self-Service Provisioning
   Identify some functional, performance and regression test combinations and provide testers with access to the integrated test automation environment. Ensure they are trained in the use of productivity features such as Test Data Suites (a scriptless wizard for Test Data Scenario creation). The nature of the test data that can be generated through Test Data Suites is limited only by the tester’s imagination. GenRocket enables a self-service alternative to the centralized model associated with traditional Test Data Management systems.
3. Facilitate Collaborative Test Design
   Encourage testers to share their knowledge of test case and test data designs. Take advantage of shared repositories for reusable test cases and test data scenarios. This provides testers with ready access to pre-built tests with on-demand test data generation. Where possible, encourage testers to reuse and repurpose their test case/test data designs for multiple test operations.
4. Phase Out Production Data
   Transition the use of production data to Real-Time Synthetic Test Data generated by the TDG platform. This approach allows you to phase out the use of sensitive customer data, improves test data quality and accelerates the test data provisioning process. It also reduces the bottleneck associated with centralized provisioning and allows the QA team to directly control the nature of the test data used for testing. Gradually increase the use of real-time synthetic test data to 90% of the data required for testing.
5. Eliminate Data Masking
   Prioritize your transition from production data to real-time synthetic test data by isolating Personally Identifiable Information (PII) and thereby eliminating the requirement for data masking. This saves time and eliminates the cost and complexity of traditional Test Data Management systems. At the same time, this step removes the risk of a data privacy breach while ensuring compliance with all data privacy laws.
6. Implement TDG Best Practices
   Stay current on the best practices for test case and test data design methods. Utilize GenRocket University, an online educational resource, to advance the skill level of your testing staff. Consult with GenRocket Certified Partners for the most effective and efficient way to deploy and operate the TDG platform. This will streamline the process of scaling up your test automation environment as you maximize the benefits of test automation as well as your return on investment.

If you would like to learn more about integrating GenRocket’s TDG technology with CI/CD pipelines and test automation tools, view our Solutions Sales Training Video