July 29, 2020

Test Data Design Should be Integral to Agile and DevOps

Agile adoption is in full swing. According to the most recent State of Agile Report, it’s being adopted by 95% of organizations. That’s not to say it’s easy to accomplish an Agile transformation. In fact, 77% of organizations are still in the process of implementing its methodologies as they struggle with cultural, organizational and technological change.

Agile Adoption in Full Swing

Agile being adopted by 95% of oraganizations
77% are in the process of implementing
Scrum is the dominant Agile framework
90% says DevOps transformation is important

And driven by a plethora of digital transformation initiatives, most organizations have also transitioned to DevOps to accelerate the pace of development as they shift away from monolithic software applications to a more flexible, adaptable and modular release pipeline.

Scrum has become the dominant Agile framework along with its use of 2 to 4-week sprints to drive continuous development of software features. The whole process is meant to function as a well-oiled machine, and if everything falls into place, it often does.

SPRINTS Drive Continuous Development of Software Features

However, the test data needed for delivering QA at the speed of Agile is often the monkey wrench thrown into this well-oiled machine. Provisioning and refreshing production data adds days to the process. And the use of prod data introduces complications like the need to subset data sets to match test requirements and the masking of sensitive customer information to ensure compliance with privacy laws.

Many QA teams have resorted to manually creating some or all of the data they need in spreadsheets to get the required test data in a shorter span of time. The problem with spreadsheet data is its limitations; it falls short of enabling a full data-driven testing approach that encompasses positive, negative, edge case and combinatorial test variations. And with spreadsheets, it’s nearly impossible to simulate multi-dimensional data models that involve database tables having parent-child-sibling relationships.

That’s why GenRocket’s Test Data Automation (TDA) approach and new Self Service platform makes so much sense for Agile. It puts complete control over the volume, variety and veracity of test data directly in the hands of the tester.

With TDA, test data is designed along with the test case as a unified process.Let’s look a bit more closely at the Agile process to see how and where GenRocket fits in. During each Agile sprint, a team composed of a product owner, business analyst, software developer and Agile tester meet to produce a new software feature from the product backlog. Together, they perform the 3 essential steps that are critical to QA as illustrated below:

In Agile, user stories are the building blocks for developing individual software components and ensuring everyone on the team is aligned with regard to the business value of the feature and the acceptance criteria for its release to production. It’s the job of QA to translate that acceptance criteria into test cases that validate the functionality and performance of each feature.

GenRocket can be integral to the process of Agile test case design. As user stories and their acceptance criteria become well-defined, the test case objectives become very clear. So do the requirements for the test data needed to validate test case assertions. This is a perfect time to design and configure the test data along with the test case needed for automated test execution. Why stop this train to go hunting for production data or to manually create spreadsheet data. Just configure and generate real-time synthetic test data with the data variations required to maximize coverage. If needed, query and blend in production data to ensure the accuracy of key data values. Or add rules to control the nature of the generated data in response to code logic and its behavior. Or simulate complex data feeds in multiple data formats to conduct integration testing. The test data for any of these scenarios can be throttled up or down to simulate the load and traffic conditions expected in the real world.

GenRocket can be Integral to the Process of Agile Test Case Design

Test data just might be the most critical element of any test case. It controls the input values that drive the behavior of the code. It can be purposefully designed with combinations and permutations to stress code in ways that will uncover more defects than data derived from production sources or spreadsheets. With GenRocket, testers configure the precise data needed for a given test scenario and generate it on-demand, as the test is run.

The graphic below outlines the typical steps followed by a user of GenRocket’s TDA platform. We call it the GenRocket Methodology. It’s a step by step process that starts by defining the data model to be used for generating test data. Then the tester selects the precise data generators needed for testing (there are over 260 data generators to choose from) and the data receiver that formats the test data in a desired output format.

Then the tester simply configures the variety and volume of data needed for testing using one or more self-service modules in the GenRocket platform. The configuration is captured as a Test Data Scenario that can be called by a test automation script during test execution.

High-Level GenRocket Methodology

This methodology can be followed to enable self-service test data provisioning for all levels of Agile testing. At the Unit Testing level, developers and testers can rapidly provision test data on-demand to quickly validate code. At the Service Testing level, the QA staff can test APIs and accelerate the process of continuous integration. At the UI Testing level, complex system testing is made more efficient and effective by directly testing the backend business logic and database functions without relying on frontend user interactions.