Test Data Management Vs Test Data Generation

There’s a lot of discussion these days about the need for better Test Data Management as demand increases for more test data sets with more conditions to match DevOp’s needs for better software testing while doing continuous integration.

At GenRocket we dismiss the concept of having to “manage” and store static test data. Static test data sets are normally files that are created by pruning/obfuscating production data or are files that are created by hand. I call the test data “static” because it’s not test data that is created in real time or at least immediately available before the test data is to be used.

Stored (“managed”) test data files have many limitations when it comes to using the test data for real time integration testing, full functional testing or intelligent load testing.

Here are four key drawbacks of static test data files that come from production data.

1. It is usually not conditioned test data because the data is coming from production that is not conditioned by its nature. Conditioned test data is needed for full functional testing and for intelligent load tests that test not only the front end but also the back end, business logic, database and hardware.
2. It takes too long to create test data It takes time to retrieve the data from production, prune the test data and turn the test data into usable test data sets.
3. Test data can quickly become “stale.” I talked to the head of QA for a Fortune 500 company and he said that it was taking them a couple months to build usable test data sets from production and by the time they had built the test data they were already out of date and they needed to start again.
4. It’s hard to use test data files in real time integration tests. To fully test all of your business logic and back end services you need each autonomous test to have its own test data set generated in real time with valid parent-child relationships – this is close to impossible to do with static test data files from production.

At GenRocket we avoid all the Test Data Management challenges that come with pruning and maintaining static test data files because we can quickly model and create an unlimited number of “Scenarios” that, with one or two lines of code, instantly generate just about any test data that is needed for a test. Our Scenarios can generate conditioned test data, realistic test data, test data with parent-child relationships and Scenarios can generate test data in real time using the GenRocket Real Time Engine.

Why is the Scenario approach better than the Test Data Management approach? Code can’t be fully tested without the right test data for the test and, as I explained above, the traditional approach for building test data is too slow and limited. As an example of what’s possible with GenRocket Scenarios is a QA engineer who tested all of the back end business logic of a complex software application in under 30 days using GenRocket.

How was one QA engineer able to build automated tests to fully test the back end business logic of a complex application so quickly? GenRocket did the following for the engineer:

• Generated the Scenarios that contain instructions to generate the test data
• Generated the code to load the test data
• Generated the code to save the test data in the database
• Generated the code that guarantees that the test data is loaded with valid parent-child relationships
• And generated the CRUD test cases

Can you see how much time we saved this QA engineer in setting up his tests and how we gave him the freedom to run all the integration tests he wanted to fully test his back end code? And all of this was made possible by using the GenRocket test data generation approach Vs the Test Data Management approach.

To see a video about integration testing with GenRocket click the image below: