Software Testing Needs Test Data Generation
While talking to some colleagues via IRC today, we were discussing funny pictures related to bugs in software. As we all laughed and shared different images, I started thinking. Have we become too accepting of bugs in our applications? I often hear things like “Every system is going to have bugs” and “No application is bug free.” Are these true statements? Absolutely. Do they have to be? Maybe not.
For most software engineers (myself included at one time), testing software was a burden. We don’t have time to test. This product has to be ready now. VC is pressuring us to deliver. These are really just excuses. Testing software has gotten a lot more advanced in the last decade. Tools like Spock, Geb, Selenium, Robotium, any many others have made testing fun and enjoyable. But no tool is enough all on its own. Complete software testing requires unit tests, integration tests, functional tests, acceptance testing, automation, QA cycles, reporting, and dare I say, most importantly, test data. Ah yes. Test data. The Everest of software testing. The only way a software engineer can truly test 100% of her code is to have enough test data to cover all the testing scenarios and the best way to get test data is to generate it. Software engineers need a tool that allows them to manage all their different testing scenarios without being burdened with mass amounts of data storage and clumsy proprietary scripts that Bob wrote, never documented, and then ended up leaving the company.
Software engineers need to be able to easily share models of required data, create their own scenarios, and choose how they want that data to be generated. The solution should be affordable because, as a manager, you want all your engineers testing. Not just the one “test data manager” because that’s all that can be afforded.
Software engineers need to be able to import existing data models and schemas. They need to be able to version testing projects so that when a release is delivered, they can immediately begin testing the next version but still be able to regression test the previous release. And they need the ability to generate realistic test data, patterned test data, negative test data and condition that data so they get the results they need for their tests.
The images posted above are very funny. And there is truth in that humor. But we’re too forgiving and too accepting of software bugs. Primarily because we get blocked when we can’t get the data we need for our tests. Test data generation can help you get unblocked and allow you to think about the bug instead of how you are going to test for the bug.
