Software Testing: Reactive vs Proactive

How proactive do you want to be when it comes to fully testing your software application’s source code? It’s an interesting question that can’t easily be answered unless you can ask the greater question, “How does one quantify how proactive one is in fully testing their application’s source code?”

Measuring how much of your code you are testing

Fully tested source code is a measure of the quantity of source code within your application that is actually being tested. A deeper definition would be a measure of the quantity of complex business logic within your source code that is actually being tested.

The higher the percentage of code being tested is directly proportional to how proactive you are in fully testing your source code. Thus, the likelihood or statistical chance of a bug entering your source code and making it to your production environment when new code is created or current code is updated is the direct inverse of the percentage of code being fully tested; this is the proactive to reactive ratio.

For example, if only 10% of your code is being tested in a proactive way, then there is a 90% chance or likelihood that the rest of your source code will have minor to major bugs causing a disproportionate reaction. A disproportionate amount of time and resources will thus be given to:

• Discovering where the bugs exist
• Determining if and how much data was corrupted
• Writing code to make data corrections
• Modifying code to make business logic corrections

However, if you’re being proactive, you would be writing white box tests to further avoid being reactive to more bug fixes in your application’s source code in the future.

How to fully test your application’s source code

So how do you fully test your application’s source code? You fully test your application’s source code through white box testing. And what is white box testing? White box testing is a method of software testing that examines the internal structures or workings of an application, as opposed to its functionality (Wikipedia). Integration tests are a good example of white box testing.

Why black box testing is not enough

So why not use black box testing? Isn’t it good enough? Not really… Black box testing is a method of software testing that examines the functionality of an application without peering into its internal structures or workings (Wikipedia). Functional tests are a good example of black box testing.

You can’t fully test your source code if you’re using black box testing only; you’ll be lucky if you can fully test 10% of your source code. Fully testing your application’s source code can only come from true white box testing.

Banking application example

Take a look at the following activity diagram that describes the business logic of executing a bank transfer from a savings account to a checking account; how many tests do you see?

Let’s count them:

Test 1

• Assert savings not enabled
• Assert checking enabled

Test 2

• Assert savings enabled
• Assert checking not enabled

Test 3

• Assert savings enabled
• Assert checking enabled
• Assert monthly max transfers exceeded

Test 4

• Assert savings enabled
• Assert checking enabled
• Assert monthly max transfers not exceeded

Test 5

• Assert savings enabled
• Assert checking enabled
• Assert monthly max transfers not exceeded
• Assert amount is less than balance

Test 6

• Assert savings enabled
• Assert checking enabled
• Assert monthly max transfers not exceeded
• Assert amount is greater or equal to balance
• Assert savings is debited the value of amount
• Assert checking is credited the value of amount
• Assert a transaction history record was created
• Assert transaction history record attributes contain the correct information

For the business logic defined within this activity diagram, a minimum of six white box tests will be necessary to fully test transferring money from a savings account to a checking account.

Couldn’t I fully test that application with just black box testing?  

Could you test all six of these conditions via black box testing, most likely not.

1. You’ll most likely not have an activity diagram that gives you a view into methods business logic.
2. Even if you had an activity diagram that gives you a view into the methods business logic, it’s not reliable because you can’t be sure that other business logic exists within the method that needs to be tested without physically looking at the code.
3. Through black box testing, it would be very difficult and laborious to setup the conditions for each test; at best one would setup ideal conditions to black box the last test, test number six.

I must be fully testing with those 6 white box tests then…right?

Even if you were able to peer into the source code and write the six white box tests necessary to fully test the transfer from savings to checking business logic, is the code now fully tested? No, not completely.

In the middle of the activity diagram, there is another black-box, checking monthly max transfers exceeded, that can only truly be tested via white box testing. Do you know what the code does? Let’s take a look.

And until this black box of business logic is fully white box tested, then the opportunity for a savings to checking transfer to perform incorrectly is reactively high.

Let’s say that initially, when this code went to production, the business logic that detects when max transfers are exceeded works perfectly; and then a minor addition to the business logic is made, but the code is not fully white box tested and thus a small but fatal bug is accidentally introduced. If the fatally bugged source code is introduced into the production environment, the fallout could be catastrophic.

The danger of not fully testing

If the banking application’s source code was servicing a base of 10,000,000 users and averaging 10,000 savings to checking transfers every 60 seconds, even if the bug were to exist in the production environment for as little as one hour, it could potentially affect over 600,000 users and millions of rows of data in the production database. The amount of time and number of resources that would have to react to correcting such a catastrophe would be extensive and extremely costly.

Does this sound scary? It should scare you a little or maybe a lot. If you’re not proactively white box testing your application’s source code, it’s only a matter time before you will have to react disproportionally to the bugs that will inevitably get introduced into your production environment, wreaking havoc, costing you and your customers a lot of unnecessary pain.

This emphasizes the point of doing proactive vs reactive testing and potentially using a solution like GenRocket to accomplish full test code coverage.