Many of us may be reluctant to consider random testing as a testing technique. But study in [1] indicates that random testing is more cost effective for many softwares.
This form of testing is useful when the time needed to implement and execute test case sequence is too long or the complexity of the problem makes it impossible to test every combination. Most of the times certain amount of Random Testing is mandatory before release which will be specified in the Release criteria.
Random testing is also known as Gorilla testing . In it we don't test the application sequentially, we just take the modules/fields randomly & perform testing whether it's functioning properly.
In this method test case/ test data is selected randomly. Even the slightest bugs can be discovered with minimal cost. It can also compete with other test techniques in terms of coverage. A hybrid approach by combing random testing with other testing techniques may yield good results.
Random Testing can imply any of the following
According to Wikipedia: A pseudorandom process is a process that appears to be random but it is not. Pseudorandom sequences typically exhibit statistical randomness while being generated by an entirely deterministic causal process. Such a process is easier to produce than a genuine random one, and has the benefit that it can be used again and again to produce exactly the same numbers, useful for testing and fixing software.
Let us write a small program and see how does pseudo-randomness work
In the below program we will try to generate a set of 50 random numbers which are in between 0 and 100.
Usage of this program
Run the Program
Analysis
If we observe the results of the above run we can see when ever the seed is the same the set of numbers are same. This is a way to create pseudo randomness where numbers are generated randomly but can repeat the same results whenever required by providing same seed.
How to use?
This behavior will be very useful for situations where reproducing the same scenarios is required.
For this purpose we can use following logic.
Put all the test cases in an Array List<String>
Create a set of Random numbers which are in between 0 and Size of Array List. Now get the elements with that index and execute tests (in TestNG) as follows
ant -Dtestcase=ArrayList[RandNumber]
For generating every set use a seed as discussed earlier and if you hit a bug with some set of tests then you can re-execute same set of tests in same order by just providing the same seed.
I hope this explains the importance of Pseudo-randomness
References:
------------------
[1] Joe W. Duran, Simeon C. Ntafos, "An Evaluation of Random Testing", IEEE Transactions on Software Engineering, Vol. SE-10, No. 4, July 1984, pp438-443.
This form of testing is useful when the time needed to implement and execute test case sequence is too long or the complexity of the problem makes it impossible to test every combination. Most of the times certain amount of Random Testing is mandatory before release which will be specified in the Release criteria.
Random testing is also known as Gorilla testing . In it we don't test the application sequentially, we just take the modules/fields randomly & perform testing whether it's functioning properly.
In this method test case/ test data is selected randomly. Even the slightest bugs can be discovered with minimal cost. It can also compete with other test techniques in terms of coverage. A hybrid approach by combing random testing with other testing techniques may yield good results.
Random Testing can imply any of the following
- Input Data generation
Example: You need to test new functionality of your application. For testing you randomly generate data for all existing and new fields in the application under test. - Selection of Test Cases
Example: You need to test new functionality of your application. For testing you randomly select test cases for the application under test.
But the biggest problem with Random Testing is the Randomness it self. "How to reproduce the bug?" is the biggest question as the steps taken are truly Random.This is the place where pseudo-randomness came to rescue.
According to Wikipedia: A pseudorandom process is a process that appears to be random but it is not. Pseudorandom sequences typically exhibit statistical randomness while being generated by an entirely deterministic causal process. Such a process is easier to produce than a genuine random one, and has the benefit that it can be used again and again to produce exactly the same numbers, useful for testing and fixing software.
Let us write a small program and see how does pseudo-randomness work
In the below program we will try to generate a set of 50 random numbers which are in between 0 and 100.
import java.util.Random;
public class PseudoRandom{
public static void main(String[] args)
throws NumberFormatException{
throws NumberFormatException{
if(args.length>0){
Long seed = Long.parseLong(args[0]);
Random pseudo = new Random( seed );
for ( int i=0; i<50; i++ )
{
// This will generate random number
// in between 0 and 100
// in between 0 and 100
int number = pseudo.nextInt( 101 );
System.out.print( number+"," );
}
System.out.println("Done...");
}else{
System.out.println("Usage: \t PseudoRandom seed");
System.out.println("\t Where seed is any number which
can be used \n\t again and again to produce
exactly the same numbers");
can be used \n\t again and again to produce
exactly the same numbers");
System.out.println("");
System.out.println("Example: PseudoRandom 284");
}
}
}
Usage of this program
java PseudoRandom
Usage: PseudoRandom seed
Where seed is any number which can be used
again and again to produce exactly the same numbers
Example: PseudoRandom 284
Run the Program
qa-by-passion:/qa# java PseudoRandom 284
2,28,50,35,1,75,6,29,49,15,44,21,62,42,26,69,2,17,34,6,98,67,15,58,69,22,90,45,16,70,64,3,72,4,41,63,62,46,37,91,35,99,4,95,67,72,100,85,68,46,Done...
qa-by-passion:/qa# java PseudoRandom 284
2,28,50,35,1,75,6,29,49,15,44,21,62,42,26,69,2,17,34,6,98,67,15,58,69,22,90,45,16,70,64,3,72,4,41,63,62,46,37,91,35,99,4,95,67,72,100,85,68,46,Done...
qa-by-passion:/qa# java PseudoRandom 284
2,28,50,35,1,75,6,29,49,15,44,21,62,42,26,69,2,17,34,6,98,67,15,58,69,22,90,45,16,70,64,3,72,4,41,63,62,46,37,91,35,99,4,95,67,72,100,85,68,46,Done...
qa-by-passion:/qa# java PseudoRandom 316
70,79,19,5,31,73,87,83,35,71,21,64,77,100,50,99,90,28,52,83,96,32,93,32,5,48,93,52,25,73,27,100,28,3,54,35,21,52,73,78,69,0,32,74,72,35,86,30,80,55,Done...
qa-by-passion:/qa# java PseudoRandom 675
19,33,4,82,93,94,31,49,2,31,4,84,11,0,97,36,25,87,75,28,3,71,96,84,3,17,50,34,86,18,29,59,15,99,78,98,4,99,88,59,23,1,49,77,74,14,55,9,75,27,Done...
qa-by-passion:/qa# java PseudoRandom 284
2,28,50,35,1,75,6,29,49,15,44,21,62,42,26,69,2,17,34,6,98,67,15,58,69,22,90,45,16,70,64,3,72,4,41,63,62,46,37,91,35,99,4,95,67,72,100,85,68,46,Done...
Analysis
If we observe the results of the above run we can see when ever the seed is the same the set of numbers are same. This is a way to create pseudo randomness where numbers are generated randomly but can repeat the same results whenever required by providing same seed.
How to use?
This behavior will be very useful for situations where reproducing the same scenarios is required.
For this purpose we can use following logic.
Put all the test cases in an Array List<String>
Create a set of Random numbers which are in between 0 and Size of Array List. Now get the elements with that index and execute tests (in TestNG) as follows
ant -Dtestcase=ArrayList[RandNumber]
For generating every set use a seed as discussed earlier and if you hit a bug with some set of tests then you can re-execute same set of tests in same order by just providing the same seed.
I hope this explains the importance of Pseudo-randomness
References:
------------------
[1] Joe W. Duran, Simeon C. Ntafos, "An Evaluation of Random Testing", IEEE Transactions on Software Engineering, Vol. SE-10, No. 4, July 1984, pp438-443.
