Skip to main content
added 221 characters in body
Source Link
Doc Brown
  • 220.6k
  • 35
  • 410
  • 625

The sheerIn this example, there is a certain risk of having instance.multiply(a, b) producing a prime number where it should not, (maybe for a pair 1,10 it delivers 11) and then leaving out the "other assertions" section.

More general, the fact you need some condition to test on the output data is a sign that your test data generation does not behave deterministic, andso you cannot easily pick the right choice of assertions beforehand. thatThat is the main problem I see here, since non-deterministic tests have the very nasty property of not creating reproducable outcomes.

Let us take your second example: if there would be a fixed set of deterministic test data (even if it was produced with a random generator once), for each pair (a,b) of numbers it would be determinable beforehand if their product should be a prime number or not. A better test then wouldcould look like this:

 [TestCase(1,41,true)]
 [TestCase(2,3,false)]
 [TestCase(10,1,false)]
 void TestMultiplier(int a, int b, bool prodIsPrime)
 {
     Multiplier instance = new Multiplier();
     int result = instance.multiply(a, b);
     assertEqual(prodIsPrime, isPrimeNumber(result));
     if(!prodIsPrime)
     {
       // some other tests
     }
  }

In the original codeOf course, there is a certain risk of having instance.multiply(a, b) producing a prime number where it should notyou might then refactor this further and split this into two tests, (maybeone for a pair 1,10 it delivers 11)prime products and then leaving out the "other assertions" sectionone for non-primes, as @Fabio wrote.

The sheer fact you need some condition to test on the output data is a sign that your test does not behave deterministic, and that is the main problem I see here, since non-deterministic tests have the very nasty property of not creating reproducable outcomes.

Let us take your second example: if there would be a fixed set of deterministic test data (even if it was produced with a random generator once), for each pair (a,b) of numbers it would be determinable beforehand if their product should be a prime number or not. A better test then would look like this:

 [TestCase(1,41,true)]
 [TestCase(2,3,false)]
 [TestCase(10,1,false)]
 void TestMultiplier(int a, int b, bool prodIsPrime)
 {
     Multiplier instance = new Multiplier();
     int result = instance.multiply(a, b);
     assertEqual(prodIsPrime, isPrimeNumber(result));
     if(!prodIsPrime)
     {
       // some other tests
     }
  }

In the original code, there is a certain risk of having instance.multiply(a, b) producing a prime number where it should not, (maybe for a pair 1,10 it delivers 11) and then leaving out the "other assertions" section.

In this example, there is a certain risk of having instance.multiply(a, b) producing a prime number where it should not, (maybe for a pair 1,10 it delivers 11) and then leaving out the "other assertions" section.

More general, the fact you need some condition to test on the output data is a sign that your test data generation does not behave deterministic, so you cannot easily pick the right choice of assertions beforehand. That is the main problem I see here, since non-deterministic tests have the very nasty property of not creating reproducable outcomes.

Let us take your second example: if there would be a fixed set of deterministic test data (even if it was produced with a random generator once), for each pair (a,b) of numbers it would be determinable beforehand if their product should be a prime number or not. A better test then could look like this:

 [TestCase(1,41,true)]
 [TestCase(2,3,false)]
 [TestCase(10,1,false)]
 void TestMultiplier(int a, int b, bool prodIsPrime)
 {
     Multiplier instance = new Multiplier();
     int result = instance.multiply(a, b);
     assertEqual(prodIsPrime, isPrimeNumber(result));
     if(!prodIsPrime)
     {
       // some other tests
     }
  }

Of course, you might then refactor this further and split this into two tests, one for prime products and one for non-primes, as @Fabio wrote.

added 221 characters in body
Source Link
Doc Brown
  • 220.6k
  • 35
  • 410
  • 625

The sheer fact you need some condition to test on the output data is a sign that your test does not behave deterministic, and that is the main problem I see here, since non-deterministic tests have the very nasty property of not creating reproducable outcomes.

Let us take your second example: if there would be a fixed set of deterministic test data (even if it was produced with a random generator once), for each pair (a,b) of numbers it would be determinable beforehand if their product should be a prime number or not. A better test then would look like this:

 [TestCase(1,41,true)]
 [TestCase(2,3,false)]
 [TestCase(10,1,false)]
 void TestMultiplier(int a, int b, bool prodIsPrime)
 {
     Multiplier instance = new Multiplier();
     int result = instance.multiply(a, b);
     assertEqual(prodIsPrime, isPrimeNumber(result));
     if(!prodIsPrime)
     {
       // some other tests
     }
  }

In the original code, there is a certain risk of having instance.multiply(a, b) producing a prime number where it should not, (maybe for a pair 1,10 it delivers 11) and then leaving out the "other assertions" section.

The sheer fact you need some condition to test on the output data is a sign that your test does not behave deterministic, and that is the main problem I see here, since non-deterministic tests have the very nasty property of not creating reproducable outcomes.

Let us take your second example: if there would be a fixed set of deterministic test data (even if it was produced with a random generator once), for each pair (a,b) of numbers it would be determinable beforehand if their product should be a prime number or not. A better test then would look like this:

 [TestCase(1,41,true)]
 [TestCase(2,3,false)]
 [TestCase(10,1,false)]
 void TestMultiplier(int a, int b, bool prodIsPrime)
 {
     Multiplier instance = new Multiplier();
     int result = instance.multiply(a, b);
     assertEqual(prodIsPrime, isPrimeNumber(result));
     if(!prodIsPrime)
     {
       // some other tests
     }
  }

The sheer fact you need some condition to test on the output data is a sign that your test does not behave deterministic, and that is the main problem I see here, since non-deterministic tests have the very nasty property of not creating reproducable outcomes.

Let us take your second example: if there would be a fixed set of deterministic test data (even if it was produced with a random generator once), for each pair (a,b) of numbers it would be determinable beforehand if their product should be a prime number or not. A better test then would look like this:

 [TestCase(1,41,true)]
 [TestCase(2,3,false)]
 [TestCase(10,1,false)]
 void TestMultiplier(int a, int b, bool prodIsPrime)
 {
     Multiplier instance = new Multiplier();
     int result = instance.multiply(a, b);
     assertEqual(prodIsPrime, isPrimeNumber(result));
     if(!prodIsPrime)
     {
       // some other tests
     }
  }

In the original code, there is a certain risk of having instance.multiply(a, b) producing a prime number where it should not, (maybe for a pair 1,10 it delivers 11) and then leaving out the "other assertions" section.

Source Link
Doc Brown
  • 220.6k
  • 35
  • 410
  • 625

The sheer fact you need some condition to test on the output data is a sign that your test does not behave deterministic, and that is the main problem I see here, since non-deterministic tests have the very nasty property of not creating reproducable outcomes.

Let us take your second example: if there would be a fixed set of deterministic test data (even if it was produced with a random generator once), for each pair (a,b) of numbers it would be determinable beforehand if their product should be a prime number or not. A better test then would look like this:

 [TestCase(1,41,true)]
 [TestCase(2,3,false)]
 [TestCase(10,1,false)]
 void TestMultiplier(int a, int b, bool prodIsPrime)
 {
     Multiplier instance = new Multiplier();
     int result = instance.multiply(a, b);
     assertEqual(prodIsPrime, isPrimeNumber(result));
     if(!prodIsPrime)
     {
       // some other tests
     }
  }