Testing and Debugging

Testing and debugging are important skills that can simplify your life as a computer scientist if you do them well. Learning these skills is one of the goals of the class.

Backup:

Is the dog going to eat your program? Yes! Dogs are descended from wolves, which are predators. Predators eat things; that is their job.

You cannot prevent the dog from eating your program. But you can keep a backup. A computer scientist always keeps a backup.

Copy your files to a flash drive, and to your CS Linux account. CS backs up your Linux files every 4 hours, under .snapshot/

Testing:

Start with simple test cases first

(= x y), solve for x.
(= x y), solve for y.
(= x 3), solve for z.
(= a (+ b c)), solve for b
(= a (+ b c)), solve for c
(= a (+ b (+ c d))), solve for c
(= a (+ (+ b c) d)), solve for c
(= x (- y)), solve for y
(= x (exp y)), solve for y

Your testing should cover:
- All base cases. Do these first.
- The null case. There are lots of null values in the world. Make sure your program can handle them, even if you do not really expect null as an input.
- Simple recursive cases.
  If you get the base cases right,
  and get the recursive case(s) right,
  your function is right.
Always read your output critically (even after you think your program is working).
If you see anything wrong or funny, track it down and fix it immediately.
Never let a bug get away.
The most exciting phrase to hear in science,
the one that heralds new discoveries,
is not "Eureka" but "That's funny ... "
-- Isaac Asimov (1920 - 1992)
Test components individually. If you have a well-defined component, call it with test data and print out the results to make sure it works. Test the engine before trying to fly.

Debugging:

Don't get stuck on logical analysis.
Of course it is okay to look at your code and see if you can spot what is wrong with it.
But, if you don't find it within 5 minutes, stop and put in printouts.
If your code cannot possibly be doing what it is doing,
the bug is something you are not looking at.
You will never find it by logical analysis (since you are not looking at it),
but you will find it by putting in printouts and looking for something that is funny.
Put in printouts.
- Make them informative. Don't just print "hello"; that doesn't tell you much. Print out the exact location and variable values.
- Build printouts in as a debugging aid.
```
public static boolean debugflag = true;

Cons solve(Cons e, String v) {
    if (debugflag)
       System.out.println("entering solve" + " e = " + e + " v = " + v);
    ...
    if (debugflag)
       System.out.println("leaving solve" + " val = " + val);
    return val; }
```
- Leave printouts in permanently as a "black box" in case of a crash. Once you think your program is debugged, you can simply set:
```
public static boolean debugflag = false;
```
  but if you later encounter a bug, you can easily turn the printouts back on.
- Put in multiple debug flags to allow you to watch different aspects of your program.
Divide and Conquer the Bugs. If the arguments of some call to solve are okay, but the value on exit is not, focus on just that problem (a smaller test case) and fix it.
Don't focus on "How can I get the right answer to the hard test case?", but on "How can I find a small test case that demonstrates a small problem that I can fix?".

Common Problems:

Stack Overflow: A Stack Overflow could be caused by using a good program on data that is larger than your language's stack. Usually, though, it means you have an infinite loop. Typical causes:
- forgetting to test for null
- forgetting to do rest()
Printouts will make an infinite loop very obvious.
Segfault: A Segfault (segmentation fault) is signaled by the CPU when the program attempts to access memory that does not exist. A common cause of segfaults is trying to dereference a null pointer, i.e. ptr.field or ptr.method() when ptr is null; or an array reference out of bounds, x[i] where i is negative or too large.
INF: INF (infinity) as a floating-point value is commonly caused by dividing by 0.
NaN: NaN (Not a Number) is a value of a numeric variable that is not a valid numeric value. Some compilers initialize floating-point variables to NaN, so a NaN value often indicates failure to initialize a variable. If you just say:
```
   double sum;
```
then you may get a NaN. NaN is contagious:
```
   trash = sum + 1.0;
```
will be a NaN if sum is a NaN.
Extra Parens or Wrong Parens: Parens are just a printed reflection of the underlying structure. If the parens are wrong, it is because the structure is wrong. Don't try to "get rid of parens" (they do not actually exist), but fix the structure instead. Incorrect structure is probably caused by using the wrong access function (rest() instead of first(rest()) ) or the wrong constructor function (cons, list, append).
```
   ((the sum of) a and b)
```
is the result of:
```
   (cons '(the sum of) '(a and b))
```
when what you probably wanted was:
```
   (append '(the sum of) '(a and b))
```
which produces:
```
   (the sum of a and b)
```
cons, list, and append all have their uses; learn when to use each.
Lost Data: You say:
```
   llmergesort(lst);
```
and some of your data is gone. Did the dog eat your data? No; since llmergesort is destructive, you must re-assign the value:
```
   lst = llmergesort(lst);
```
See the slide entitled On Not Dropping the Ball.