CSC323 2010S Software Design

Unit Testing

Purcell, Steve (n.d.). Python Unit Testing. Online document available at http://pyunit.sourceforge.net/pyunit.html.

Savoia, Alberto (2007). Beautiful Tests. Chapter 7 (pp. 85-104) in Oram & Wilson (eds), Beautiful Code. Sebastapol, CA: O'Reilly.


Questions Sam Didn't Understand

On page 94 in the first paragraph:

The code for calculateMidpoint isn't given, but I assume it is the same as in the original code (i.e. using shifting).

It's on page 93 of my book.

assertEquals(Integer.MAX_VALUE -1,
      calculateMidpoint (Integer.MAX_VALUE-2, Integer.MAX_VALUE-1)));

The midpoint should be Integer.MAX_VALUE-1.5, which has to be changed to an integer. I thought integers truncated - but in this example, the integer rounds up from 1.5 to 2. Is this caused by the shifting operation? Could we review shifting again?

I don't see the same assertion that you gave. You are correct that integer division truncates. Without a formal definition of calculateMidpoint, it's hard to know whether or not rounding up would be acceptable. (The tests suggest that it's not.)

Short Questions

How does GUI Testing work in PyUnit? Is it only for testing GUI functions such as this widget maker, or does it provide a visual interface for running test suites?

It's a visual interface for running test suites. I don't think I managed to get it installed.

Savoia writes that "we still have a long way to go before the majority of developers are test-infected" (BC 89). Do most program developers leave the testing of their programs to someone else, or am I reading that incorrectly?

Many programmers don't even think about testing at the unit level.

This seems pretty straightforward. Do many Python programmers include a 'pyku' or two with their code?

I have no idea.

Can you explain the function of the @ in java? Specifically on page 90 what does it mean to annotate a method with @org.junit.test?

Traditionally, @ text is an annotation, meant for the reader, for some kinds of processors, and for introspection. But I don't see the particular annotation you mention.

Clarification

On page 89, the code incorporates the 'bit shift operator'. Could you explain a bit more (HA! PUN!) about how this works, and what it exactly does? Also, doesn't the programmer still need to divide (low + high) by two in order to achieve the midpoint, even if they are using the '>>>'?

Could you please explain the reason for adding theories 4 and 5? Why does he need to check whether the target is in the array if -1 is returned when he has already tested whether -1 is returned when element is not in the array?

Those are different orders of implication.

What are Set-up and Tear-down functions?

The construction and destruction of the fixtures.

What are fixtures?

Any data that you need to prepare before a series of tests.

Critiques

The author's bulk testing involves using a large number of randomly generated arrays. Why doesn't he just use a more systematic approach to testing like we did for binary search and get rid of any randomness?

I don't understand why the author in Beautiful Code uses random tests (admittedly a very large number of random tests). Your use of systematically changing the size and contents of arrays (or whatever you happen to be checking) seems like a much more reliable method of determining the accuracy of code. It would also make it easier to recognize patterns of bugs.

In the Beautiful Code reading, the author goes to the trouble of considering the logical underpinnings of his testing methodology, but instead of systematically trying every position in arrays of different sizes, he hopes that he will find randomly the one case where his implementation of binary search doesn't work. Isn't this a bit sloppy?

Those are good questions. I'm not sure. Some folks seem to have more confidence in lots of random testing than in systematic testing.

He also worries about the O(n) runtime of his 'guaranteed' search method. Couldn't he have allocated a sized array, chosen and filled a random place, and then filled the array around it?

That's still O(n).

Is it really necessary to run millions of tests to be confident that binary search works, or am I correct in interpreting that as overkill for the sake of demonstration?

Were you there for the day I tested binary search?

Miscellaneous

How do annotations work in Java? I was trying to read the Java documentation for them, but I only got more confused.

In test case methods in python, strings placed directly after the method declaration are displayed when the test is run. This is in contrast to the use of strings directly after most method declaratons, in which the string is only a comment. This seems like an unnecessary deviation from convention, or at least difficult to implement as opposed to having a "display" method to serve the same purpose.

The tests described in Beautiful Code are fine, but how do you test output that can only be generated by hand? What is the proper way to test a program of unique function that performs a process painful enough that people don't do it?

Disclaimer: I usually create these pages on the fly, which means that I rarely proofread them and they may contain bad grammar and incorrect details. It also means that I tend to update them regularly (see the history for more details). Feel free to contact me with any suggestions for changes.

This document was generated by Siteweaver on Thu Feb 25 12:53:25 2010.
The source to the document was last modified on Thu Feb 25 12:53:21 2010.
This document may be found at http://www.cs.grinnell.edu/~rebelsky/Courses/CSC323/2010S/Readings/unit-testing.html.

You may wish to validate this document's HTML ; Valid CSS! ; Creative Commons License

Samuel A. Rebelsky, rebelsky@grinnell.edu

Copyright © 2010 Samuel A. Rebelsky. This work is licensed under a Creative Commons Attribution-NonCommercial 2.5 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/2.5/ or send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA.