A quick and dirty intro to command-line JUnit testing (#1164)

Topics/tags: Miscellaneous, Technical

As a teacher of computer science and computer programming, I try to inculcate a wide variety of habits in my students. One of those is testing, particularly some forms of unit testing. That is, I want my students to write tests (or experiments) that help build confidence that their code achieves what it is supposed to achieve, at least at the small level (e.g., individual procedures or small sets of procedures). I know that writing good tests is hard, so I try to give my students practice. That is, nearly whenever I have students write code, I ask them to write tests or at least think about the tests they would write [1].

This semester, I’m teaching our upper-level course in algorithms. I’m also one of those folks who thinks that you best understand algorithms and data structures whe you implement them. The first assignment asks them to implement data structures they should have already encountered in data structures and algorithms, the third course in the CS major [2]. As you might guess, I asked them to write tests.

Unfortunately, not all of my colleagues feel the same way about testing that I do. And not all of my colleagues teach all the parts of Java that real programers use [3]. So some of my students have asked for guidance on how they write JUnit tests on the command line. This document contains my quick and dirty instructions for using Java and JUnit on the command line, without any additional frameworks or build systems, not even Maven or Ant.

I haven’t attempted anything close to deep guidance on the many features of JUnit or the common aspects of unit testing frameworks. I haven’t attempted to describe the unit testing habits I expect, from careful thought about edge cases to looking for ways to automate more complicated and even randomized tests, such as the test for binary search that I adapted from Jon Bentley’s book. Those are topics for another day.

Here goes.

Getting started

JUnit is a standard unit testing framework for Java. You use JUnit slightly differently on each development platform for Java. The JUnit User Guide provides instruction for most major platforms. It’s a little bit vague on command-line Java, and that’s what we will often use, so I will focus primarily on those issues.

What should you know?

A test is just a void method that is annotated as a test using @Test. A typical test method contains assertions about the expected behavior or values of a procedure. For example, in testing a square procedure, I might assert that square(4) is 16. We tend to write different test procedures when testing different aspects of our procedure or program. (At least I tend to do that; different testers have different customs.)

What can you assert? The class org.junit.jupiter.api gives the standard assertion methods. The most common ones are

• assertEquals, which checks that two expressions have the same value (or close to the same value, if you use the variant designed for real numbers);
• assertTrue, which checks that a Boolean expression returns true;
• assertAll, which checks whether a sequence of executable expressions all succeed (do not throw exceptions); and
• assertThrows, which checks whether an executable expression throws a given type.

Both assertAll and assertThrows require what Schemers often call thunks and what JUnit calls executables. An executable is an object that implements the Executable interface, providing a method called execute. Since that interface requires only one method, we can create executable objects on the fly using lambda expressions. For example, if a is an [ArrayList](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/ArrayList.html), then an executable that adds a null at index 5 might read () -> a.set(5, null).

Suppose we want to test whether adding that null throws an exception (which it should, if the ArrayList has fewer than four elements). Here’s what we might write.

  @Test
  void testInvalidAddition() {
    ArrayList<String> a = new ArrayList<String>();
    assertThrows(IndexOutOfBoundsException,
                 () -> a.set(5, null));
  } // testInvalidAddition()

Grouping tests

Of course, tests can’t live by themselves. Like most methods in Java, they need to appear in a class.

class MyArrayListTests {
  @Test
  void testInvalidAddition() {
    ArrayList<String> a = new ArrayList<String>();
    assertThrows(IndexOutOfBoundsException.class,
                 () -> a.set(5, null));
  } // testInvalidAddition()
} // class MyArrayListTests

And, as is typical in Java classes, we need to import things. When working with JUnit, we typically import the assertion methods (which are static) and the @Test annotation.

import static org.junit.jupiter.api.Assertions.assertAll;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;

import org.junit.jupiter.api.Test;

class myArrayListTests {
  // ...
} // class MyArrayListTests

Preparing to run tests

We’ll be running the standalone JUnit console. You’ll need to download the appropriate jar file from https://mvnrepository.com/artifact/org.junit.platform/junit-platform-console-standalone and store it somewhere sensible. I keep mine in ~/lib, but you can choose whatever you’d like. I’ve been using version 1.7.2, which is relatively recent.

Running tests

Once we’ve set up the tests in a class (in a .java file), we need to compile the .java file and then run it. (Note that we should compile all the .java files we’re working with.)

$ ls
MyArrayListTests.java
$ javac MyArrayListTests
... lots of errors, such as package org.junit.jupiter.api does not exist

Whoops. We need to make sure that we have the .jar file on our CLASSPATH.

$ export CLASSPATH=/path/to/junit-platform-console-standalone-1.7.2.jar:$CLASSPATH
$ ls
MyArrayListTests.java
$ javac MyArrayListTests
$ ls
MyArrayListTests.class  MyArrayListTests.java

The .class file is the compiled version.

Now we’re ready to run the tests. There’s probably a better approach than the one I use, but this seems to suffice. The -cp . flag says to look for class files in the current directory and the -c MyArrayListTests gives the name of the class.

$ java -jar /path/to/junit-platform-console-standalone-1.7.2.jar -cp . -c MyArrayListTests

Thanks for using JUnit! Support its development at https://junit.org/sponsoring

╷
├─ JUnit Jupiter ✔
│  └─ MyArrayListTests ✔
│     └─ testInvalidAddition() ✔
└─ JUnit Vintage ✔

Test run finished after 76 ms
[         3 containers found      ]
[         0 containers skipped    ]
[         3 containers started    ]
[         0 containers aborted    ]
[         3 containers successful ]
[         0 containers failed     ]
[         1 tests found           ]
[         0 tests skipped         ]
[         1 tests started         ]
[         0 tests aborted         ]
[         1 tests successful      ]
[         0 tests failed          ]

That’s about it. You should be able to figure out the rest. Send me questions if you can’t (and you’re one of my students).

Postscript: Here’s the complete file.

import java.util.ArrayList;

import static org.junit.jupiter.api.Assertions.assertAll;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;

import org.junit.jupiter.api.Test;

class MyArrayListTests {
  @Test
  void testInvalidAddition() {
    ArrayList<String> a = new ArrayList<String>();
    assertThrows(IndexOutOfBoundsException.class,
                 () -> a.set(5, null));
  } // testInvalidAddition()
} // class MyArrayListTests

[1] Strangely enough, the course in which I was worst about enforcing testing was probably CSC-324, Software Design.

[2] Officially, the course has a title something like Object-Oriented Design, Data Structures, and Algorithms. Historically, that course is called CS2 in the CS education literature. At Grinnell, we’ve inserted an extra course in the introductory sequence that does some linked data structures while getting students used to memory management before we hit them hard with ADTs, data structures, and algorithms.

[3] Admittedly, I don’t, either.

Version 1.0 released 2021-08-30.

Version 1.0.1 of 2021-08-30.