My
quest for a way to handle known issues as
JUnit tests seemed already
finished, when Marcel Reutegger, a committer of the
Apache Jackrabbit project and a developer of the
Technology Compatibility Kit (TCK) of
JSR 170, displayed some serious JUnit-fu by
pointing to the
TestResult class in JUnit.
It turns out that JUnit creates a TestResult instance for each test being run and uses that instance to store the test results. It is possible to customize the TestResult being used by overriding the
TestCase.createResult() method. You can then decide in
TestResult.addFailure(Test, AssertionFailedError) and
TestResult.addError(Test, Throwable) whether to skip some failure or error reports. This is what we ended up doing in Jackrabbit.
Digging deeper along these lines I found out that you could actually implement similar functionality also directly in a
TestCase subclass, thus avoiding the need to override TestCase.createResult(). The best way to do this is to override the
TestCase.runBare() method that gets invoked by TestResult to run the actual test sequence. The customized method can check whether to skip the test and just return without doing anything in such cases.
I implemented this solution as a generic JUnit 3.x
ExcludableTestCase class, that you are free to copy and use under the
Apache License, version 2.0. The class uses system properties named
junit.excludes and
junit.includes to determine whether a test should be excluded from the test run. Normally all tests are included, but a test can be excluded by including an
identifier of the test in the
junit.excludes system property. An exclusion can also be cancelled by including a test identifer in the
junit.includes system property. Both system properties can contain multiple whitespace-separated identifiers. See the
ExcludableTestCase javadocs for more details.
You can use this class by subclassing your test cases from ExcludableTestCase instead of directly from TestCase:
package my.test.package;
public class MyTestCase extends ExcludableTestCase {
public void testSomething() {
// your test code
}
}
You can then exclude the test case with
-Djunit.excludes=my.test.package,
-Djunit.excludes=MyTestCase, or
-Djunit.excludes=testSomething or a combination of these identifiers. If you've for example excluded all tests in
my.test.package, you can selectively enable this test class with
-Djunit.includes=MyTestCase.You can also add a custom identifiers to your test cases. For example, if your test case was written for Bugzilla issue
#123, you can identify the test in the constructor like this:
public MyTestCase(String name) {
super(name);
addIdentifier("#123");
}Then you can exclude tests for this issue with
-Djunit.excludes=#123.