To fix the situation I started wondering if there was a JUnit version of the TODO blocks in Perl's Test::More. The problem is that JUnit can only report tests as successful or failing (or erroneous if they throw an exception), there is no way to easily mark test failures as TODOs. Googling around and asking the Jackrabbit mailing list produced some workarounds:
- Use a system property to determine whether to perform or skip the known issue test cases.
- Put the known issues tests in separate test case classes and exclude them from the test suite.
- Use a JUnit addon to ignore marked test cases as explained in an article that discusses this same issue.
- Use an alternative test framework like TestNG, that has this functionality built-in.
I didn't want to start changing the entire test suite or even tweaking the build environment, so the last two options were out. I also wanted to make the setup easily configurable so a developer can selectively enable testing for a known issue, thus the first alternative of using a system property looks like the best solution. It seems that the Apache OJB project has reached the same solution.