we have a custom junit runner which we recently discovered some bugs in that made it crash while running the JUnit test. Unfortunately, the ant build we use on our jenkins server did not "get the message" and still said the build was successful. After digging in the code of both JUnit and the junit-related parts of ant, I can't figure out how to indicate that the test framework crashed. That is, my tests are passing so the org.junit.runner.Result looks sort of ok, but in fact the test runner itself crashed before starting some test classes so I'm missing results. Doing System.exit(1) still doesn't seem to make the ant build fail.
I can't see any special handling of crashes in the standard junit runner either, but perhaps I've missed something.