sUnit and should:/shouldnt: vs. assert:/deny:

[Update]Be sure to read the comments if you’re interested in the topic. There’s a lot of interesting info to find there[/Update]

Last month I’ve given a fast-start Smalltalk training session to a new team member at a Customer’s project and since that person was an experienced developer we not only covered Smalltalk basics but also some special areas like sUnit. As I’ve mentioned before, I found a few errors in my material and I am currently working on corrections and extensions of the slides. So now I am taking a fresh look at sUnit – especially the variants of #should: – and found a few interesting things that I’d like to share with whoever is interested.

Let’s kill should:!

Let me first explain what should: is good for. In essence, it is the same as assert:, but it accepts a zero-argument Block. So the following two assertion statements are completely equivalent:

self assert: 1 < 3. self should: [1 < 3].

The only difference between the two is that assert: expects a Boolean parameter and should: expects a Block which it will evaluate before delegating to assert:. The same is true for deny: and shouldnt:.

[Update] To be a bit more precise: assert: means “I expect this expression to be true” while should: means “I expect this Block to evaluate to true”. In effect, both means the same thing: the result of the parameter is expected to be true.

So my first thought here is: why not eliminate should: completely and implement assert: in a way that it can work with both Booleans and Blocks. While this sounds complicated, it really isn’t: since in many Smalltalk dialects there is an implementation of value in Object (so that Objects return themselves and Blocks the result of their evaluation*), the only thing to do is to send value to the parameter of assert:.

[Update] The Implementation of TestAsserter>>should: is in fact nothing more than

should: aBlock  self assert: aBlock value

While assert: looks like this:

assert: aBoolean aBoolean ifFalse: [self logFailure: 'Assertion failed'. TestResult failure sunitSignalWith: 'Assertion failed'].

In the end, the implementation of our new assert: is easy:

assert: aBooleanOrBlock aBooleanOrBlock value ifFalse: [self logFailure: 'Assertion failed'. TestResult failure sunitSignalWith: 'Assertion failed'].


This would mean there’s no need to memorize a different method for making assertions about Blocks. We could simply write

self assert: [a < b].

So let’s move on to should:raise: and kill it also

The idea of should:raise: is to formulate an assertion about whether a piece of code will raise a certain exception. Like this:
self should: [5/0] raise: ZeroDivide
description: 'Division by zero was not signalled'.

This will work like a Clockwork: The exception is raised, so the test case will pass, because we expected it to.

My first critique here is again the naming: why not call it assert:raises: instead of should:raise:? I find should:raise: is ambiguous. Does it mean the Block should run and not raise an exception (because #should: means the Block is expected to evaluate to true, and therefor run correctly), or does it mean evaluating the Block should raise the exception? In fact the latter is the case, but I find it hard to memorise.
A second argument for assert:raises: would be that it is in line with method names like assert:equals: as they were backported from jUnit.
Together with the above-mentioned value trick, this would mean that you can even test every piece of code (not only Blocks) whether it raises an exception or not. The following two assertions would be equivalent, while there is currently no way of achieving the second one:

self assert: [5/0] raises: ZeroDivide description: 'Division by zero was not signalled'.

self assert: (5/0) raises: ZeroDivide description: 'Division by zero was not signalled'.

[Update]Giving this an even deeper look, there still is somethng wrong with all this should: stuff. While should: means “I expect this Block to evaluate to true”, should:raise: means “This Block should raise the following exception, no matter what it evaluates to if it doesn’t”. So the meaning of should: and should:raise: is far more different than their names suggests. Not sure what exactly that means for my naming idea yet, but I’m still meditating on it ;-).
Also, I am not yet decided if I like assert:doesntRaise: more than deny:raises: because the latter one suggests I want the parameter (namely Block or expression) to be false. Well, I guess I like the *doesntRaise: variant more…[/Update]

So, back to shouldnt:raise:: I found a bug!

Last, I found out that while should:raise: can test whether an expected exception gets thrown, the opposite case is only partially testable:

self shouldnt: [5/1] raise:  ExError description: 'An Error was signalled unexpectedly'.

Can test whether an exception is not thrown. Unfortunately, this only works if no exception is thrown at all in the Block. The case of an explicit exclusion of an exception that should not be thrown by a Block is not covered by sUnit. What I mean is that I want to formulate an assertion that means something like

No matter what kind of exception this piece of code may throw, I want to be sure it is not an exception of Kind X.

"This doesn't work!!! - results in an error instead of passing" 
self shouldnt: [5/0] raise:  ExHalt description: 'A Halt was signalled unexpectedly'.

I’d be interested in hearing what other sUnit users think about my suggestion of replacing should: with assert: and whether there are any reasons for keeping them that I haven’t thought of. Also I’d like to learn what others think about the “Exception Exclusion bug”.

*) The implementation of value in Object is a brilliant example of how useful polymorphism can be. I like the value trick very much and find it extremely helpful in making object protocols lightweight and elegant (just like in this example of sUnit). Being able to pass either an object or a Block as a parameter into a message without a performance penalty or code overhead other than sending value to a method argument is extremely powerful. The whole topic is probably worth a blog post on itself…

Please note: the code examples were tried on VA Smalltalk 8.0.

Another note: I’ve been wanting to post these ideas to some forum that is read by Smalltalkers of all flavors, but since I think the relevance of comp.lang.smalltalk is somewhat extinct (and my carrier even won’t let me use Usenet any more) and I know of no suitable mailing list for the topic, It’s only here…