What does it do?
It simply keeps the text that were defined in test assertions like assert:description: stored in an individual TestCase when the test fails, or the name of an exception thrown when a TestCase ended in an error. This is achieved with an additional instance variable in TestCase and a tiny change to TestCase>>performTest.
What is that good for?
One thing that I’ve been doing for quite a while now was helping Smalltalk Legacy projects to get back to the speed and agility that Smalltalk enables, but these projects never reached due to their corporate state of being legacy and about to be switched off any time soon now. Knowledge and Motivation was lost to other projects and therefor the code quality and project techniques are sometimes in a very sad state.
The most unwanted job in such project teams is the packager’s and code manager’s, because it takes a lot of time and is a mine field in most projects: rotten code structures, undocumented procedures for packaging and deployment preparation and lack of knowledge, because the guy who used to do it left the team some years ago and never kept records of what to do and why.
In comes continuous integration and tools like hudson/jenkins that are excellent in performing automatic tasks like running tests, packaging, preparing a deployment directory and such.
A big shortcoming of SUnit in combination with hudson is that it doesn’t keep the description texts of assumptions anywhere. You can only see them if you debug a TestCase in the Test Browser or a workspace snippet, and only if you run that very test individually. Running a big TestSuite means losing these texts completely.
This is especially bad in combination with a tool that can keep a list of all failed tests for statistical reasons and more importantly for all team members to check every morning. It’s really annoying if you have to rerun a single t
So I wanted to add the descriptive Text to some kind of list that I can use to provide useful feedback.
So what is different now with failureTexts?
You can now run a TestSuite and inspect the TestResult. TestResult keeps a list of passes, failures and errors, which are the TestCases themselves (an instance of TestCase with the name of the individual test method it ran in the variable testSelector). Now a TestCase has an additional variable named failureText which holds a descriptive text of what failed or error’d (If you used assert:description:).
So if you write an assertion like
self assert: (1+1=3) description: ‘Smalltalk knows that 1+1 is not 3’.
You will find the TestCase in the failures list of the TestResult, and its failureText variable will contain the text ‘Smalltalk knows that 1+1 is not 3’.
That’s all folks. A little change that can help a lot!
What does it help?
In our projects we have a little exporter tool for SUnit test result in a jUnit compatible XML file. This can be imported by hudson and kept as one of many statistics of a build. So now we have a list of test results that each team member can browse each morning and see if their tests failed and what exactly failed in it.
Can I use it in my dialect?
I haven’t tried yet, but the code change is very small and I suspect it is completely portable. One of the tests in SUnit Testing references an exception class by name (ZeroDivide) and that may have to change in other dialects, but other than that I see no reason why it shouldn’t be portable. I’m happy to provide file out instead of a .dat to anybody who’d like to try the change in their favorite smalltalk. I’d also be happy to hear from you if you tried it and what you think of it.