My MbUnit first experience
I needed to run some unit tests today against an open source project that has some written using the MbUnit fixtures. Having previously used csUnit and NUnit I felt confident the process would be a breeze. Oh how wrong I was… I decided to show the process I took as a screencast, but some of the highlights are covered below:
View the MbUnit first experience screen cast (Flash) (7MB)
Installer abandons you
No icons created for you, no getting started info, no indication of where it put files unless you caught the window on-screen for 1 second. It may as well be in ZIP format – at least you’d know where you unpacked it.
Web site “Overview” and “Getting Started” pages have nothing about running tests
Some information on creating them and what the product is about but nothing about how to actually use it or get started running tests. That is buried away below text fixture type information.
Silently failing tests in GUI tester
Once you finally make it to the GUI tester and load the assembly and hit run… should an exception fire then the GUI doesn’t so much as display an error message or even indicate a test failed. It simply says there are x number of tests to run, 0 success, 0 failed, 0 ignored and 0 skipped.
If you want to actually see indication of the error you have to head into Report whereupon the option to output to text throws another exception.
Thankfully HTML works.
Inconsistent GUI
Not as essential as the other points but still valid…
Major mismatch between context menu and main menu options. Some options available on one and not the other (NCover, Tree) and where they are available on both are named differently (Load Project becomes Load Config).
“Remove Assemblies…” does not prompt with dialog as “…” might suggest but removes All assemblies.
Given the amount of discussion I’ve seen on MbUnit I can’t help but imagine I’m missing something.
Update
It would appear the GUI has been neglected in favour of IDE integration with TestDriven.NET. It still silently fails however.
[)amien
4 responses
Never heard of it before today. Having quickly skimmed the site the main thing that struck me was how weird it seemed to use the term ‘RTM’ for your releases when you’re download-only ;) Taking MS naming convention a little far I think.
My best experience with unit testing was with JUnit. Eclipse integration is really good, interactive and batch running worked great (the latter via ant, mailing error reports etc), coverage analysis through Clover – generally very good. boost::test is the best I’ve found in C++ but it hasn’t got as much of an ecosystem around it like JUnit had.
Hello Damien.
My name is Ben Hall, I help develop the MbUnit project.
I’m sorry you have had a bad experience, I would just like to comment on your points.
1) Installer abandons you. I agree with you on this and I feel start menu items would be helpful for new users. Once installed, the framework assemblies can easily be referenced from within Visual Studio and tests can be executed from TestDriven.NET and via CruiseControl.NET.
2) Web site “Overview” and “Getting Started” pages have nothing about running tests. We are in the process of doing a complete rework of every bit of documentation on both the main site and on the wiki to help users coming to the framework and we have some great people working on this to help improve the experience. Not sure on the timescales of this at the moment, the changes you start to become apparent within the next month.
3) Silently failing tests in GUI app. Interesting problem. Because it failed in the Test Setup and not within an actual test, the GUI did not display the error. There might be a reason for this, however it does need looking at. Thank you for bring this to our attention.
4) Inconsistent GUI. We are currently working on the development of Mbunit Gallio which includes a brand new UI however we feel TestDriven.Net and other test runners provide a good base for executing tests.
Thank you for your comments and screencast, they will be feed back into the development team and will be taken into account to improve the project. I hope the problems you have experience does not distract away from the core framework which does provide some excellent features.
I wrote an article on getting started which other users reading this might find useful .
If you have any questions then please email me or the mailing list
Thank you
Ben
Hi,
I am lead on the MbUnit project, I have only just become aware of this as I was away on vacation (but you have my undivided attention now ;-))
I appreicate you taking the time to raise these concerns but I ask that if you do have any further concerns that these are outlined on the mailing lists that Ben has pointed out. The issues are unlikely to get fixed if the team are not aware of them and the very best way of doing this is to send a mail to the list.
Thanks
Andy
Hi
I replied here