I believe that unit testing is extremely important and that the lack of its adoption in Dynamics AX is a serious problem. One reason why people don’t write unit tests is that they don’t really know how. Let’s if I can help a little bit. My intention is neither to write a theoretical work nor a reference guide – I’ll try to explain the fundamentals and show a few examples without going into unnecessary details. The challenge is that although unit testing is trivial in principle, it requires skills and experience to do it right. Please keep it in mind and don’t jump immediately to anything overwhelmingly complex.
First of all, why should we bother with unit tests?
Imagine that you’re developing a complex piece of logic and after every change, you have 50 different cases to test. You either waste a lot of time with tedious testing, or you have a developer mindset and you immediately want to automate it (or you give up testing, but you aren’t that nasty, are you?). If you automate it, you make your development more efficient and you may be finished sooner that if you tested it manually. As a bonus, these tests can be used at any time, perhaps two years later when somebody needs to add a new feature without breaking the original logic. They also work as executable documentation – other developers can see how the code should be used and what test cases must be taken into account.
Now, what exactly do I mean by a unit test? It’s basically a piece of code testing that a unit (typically a method or a class) works as designed. Look at this example:
// Prepare object to test Number n = new Number(5); // Perform some action n.Add(1); // Test the result assetEquals(6, n.value());
As you can see, it’s all about code. This kind of testing is done during development by developers; it’s not a distinct phase performed by a QA department or somebody. These tests are typically written together with the functionality being tested, which also have positive impact on design of code.
As a side note, please realize that every project requires many different types of tests, because no single type can cover everything. For example, you’ll never test all possible path through code and boundary values if you test only through UI. That’s where unit tests excel. On the other hand, that all unit tests pass doesn’t mean that users will find the product useful. That requires different tests.
The fact that we’re testing a single unit is important – with these tests, you want to test a method or a class rather than something like invoicing. Why?
One reason is complexity. If you’re testing a large component consisting of many classes, it has many different states and paths through code and complexity grows exponentially as each class multiplies its number of states (for example) with other classes. The complexity and the number of needed tests get quickly out of hand. The solution is testing smaller parts individually before their complexity grows uncontrollably.
Sure, there may be problems in the integration of classes and you’ll need same tests for it. But these integration tests won’t bother to test all details; instead, they will focus on how the classes communicate. What they do alone has already been tested.
Another reason for testing small units is the ability to quickly locate the problem. Ideally, the failed unit test will tell you which particular method doesn’t work and which assertion failed. If it told you only that something was broken in invoicing, it wouldn’t be very helpful. If you’re often forced to use debugger to find why a unit test failed, your tests don’t fulfil their role well and should be improved.
One more reason is maintainability. If you test a small unit, you need a small number of tests and if the interface changes (e.g. a method is renamed), fixing tests isn’t too much work. If you test a huge component, you need a huge number of tests and maintaining them may be very expensive. It’s important to realize that it wouldn’t be a fault of unit testing – the cause would be poorly designed tests.
In general, unit tests tend to be small and isolated. You don’t want one test method to test many different things (because it wouldn’t be clear what failed), nor you want to test the same code by too many tests (because you would have to fix them if code changes). You also don’t want tests to influence each other, because that could also prevent from you from locating the cause or you would get false positives.
It’s different from how human testers design tests – they often chain tests together rather than trying to isolate them, because starting each test from scratch would present too much overhead to them. You mustn’t forget that various types of tests require various approaches to design.
I hope this gave you some idea about how unit tests are designed and used, despite that it’s necessarily oversimplified. I’m going to write at least one, more practical article with examples of SysTest framework in AX. If you have some questions, please let me know – I’ll try to incorporate answers (if I have some) in the subsequent post(s).
Thanks for highlighting this. I’ve been writing unit tests for AX for about three years now and I think after you go beyond the basics, there are two big challenges:
(1) Test data: Although a common problem for unit testing, I find it sometimes especially hard to prepare good test data for AX unit tests, especially if you go beyond main data like products or customers and need transactional data. Sometimes it is just not possible to mock up the data of e.g. a sales invoice and you need to do the whole process in your unit test, which breaks one of the fundamental requirements of unit tests: speed.
(2) Testing of lower layer (aka “standard AX”) objects: Quite frequently you need to customize objects from lower layers like sys layer. Writing unit tests for these customizations can be really hard because you sometimes only know or care about a fraction of the functionality of the object, but you should test the whole functionality of the object to make sure the customization does not break any other processes the object is involved in.
Both challenges would in my opinion be much easier if Microsoft gave access to the unit tests that it uses to test the standard AX application. But despite several requests, Microsoft has not provided such an access, their reasoning being that these unit tests would not be useful (because everybody customizes AX, which would make these unit tests fail) and it would be hard to support them. I think the second is a fair argument, but I wonder how much support Microsoft would not have to provide if more developers would unit test their customizations.
That said I guess I’m looking for tips on how to overcome these two challenges.
is it possible to do property based testing, any tool to do that?
There is no such tool for AX as far as I know.