This post is based on my replay in Dynamics Community forum, therefore if you read it there, you can safely move on.
When I hear “automated testing” in Dynamics AX world, people almost always mean replicating what a human tester does – opening a form, clicking around and so on. It makes some sense, because that’s what end users do as well.
On the other hand, it’s a poor choice for automation, because:
- It’s extremely fragile. A simple change in user interface (even caused by things like security) can break all your tests. If it uses image recognition, the problem is insanely huge – I see people struggling with tests if they change screen resolution, for example. You want tests fail if your application is broken, not because the resolution has changed.
- It lacks isolation. If something fails, you know that your have a problem somewhere in a business process, calling several thousand methods. Or maybe you have a problem with data. You simply don’t know. You’ll have to perform expensive debugging before you finally isolate the bug.
That’s why automated tests should test small isolated units.
- You must use AX database, therefore you have to maintain test data. Because database is shared for all tests, you can get into all the problems associated with shared test fixture (such as that one test can influence other tests, or you have to expensively cleanup the state before/after each test).
- It’s the slowest type of test you can imagine.
From my opinion, this is a kind of testing that you should consider when you already have all easier, faster and more reliable tests in place, you have experience with test automation and you want to get rid of the last manually intervention in integration and UI tests. If this is your first attempt for automated testing, it will very likely fail (you won’t achieve it at all or maintenance cost will negate all gains) and people often give up automated testing completely because of this experience. But the problem was that they knew nothing about automatic testing and therefore they made wrong decisions.
I would recommend two things:
- Use unit tests as the primary mean for verification of implementation. If you don’t test code at the lowest level and get poor implementation of methods, you’ll never build a high-quality product from them. Don’t worry about code coverage too much, but ensure yourself that all complicated logic you write is covered by unit tests.
- Use Microsoft Test Manager (or a similar tool) to plan and design your manual tests. Designing tests and expected results is the first and mandatory step towards automated UI tests, because scripts (unlike human testers) can’t just figure it out by themselves. It’s surprising how often people want to automated tests they’ve never defined.
Then record these tests and use fast forward to automate some steps. You’ll see that some actions can’t be replayed, therefore many tests can’t be fully automated in this way. That’s the moment when you can start thinking of converting test recordings to coded UI tests and implement logic needed for full automation. Nevertheless even if you never do that, you’ll already have quite a lot of automation and tools to highly improve productivity of human testers.