AX compilation log parser – Goshoom.NET Dev Blog

Dynamics AX allows you to export compilation output to a file (.html). The file can be imported back into AX and then you can work with compilation results in Dynamics AX application in the same way as after a normal compilation. That’s useful for later analysis without a need of recompilation, for automated builds and so on.

Nevertheless, it would be sometimes useful to analyse the results outside Dynamics AX and it’s needed to get data into a suitable format. If you look into the generated .html file, you’ll find that it’s really an XML with an HTML header and a transformation to HTML. Therefore import and export actually work simply with XML.

But… XML suitable for import to and export from Dynamics AX doesn’t have to be good for processing by other means. The generated XML contains just two types of elements – Table a Field, which significantly complicates queries and some values are not represented in any intuitive way:

<Table:Record name="TmpCompilerOutput" xmlns:Table='urn:www.microsoft.com/Formats/Table'>
    <Table:Field name="TreeNodePath">\Classes\Class1\classDeclaration</Table:Field>
    <Table:Field name="Column">5</Table:Field>
    <Table:Field name="Line">4</Table:Field>
    <Table:Field name="CompileErrorCode">-1</Table:Field>
    <Table:Field name="CompileErrorString"></Table:Field>
    <Table:Field name="UtilElementType">45</Table:Field>
    <Table:Field name="SysUtilElementName">Class1</Table:Field>
    <Table:Field name="SysCompilerSeverity">0</Table:Field>
    <Table:Field name="CompileErrorImage">0</Table:Field>
    <Table:Field name="SysPropertyName"></Table:Field>
    <Table:Field name="SysCompileErrorMessage">Syntax error.</Table:Field>
    <Table:Field name="SysAotMethodName">classDeclaration</Table:Field>
    <Table:Field name="UtilElementImage">0</Table:Field>
    <Table:Field name="LatestCompile">1</Table:Field>
    <Table:Field name="SysCompilerOutputTab">2</Table:Field>
    <Table:Field name="createdDateTime">10/1/2012 03:08:20 pm</Table:Field>
    <Table:Field name="recVersion">0</Table:Field>
</Table:Record>

This is a good solution for its purpose, i.e. for import and export from table TmpCompilerOutput, but it is not very useful for anything else.

If you wonder what additional processing I’m talking about, imagine that I have compilation logs from night builds over several months and I wound like to mine some information from them – where errors occur frequently, how the number of errors is changing with time and so on. And I really didn’t want to work with this structure of XML.

At the end, I didn’t chose the way of direct transformation of XML, instead I’ve written a simple .NET library for parsing compilation output to objects. I’m not going to describe it in detail (the link to the complete project is at the end of this post), it’s really simple – the whole parsing is done by LogParser class and you can get by with just ParseFile() method at the beginning.

The core of the whole implementation is the following query:

XDocument loaded = XDocument.Parse(xml);
XNamespace tableNs = "urn:www.microsoft.com/Formats/Table";
 
var faults = from record in loaded.Descendants(tableNs + "Record")
             let fields = record.Descendants()
             select new Fault
             {
                 Column = Int32.Parse(FindValue(fields, "Column")),
                 CompilationTime = DateTime.Parse(FindValue(fields, "createdDateTime")),
                 ElementName = FindValue(fields, "SysUtilElementName"),
                 ElementType = (UtilElementType)Int32.Parse(FindValue(fields, "UtilElementType")),
                 FaultCode = Int32.Parse(FindValue(fields, "CompileErrorCode")),
                 FaultMessage = FindValue(fields, "SysCompileErrorMessage"),
                 FaultString = FindValue(fields, "CompileErrorString"),
                 FaultType = ParseFaultType(fields),
                 Line = Int32.Parse(FindValue(fields, "Line")),
                 MethodName = FindValue(fields, "SysAotMethodName"),
                 PropertyName = FindValue(fields, "SysPropertyName"),
                 TreeNodePath = FindValue(fields, "TreeNodePath")
             };

As you can see, the XML is converted to a collection of Fault instances, it uses some other types (e.g. UtilElementType enum) and omits or transform some fields.

Objects of Fault type can be serialized directly to XML – the error stated above as XML from AX would look like this:

<Fault>
    <Column>5</Column>
    <CompilationTime>2012-01-10T15:08:20</CompilationTime>
    <ElementName>Class1</ElementName>
    <ElementType>Class</ElementType>
    <FaultCode>-1</FaultCode>
    <FaultMessage>Syntax error.</FaultMessage>
    <FaultString />
    <FaultType>Error</FaultType>
    <Line>4</Line>
    <MethodName>classDeclaration</MethodName>
    <PropertyName />
    <TreeNodePath>\Classes\Class1\classDeclaration</TreeNodePath>
</Fault>

That’s a significantly friendlier format for processing, but it’s not necessary to limit yourself to a simple transformation of a single file – you can, for example, filter just the relevant data or to work with multiple files in the same time.

Let’s look at an example with my night builds. At first we will load information about compilation errors (not warnings etc.) in all builds and save them to a single file.

string path = @"c:\AX\NightBuild\";
LogParser parser = new LogParser();
 
var faults = from file in Directory.EnumerateFiles(path, "AxCompileAll.html", SearchOption.AllDirectories)
             from fault in parser.ParseFile(file)
             where fault.FaultType == FaultType.Error
             select fault;
 
using (FileStream fs = new FileStream(@"c:\AX\errors.xml", FileMode.CreateNew))
{
    new XmlSerializer(typeof(List<Fault>)).Serialize(fs, faults.ToList());
}

It’s worth mentioning that the implementation is not exactly optimal from performance perspective, but it’s good enough for my needs, so there is no reason to do much optimization. The advantage of saving to a file is that this step doesn’t have to be repeated for further queries using the same data.

We could do a lot of things with the list of errors over a period of time. For example, the code below displays objects based on how many builds their occurred in and it shows the list of those days. It gives you a rough idea about where errors are often made or where errors are fixed slowly.

List<Fault> faults = null;
using (FileStream fs = new FileStream(@"c:\AX\errors.xml", FileMode.Open))
{
    faults = (List<Fault>)new XmlSerializer(typeof(List<Fault>)).Deserialize(fs);
}
 
var transformed = from fault in faults
                  group fault.CompilationTime.Date by new { fault.ElementType, fault.ElementName } into grouped
                  let output = new
                  {
                      grouped.Key.ElementType,
                      grouped.Key.ElementName,
                      Dates = grouped.AsEnumerable().Distinct().OrderByDescending(d => d)
                  }                   
                  orderby output.Dates.Count() descending, output.ElementType, output.ElementName
                  select output;
 
foreach (var fault in transformed)
{
    Console.WriteLine("{0} {1}", fault.ElementType, fault.ElementName);
    foreach (DateTime date in fault.Dates)
    {
        Console.WriteLine("    {0:d}", date);
    }
}

The important thing is that you’ll have a collection of strongly-typed objects instead of some .html file. It’s completely up to you how you’ll use them.

Download

CompilationLogParser.zip (written in Visual Studio 2010 for .NET 4.0)