- Define your search terms
- Search your files with RegExTractor
- Create a transformation file
- Transform your xml
- Import the transformed xml into Excel and analyze your data.
For the next steps we'd like to use mature tools and technologies instead of inventing new ones. In this post I'd like to show, how to go the whole way to get an Excel chart out of your data.
Remember the example log from the "Getting Started" tutorials. As this is a log from one of my applications I know, that every message contains the class name and the method from which the log entry was written.
What I'd like to know now is, how often a class or method has been called. We're not talking about a debug session, we're talking about real life data, about real users who left their traces in our logs. We may gain knowledge about the most used functions, for example.
In our search term file we will define a search term similar to the one we've used in the "Getting Started" tutorials. We'd like to extract:
- Date
- Time
- Value between the first square bracktes (class name)
- Value between the second square brackets (method name)
As I mentioned this is not a tutorial of how to use regular expression. You should look for such a tutorial to get familiar with regular expressions!
The result xml after processing the log files with RegExTractor will look like this:
This is a generic RegExTractor format. You could try to import this xml into Excel - what will work at all. But finally, this xml is to complex for Excel to satisfy our needs. So we have to simplify our xml, using xsl transformation:
We transform our RegExTractor format into a more simpler xml.
Finally, this xml could be imported into Excel to analyse our data to gain knowledge about our system.
Once you've created your search terms and your xsl transformation file, the whole process could be automated to get the data from your logs within seconds.
I will show such an automation scenario in another post.