Join 34,000+ subscribers and receive articles from our blog about software quality, testing, QA and security.

Automating extracting data from logs


Hi all,

I use SmartInspect extensively within my product, in dozens of projects, and it’s great - really helpful. One of the things I use it for is logging of raw data (data received from an external source). It’s often handy to extract this data to “replay” through a system to find an elusive bug (or to prove - yet again - to the client that what WE do with the data is correct…)

Mostly, I do this by creating a view of my log data with just the raw lines in it; extract this to a CSV file, and run an AWK script to convert it to the format I use. Which is fine with just one file, but gets a little bit tedious when one has a couple of dozen half-gig log files to process…

Is there any way of automating the process? e.g. programatically open the .sil file, extract the log entries I want, I can do the AWK stuff internally (in fact, I’d not even need AWK; I only want the description field & the time field), move onto the next entry, etc.