Day 1: The case for using PowerShell to Analyze Your Log
Files.
Over the years, I have worked with a variety of products
that utilized some type of text based log file.
I’ve had to use these log files for various tasks. One of my favorite examples of automation
came from an engineering company that I once worked for.
At this particular company, we had a small problem. When I arrived as the Network Administrator, I
noticed that single licensed software was freely passed around without any
controls. When I requested to lock up
all the copies, I was told not to worry about it. A year later, we received a letter from the
creator of the software informing us that we are being audited. This is a standard practice in the industry
so normally I would consider this not to be a big deal. Oh, but wait.
I have no control over this software.
At the conclusion of my audit, I discovered that we were now
6 figures in debt to this company. All
of the sudden, it became my problem.
After I successful removed myself from the trail of accountability using
a few emails that I saved, I began the work of cleaning up this mess while
management began the process of how to pay for the mess that they created. My part was simple. Once I was able to confiscate all the software
and all copies, we simply removed the software from all systems. Now, the fun part.
When I was presented with the plan for future licensing from the the regional managers, they
decided that they only needed one license per 3 users. I informed them that our new license server for
the product would be depleted of all available license in under 4 hours. I was reminded that the regional managers
studied this issues thoroughly and that my concerns were not warranted. OK, so I had them sign off that they made the
decision with my protest duly noted and I
implemented their grand scheme. Once we
turned it on, the entire pool of licenses were depleted in just 2 hours.
The individuals using this product were each working on multimillion
dollar projects. Each one of them were
calling me demanding that I provide them with a license. This placed me in the situation where I had
to continuously decipher a text based log file and start calling people and ask them to
release their license. Most of the time
their answer was “no.”
Well, this became too much of a burden so off I went and
started scripting. In the end, I developed
a script that deciphered the log file and determined who had a license and for
how long. This information was then
further processed into HTML code and then placed online. I then integrated the script with Active
Director so this internal web site would also provide the license holders cell phone
number. This stopped nearly all calls to
me and created what I called “Peer Accountability”. Under this model of Peer Accountability, if
one user was keeping a license to long they would hear about it from their
peers. It also allowed me to create a
clear and concise report of every failed attempt to get a license from each
region and present some concrete numbers to the regional managers. In the end, they had to purchase one license
per user. in the end, it made economic sense to
spend a few hundred thousand dollars once to make a few million dollars each
month.
Since different products have different log file formats, it
is difficult to create a script that can read them all. This series of posts will walk you through
the steps needed to create scripts to read your text based log files and create
PowerShell objects out of each record. Here is a list of steps that we will
need to accomplish:
1.
Identify a rule to separate records
2.
Identify a rule to separate property names from
property values
3.
Create an object that contains all possible
property names.
4.
Read the log records into the object.
5.
Send to the pipeline.
We will first work on a log file that has one record per
line and then build a script to handle logs that span multiple lines. This code will not work for your specific text based log. You will have to create the exact code for
your specific log so used these examples as general steps in the process. In the end, you will have PowerShell objects that you can then send into the pipeline to utilize all the automation that PowerShell has to offer.
Comments