Advanced Windows PowerShell Scripting Video Training

Advanced Windows PowerShell Scripting Video Training
Advanced Windows PowerShell Scripting Video Training

Monday, June 17, 2013

Using Text Based Logs with Windows PowerShell (4 of 8)


Day 4: Creating a PowerShell Object out of a Simple Text Log
Last Friday we left off with a list of properties from our log. Today we will complete the final 3 steps in this process:
3.       Create an object that contains all possible property names.
4.       Read the log records into the object.
5.       Send to the pipeline.
Steps 3 and 4 are actually integrated together while Step 5 is an easy one liner.
We are going to add 3 new functions to our cmdlet.  The first function Import-LogData will actually call the function Create-EmptyObject which will call the function Get-Hash to support their individual purposes.  Let’s take a look at the shorter of the functions, Get-Hash
    Function Get-Hash
    {
    Param ($Name)
    # This brief function is used to used to reduce the amount of code
    # while building a new empty object.
 
        $Hash = @{MemberType = "NoteProperty";
                    Name = $Name;
                    Value = $Null}
 
        Write-Output $Hash
           
    } # End: Function Get-Hash
 
This function will cut down on a lot of typing.  It will be called while creating a new, empty object.  The Create-EmptyObject function will use this function for each property that needs to be added to the empty object being created.
    Function Create-EmptyObject
    {
    Param ($Prop)
    # Here the list of discovered properties is sent so that
    # a blank object can be created containing all of the potential
    # properties of the object.  this will allow for a consistent
    # set of members for the final object.
       
        # Create a blank object.           
        $Obj = New-Object -TypeName PSObject
           
        # Change the type name.
        $Obj.pstypenames.insert(0,'Custom.LogObject')
 
        # This will create a new property for each of the derived
        # properties.
        ForEach ($P in $Prop.prop)
        {
            $Hash = Get-Hash -Name $P 
            $Obj | Add-Member @Hash
        }
  
        # Add the TimeWritted Property.
        $Hash = Get-Hash -Name "TimeWritten" 
        $Obj | Add-Member @Hash
       
        # Return empty object to calling statement.
        Write-Output $Obj           
 
    } # End: Function Create-EmptyObject
 
Creating an empty object is the sole duty of Create-EmptyObject.  We will pass to it the list of properties that we derived from the log file earlier.  This will create an object of type LogObject.  This will also ensure that each instance of this object has the same property values.   Also, an additional property call TimeWritten is added to the empty object.  The ForEach loop will add the property names that we derived earlier in the Get-PropertyList function.  This one allows us to also add the time stamp information.
    Function Import-LogData
    {
    Param ($Records, $ObjProps)
    # This is where we put it all together. Each record is cycled through.
    # If the record contains a property that we derived earlier, a new object
    # is created containing the time stamp and the data for that property.
 
        ForEach ($R in $Records.Record)
        {
            $Obj = Create-EmptyObject -Prop $ObjProps
            ForEach ($O in $ObjProps.Prop)
            {
                If ($R -like "*$($O)*")
                {
                    # Add the data time stamp
                    $End = $R.IndexOf("]")
                    $Obj.TimeWritten = $R.Substring(1,$End-1)
               
                    # Add the property that caused the match.
                    $Obj.$O = $R.Remove(0,22).Replace("$($O):","").Trim()
                   
                }
            }
            Write-Output $Obj
   
        } # End: ForEach ($R in $Records)
    } # End: Function Import-LogData
 
Import-LogData is called from the main code to start the process of creating an object for the log data.  We pass to it the list of records created in the Get-Record function and the list of properties created in the Get-PropertyList function.
The first ForEach loop will cycle through each record one at a time. The second ForEach loop will compare the current record to see if it contains one of the properties that we gathered from Get-PropertyList.  If it does, it first adds the date/time information to the TimeWritten property.  It then adds the remaining string data to the property of the object that has the same name that caused the match. Finally, our object is sent to the pipeline.
Here is our final code:
Function Import-MyLog1
{
    # -----------------------------------------------------------------------
    Function Get-Log
    {
    # Reads the log file into memory.  Will display an error if the
    # file cannot be found.
 
        Try
        {
            Get-Content -path "\\Server1\Data.txt" `
                        -ErrorAction Stop |
            Write-Output
        }
        Catch
        {
            Write-Error "The data file is not present"
            BREAK
        }
    } # End: Function Get-Log
    # -----------------------------------------------------------------------
    Function Get-Record
    {
    Param ($Log)
    # Determines what is a record.  This eliminates any header information.
    # For this to work properly, you need to utilize some type of rule
    # to determine what is a new record.  In this case, a pattern
    # is being utilized that we will send to a regular expression.
    # The pattern is:
    # -- ::
    # Example - 2013-05-30 16:06:40
    #
    # See About_Regular_Expressions for more information on how to use
    # Regular Expressions in the built in PowerShell help files.
 
 
        # Array to hold all of the objects.
        $Object = @()
 
        # Cycle through each line of the log and look for a line that
        # matches the pattern that we are using to denote a new record.
        ForEach ($L in $Log)
        {
            If ($L -match "\d*-\d*-\d* \d*:\d*:\d*")
            {
                $Obj = New-Object -TypeName PSObject
 
                $Splat = @{NotePropertyName = "Record";
                          NotePropertyValue = "$($L)"}
                $Obj | Add-Member @Splat
                $Object += $Obj 
            }
        } # End: ForEach ($L in $Log)
       
        # Send the records to the calling statement.
        Write-Output $Object
 
    } # End: Function Get-Record
    # -----------------------------------------------------------------------
    Function Get-PropertyList
    {
    Param ($Records)
    # To help build a more flexible object, we need to identify possible
    # property names.  This function will cycle through each record and use
    # a rule to identify the name of a property.  Our rule is any string
    # after the "]" in the time stamp and before the next ":". And
    # object containing the property names is returned.
 
        # Array to hold all of the objects.
        $Object = @()
 
        <# Cycle through each record to get a list of properties. This is
           done to discover all the potential properties.
        #>
       
        ForEach ($R in $Records.record)
        {
            $Obj = New-Object -TypeName PSObject
            # Find the end of the Date time stamp and remove it along
            # with the leading space.
            $Start = $R.IndexOf("]")
            $Value = $R.Substring($Start+2)
 
            # Find the next colon which will mark the end of the
            # property.  Remove this colon and the rest of the string.
            $End = $Value.IndexOf(":")
            $value = $Value.Substring(0,$End)
           
            # Add the property to the list.
            $Splat = @{NotePropertyName = "Prop";
                        NotePropertyValue = "$($Value)"}
            $Obj | Add-Member @Splat
            $Object += $Obj              
        } # End: ForEach ($R in $Records.record)
 
        # Make sure that each property is unique.
 
        $ObjectList = $Object | Select-Object -Property Prop -Unique
 
        # Send the records to the calling statement.
        Write-Output $ObjectList
    } #End: Function Get-ADResourcePropertyList
 
    # -----------------------------------------------------------------------
    Function Get-Hash
    {
    Param ($Name)
    # This brief function is used to used to reduce the amount of code
    # while building a new empty object.
 
        $Hash = @{MemberType = "NoteProperty";
                    Name = $Name;
                    Value = $Null}
 
        Write-Output $Hash
           
    } # End: Function Get-Hash
 
    # -----------------------------------------------------------------------
    Function Create-EmptyObject
    {
    Param ($Prop)
    # Here the list of discovered properties is sent so that
    # a blank object can be created containing all of the potential
    # properties of the object.  this will allow for a consistent
    # set of members for the final object.
       
        # Create a blank object.           
        $Obj = New-Object -TypeName PSObject
           
        # Change the type name.
        $Obj.pstypenames.insert(0,'Custom.LogObject')
 
        # This will create a new property for each of the derived
        # properties.
        ForEach ($P in $Prop.prop)
        {
            $Hash = Get-Hash -Name $P 
            $Obj | Add-Member @Hash
        }
  
        # Add the TimeWritted Property.
        $Hash = Get-Hash -Name "TimeWritten" 
        $Obj | Add-Member @Hash
       
        # Return empty object to calling statement.
        Write-Output $Obj           
 
    } # End: Function Create-EmptyObject
 
    # -----------------------------------------------------------------------
 
    Function Import-LogData
    {
    Param ($Records, $ObjProps)
    # This is where we put it all together. Each record is cycled through.
    # If the record contains a property that we derived earlier, a new object
    # is created containing the time stamp and the data for that property.
 
        ForEach ($R in $Records.Record)
        {
            $Obj = Create-EmptyObject -Prop $ObjProps
            ForEach ($O in $ObjProps.Prop)
            {
                If ($R -like "*$($O)*")
                {
                    # Add the data time stamp
                    $End = $R.IndexOf("]")
                    $Obj.TimeWritten = $R.Substring(1,$End-1)
               
                    # Add the property that caused the match.
                    $Obj.$O = $R.Remove(0,22).Replace("$($O):","").Trim()
                   
                }
            }
            Write-Output $Obj
   
        } # End: ForEach ($R in $Records)
    } # End: Function Import-LogData
    # -----------------------------------------------------------------------
 
    # Load the log into memory
    $Log = Get-Log
 
    # Extract the records
    $Records = Get-Record -Log $Log
 
    # Get a list of all properties.
    $ObjProps = Get-PropertyList -Records $Records
 
    # Split the record into an PowerShell Object
    Import-LogData -Records $Records -ObjProps $ObjProps
 
} #End: Function Import-MyLog1
 
Here is our output:
Info                         Warning                      Critical                     TimeWritten                 
----                         -------                      --------                     -----------                
InstallHelper Start                                                                    2013-05-30 16:06:40        
ST_Drive::IsInternalDrive...                                                           2013-05-30 16:06:40        
IsTablePC return FALSE                                                                 2013-05-30 16:06:40        
InstallHelper End                                                                      2013-05-30 16:06:40        
                             InstallHelper Experienced...                              2013-05-30 16:10:40        
                                                          Help files are not availa... 2013-05-30 16:10:40    
 
Here is the member information:
    TypeName: Custom.LogObject
 
Name        MemberType   Definition                                  
----        ----------   ----------                                  
Equals      Method       bool Equals(System.Object obj)              
GetHashCode Method       int GetHashCode()                           
GetType     Method       type GetType()                              
ToString    Method       string ToString()                            
Critical    NoteProperty  Critical=null                              
Info        NoteProperty System.String Info=InstallHelper Start      
TimeWritten NoteProperty System.String TimeWritten=2013-05-30 16:06:40
Warning     NoteProperty  Warning=null                               
 
 
This code will have to be customized for each text based log that you want to use it with.  The general steps are all the same. 
Tomorrow we will begin this process all over again.  This time however, we will work with a log file that has records spread across multiple lines in the text file with property values that are both common, and unique to each record.
 

No comments: