Skip to main content

Working with Text Based Logs

So today while I was setting up my new office furniture (Complete with Ms. Pacman) I decided to take a quick look over at PowerShell.com to see if I can lend a hand.  There was one thread where a text based log file was needed to be analyzed. I’ve had a lot of experience converting text based log files into object orientated log files for use with PowerShell so I decided to jump in and help. First off to the other contributors, very nice code.  Craig Duff converted the Date/Time stamp much better than the route that I was going to take (Line 37).  The remaining requirement was a final count of the desired objects.  I re-wrote the code into a PowerShell function and pasted it up on the thread.  Now, for the fun part.

The original task was to count the number of warning events that have occurred over the past 5 minutes.  I decided to count the number of each event over the course of what ever number of minutes that you want.  The unique thing here is the object placed in the pipeline.  Normally I would not place an object with unknown properties in the pipeline, but like I said, this is the fun part.  The END block actually makes an object based on the unique instances of the message property.  In other words, if you had a log with INFO, ERROR, and WARNING, you would get a property for each and the total count of each for the time specified.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

 

 

Function Get-LogInformationCount

{

[cmdletbinding()]

Param (

    [parameter(Mandatory=$true,

              ValueFromPipeline=$true)]

    [String[]]

    $Path,

 

    [INT]

    $TimeInMinutes = 5

)

 

BEGIN

{

    # This will create the object used to turn the text based

    # log information into object based.

    Function New-DataObject

    {

        $Obj = New-Object -TypeName PSObject -Property @{

            TimeStamp = $Null

            Message = $Null

        }

        Write-Output $Obj

    }

} # END: BEGIN Block

   

PROCESS

{  

    # Since we are doing a sum of objects, we cannot pass directly to

    # the PowerShell pipeline.  Instead, we will cache the objects in

    # $colData.

    $colData = @()

    ForEach ($Data in (Get-Content -Path $Path))

    {

        $Obj = New-DataObject

        $Obj.TimeStamp = [DateTime]::Parse(($Data -split "\s",2)[0])

        $Obj.Message = $Data.Remove(0,$Data.LastIndexOf("-")+1).Trim()

        $colData += $Obj

    }

 

 

} # END: PROCESS Block

 

END

{

    # Get the number of unique messages.  To make this code more dynamic, I

    # am not hard coding the possible message names.

    $Messages = $ColData | Select-Object -ExpandProperty Message -Unique

   

    # Create the custome object. This will create an object that will have as many

    # property names as needed to cover all unique instances of the Massage property.

    $Obj = New-Object -TypeName PSObject

    ForEach ($M in $Messages)

    {

        $Obj | Add-Member -MemberType NoteProperty -Name $M -Value $Null

 

        $colData| 

        Where-Object {($_.Timestamp -gt (Get-date).AddMinutes(-$TimeInMinutes)) -and

                       ($_.Message -eq $M)} |

        ForEach-Object {

            $String = "$M"

            $Obj."$String"++

            }

    }

   

    # Write the data to the pipeline.

    Write-Output $Obj

 

 

} # END: END Block

 

<#

.SYNOPSIS

Returns the number of specific types of information in a text based log file.

 

.DESCRIPTION

Returns the number of specific types of information in a text based log file.

This code is very specific to a log of this format and will not work on any other log

without customization.

 

2015-07-02T00:00:11 - INFO

2015-07-02T00:01:30 - INFO

2015-07-02T00:03:21 - WARNING

2015-07-02T00:06:13 - WARNING

2015-07-02T00:09:21 - WARNING

2015-07-02T00:15:13 - INFO

2015-07-02T00:34:41 - WARNING

 

.PARAMETER Path

The path to the log file

 

.PARAMETER TimeInMinutes

The number of minutes in the past to examine from the log file.

 

.EXAMPLE

Get-LogInformationCount -Path E:\Log.txt

INFO WARNING

---- -------

   2       2

 

Recovers all of the log files from the previous 5 minutes stored in E:\Log.txt

 

.EXAMPLE

"E:\Log.txt" | Get-LogInformationCount -TimeInMinutes 10

INFO WARNING

---- -------

   3       4

 

Recovers all of the log files from the previous 10 miuntes stored in e:\Log.txt

 

.NOTES

===============================================================================

== Cmdlet: Get-LogInformationCount                                           ==

== Author: Jason A. Yoder                                                    ==

== Company: MCTExpert of Arizona                                             ==

== Date: 2015JUL07                                                           ==

== Copyright: All rights reserved.                                           ==

== Version: 1.0.0.0                                                          ==

== Legal: The user assumes all responsibility and liability for the usage of ==

== this PowerShell code.  MCTExpert of Arizona, Its officers, shareholders,  ==

== owners, and their relatives are not liable for any damages.  As with all  ==

== code, review it and understand it prior to usage.  It is recommended that ==

== this code be fully tested and validated in a test environment prior to    ==

== usage in a production environment.                                        ==

==                                                                           ==

== Does this code make changes: NO                                           ==

===============================================================================

#>

} # END: Function Get-LogInformationCount

 

 

Comments

Popular posts from this blog

Adding a Comment to a GPO with PowerShell

As I'm writing this article, I'm also writing a customization for a PowerShell course I'm teaching next week in Phoenix.  This customization deals with Group Policy and PowerShell.  For those of you who attend my classes may already know this, but I sit their and try to ask the questions to myself that others may ask as I present the material.  I finished up my customization a few hours ago and then I realized that I did not add in how to put a comment on a GPO.  This is a feature that many Group Policy Administrators may not be aware of. This past summer I attended a presentation at TechEd on Group Policy.  One organization in the crowd had over 5,000 Group Policies.  In an environment like that, the comment section can be priceless.  I always like to write in the comment section why I created the policy so I know its purpose next week after I've completed 50 other tasks and can't remember what I did 5 minutes ago. In the Group Policy module for PowerShell V3, th

Return duplicate values from a collection with PowerShell

If you have a collection of objects and you want to remove any duplicate items, it is fairly simple. # Create a collection with duplicate values $Set1 = 1 , 1 , 2 , 2 , 3 , 4 , 5 , 6 , 7 , 1 , 2   # Remove the duplicate values. $Set1 | Select-Object -Unique 1 2 3 4 5 6 7 What if you want only the duplicate values and nothing else? # Create a collection with duplicate values $Set1 = 1 , 1 , 2 , 2 , 3 , 4 , 5 , 6 , 7 , 1 , 2   #Create a second collection with duplicate values removed. $Set2 = $Set1 | Select-Object -Unique   # Return only the duplicate values. ( Compare-Object -ReferenceObject $Set2 -DifferenceObject $Set1 ) . InputObject | Select-Object – Unique 1 2 This works with objects as well as numbers.  The first command creates a collection with 2 duplicates of both 1 and 2.   The second command creates another collection with the duplicates filtered out.  The Compare-Object cmdlet will first find items that are diffe

How to list all the AD LDS instances on a server

AD LDS allows you to provide directory services to applications that are free of the confines of Active Directory.  To list all the AD LDS instances on a server, follow this procedure: Log into the server in question Open a command prompt. Type dsdbutil and press Enter Type List Instances and press Enter . You will receive a list of the instance name, both the LDAP and SSL port numbers, the location of the database, and its status.