Skip to main content

Working with Text Based Logs

So today while I was setting up my new office furniture (Complete with Ms. Pacman) I decided to take a quick look over at PowerShell.com to see if I can lend a hand.  There was one thread where a text based log file was needed to be analyzed. I’ve had a lot of experience converting text based log files into object orientated log files for use with PowerShell so I decided to jump in and help. First off to the other contributors, very nice code.  Craig Duff converted the Date/Time stamp much better than the route that I was going to take (Line 37).  The remaining requirement was a final count of the desired objects.  I re-wrote the code into a PowerShell function and pasted it up on the thread.  Now, for the fun part.

The original task was to count the number of warning events that have occurred over the past 5 minutes.  I decided to count the number of each event over the course of what ever number of minutes that you want.  The unique thing here is the object placed in the pipeline.  Normally I would not place an object with unknown properties in the pipeline, but like I said, this is the fun part.  The END block actually makes an object based on the unique instances of the message property.  In other words, if you had a log with INFO, ERROR, and WARNING, you would get a property for each and the total count of each for the time specified.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

 

 

Function Get-LogInformationCount

{

[cmdletbinding()]

Param (

    [parameter(Mandatory=$true,

              ValueFromPipeline=$true)]

    [String[]]

    $Path,

 

    [INT]

    $TimeInMinutes = 5

)

 

BEGIN

{

    # This will create the object used to turn the text based

    # log information into object based.

    Function New-DataObject

    {

        $Obj = New-Object -TypeName PSObject -Property @{

            TimeStamp = $Null

            Message = $Null

        }

        Write-Output $Obj

    }

} # END: BEGIN Block

   

PROCESS

{  

    # Since we are doing a sum of objects, we cannot pass directly to

    # the PowerShell pipeline.  Instead, we will cache the objects in

    # $colData.

    $colData = @()

    ForEach ($Data in (Get-Content -Path $Path))

    {

        $Obj = New-DataObject

        $Obj.TimeStamp = [DateTime]::Parse(($Data -split "\s",2)[0])

        $Obj.Message = $Data.Remove(0,$Data.LastIndexOf("-")+1).Trim()

        $colData += $Obj

    }

 

 

} # END: PROCESS Block

 

END

{

    # Get the number of unique messages.  To make this code more dynamic, I

    # am not hard coding the possible message names.

    $Messages = $ColData | Select-Object -ExpandProperty Message -Unique

   

    # Create the custome object. This will create an object that will have as many

    # property names as needed to cover all unique instances of the Massage property.

    $Obj = New-Object -TypeName PSObject

    ForEach ($M in $Messages)

    {

        $Obj | Add-Member -MemberType NoteProperty -Name $M -Value $Null

 

        $colData| 

        Where-Object {($_.Timestamp -gt (Get-date).AddMinutes(-$TimeInMinutes)) -and

                       ($_.Message -eq $M)} |

        ForEach-Object {

            $String = "$M"

            $Obj."$String"++

            }

    }

   

    # Write the data to the pipeline.

    Write-Output $Obj

 

 

} # END: END Block

 

<#

.SYNOPSIS

Returns the number of specific types of information in a text based log file.

 

.DESCRIPTION

Returns the number of specific types of information in a text based log file.

This code is very specific to a log of this format and will not work on any other log

without customization.

 

2015-07-02T00:00:11 - INFO

2015-07-02T00:01:30 - INFO

2015-07-02T00:03:21 - WARNING

2015-07-02T00:06:13 - WARNING

2015-07-02T00:09:21 - WARNING

2015-07-02T00:15:13 - INFO

2015-07-02T00:34:41 - WARNING

 

.PARAMETER Path

The path to the log file

 

.PARAMETER TimeInMinutes

The number of minutes in the past to examine from the log file.

 

.EXAMPLE

Get-LogInformationCount -Path E:\Log.txt

INFO WARNING

---- -------

   2       2

 

Recovers all of the log files from the previous 5 minutes stored in E:\Log.txt

 

.EXAMPLE

"E:\Log.txt" | Get-LogInformationCount -TimeInMinutes 10

INFO WARNING

---- -------

   3       4

 

Recovers all of the log files from the previous 10 miuntes stored in e:\Log.txt

 

.NOTES

===============================================================================

== Cmdlet: Get-LogInformationCount                                           ==

== Author: Jason A. Yoder                                                    ==

== Company: MCTExpert of Arizona                                             ==

== Date: 2015JUL07                                                           ==

== Copyright: All rights reserved.                                           ==

== Version: 1.0.0.0                                                          ==

== Legal: The user assumes all responsibility and liability for the usage of ==

== this PowerShell code.  MCTExpert of Arizona, Its officers, shareholders,  ==

== owners, and their relatives are not liable for any damages.  As with all  ==

== code, review it and understand it prior to usage.  It is recommended that ==

== this code be fully tested and validated in a test environment prior to    ==

== usage in a production environment.                                        ==

==                                                                           ==

== Does this code make changes: NO                                           ==

===============================================================================

#>

} # END: Function Get-LogInformationCount

 

 

Comments

Popular posts from this blog

Sticky Key problem between Windows Server 2012 and LogMeIn

This week I instructed my first class using Windows Server 2012 accessed via LogMeIn and discovered a Sticky Key problem every time you press the Shift key. Here is my solution to resolve this.  First off, in the Preferences of LogMeIn for the connection to the Windows Server, click General . Change the Keyboard and mouse priority to Host side user and click Apply at the bottom. On the Windows 2012 server, open the Control Panel – Ease of Access – Change how your keyboard works . Uncheck Turn on Sticky Keys . Click Set up Sticky Keys . Uncheck Turn on Sticky Keys when SHIFT is pressed five times . Click OK twice. If you are using Windows Server 2012 as a Hyper-V host, you will need to redo the Easy of Use settings on each guest operating system in order to avoid the Sticky Key Problem. Updated Information: March 20, 2013 If you continue to have problems, Uncheck Turn on Filter Keys .

Where did a User’s Account Get Locked Out?

Updated: May 15, 2015 When this article was originally published, two extra carriage returns were add causing the code to malfunction.  The code below is correct.   My client for this week’s PowerShell class had a really interesting question. They needed to know where an account is being locked out at. OK, interesting. Apparently users hop around clients and forget to log off, leading to eventual lock out of their accounts. The accounts can be unlocked, but are then relocked after Active Directory replication. This problem is solved in two parts. The first one is to modify the event auditing on the network. The second part is resolved with PowerShell. The first part involves creating a group policy that will encompass your Domain Controllers. In this GPO, make these changes. Expand Computer Configuration \ Policies \ Windows Settings \ Security Settings \ Advanced Audit Policy Configuration \ Audit Policies \ Account Management Double click User Account Management C...

Backup and Restore AD LDS with DSDBUTIL.exe

Active Directory Lightweight Directory Services allow you to create a directory service that allows applications to have access to user accounts, groups, and authentication similar to Active Directory Domain Services.  The big advantage here is that the schema of the directory service will not be bound by the rules of an Active Directory database.  Exchange 2007/2010, for example, use an instance of AD LDS on the Edge Transport Server to provide for user authentication from the internet.  Because your Active Directory database is not exposed to the internet, this is more secure. Applications will handle most of the dirty work should they require AD LDS.  You may want to make sure the database is being backed up and also have a restore plan in place.  Should the database become corrupt, the application that uses that database will fail.  This document will walk you through backing up and restoring an instance of AD LDS using the dsdbutil.exe command. Fi...