Advanced Windows PowerShell Scripting Video Training

Advanced Windows PowerShell Scripting Video Training
Advanced Windows PowerShell Scripting Video Training

Thursday, May 19, 2016

Saving data While it Travels Through the PowerShell Pipeline

A good question came up in class today while we were studying PowerShell Format commands.  We took a good look at how Format cmdlets consume the original object and make piping limited.  One of the questions that came up was if it is possible to save the information to a file and still be able to use it in the pipeline?  Fortunately for us, there is Tee-Object.
We have a couple of ways that we can implement Tee-Object.  Let’s look at it from a reusability perspective.  Here is our starting code:

Get-Process |
    Select-Object -first 3 -Property Name, VM, PM

And here is the output:
Name            VM        PM
----            --        --
AcroRd32 472272896 216985600
AcroRd32 132603904  10866688
ApMsgFwd  96550912   1626112

We will now add Tee-Object and send the data to another cmdlet.

Get-Process |
    Select-Object -first 3 -Property Name, VM, PM |
    Tee-Object -FilePath C:\PS\Data1.txt |
    Sort-Object -Property VM

The object was successfully piped to Sort-Object and processed.  We can also read the text file that was created.

PS C:\ps> Get-Content data1.txt

Name            VM        PM
----            --        --
AcroRd32 472272896 216985600
AcroRd32 132603904  10866688
ApMsgFwd  96550912   1626112

The reusability question now comes into play.  What if I needed to use the data that was sent to the text file in a later operation?  Well, it is now text.  That is not information that we could easily reincorporate into a PowerShell object.  Utilizing ConvertTo-CSV and ConvertFrom-CSV we can.  By placing ConvertTo-CSV before Tee-Object and ConvertFrom-CSV after Tee-Object was are able to send a comma separated file to disk.

Get-Process |
    Select-Object -first 3 -Property Name, VM, PM |
    ConvertTo-CSV |
    Tee-Object -FilePath C:\PS\Data1.csv |
    Sort-Object -Property VM

Name     VM        PM      
----     --        --      
AcroRd32 132603904 10866688
AcroRd32 472272896 216985600
ApMsgFwd 96550912  1626112

The screen out shows the Sort-Object successfully worked with the object in the PowerShell pipeline.  Import-CSV is also able to successfully read the object back into member from disk.

PS C:\ps> Import-Csv -Path Data1.csv

Name     VM        PM      
----     --        --      
AcroRd32 472272896 216985600
AcroRd32 132603904 10866688
ApMsgFwd 96550912  1626112  

Let’s take a look at using the –Variable parameter of Tee-Object.

Get-Process |
    Select-Object -first 3 -Property Name, VM, PM |
    Tee-Object -Variable Data |
    Sort-Object -Property VM

Here we removed the –FilePath parameter and added the –Variable parameter.  This is a formal usage of a variable so we omit the $ in front of the variable name. Here is the contents of the variable
PS C:\ps> $Data

Name            VM        PM
----            --        --
AcroRd32 472272896 216985600
AcroRd32 132603904  10866688
ApMsgFwd  96550912   1626112

More importantly, here is the object:

PS C:\ps> $Data | Get-Member

   TypeName: Selected.System.Diagnostics.Process

Name        MemberType   Definition                   
----        ----------   ----------                   
Equals      Method       bool Equals(System.Object obj)
GetHashCode Method       int GetHashCode()            
GetType     Method       type GetType()               
ToString    Method       string ToString()            
Name        NoteProperty string Name=AcroRd32         
PM          NoteProperty long PM=216985600            
VM          NoteProperty long VM=472272896  

The information stored in $data is retained as an object.

So the question now comes up as to when to use the –FilePath parameter and when to use the –Variable parameter. If you will consume the data that was removed from the pipeline within the same scope of memory, use the –Variable parameter.  If you will consume the data in another scope of memory, such as another cmdlet or a different script, the use the –FilePath parameter

Thursday, May 5, 2016

Line Continuation in PowerShell – The Big Debate

A common point of confusion for IT Pros learning PowerShell is “when can I press the Enter Key?”  Line continuation allows us to prevent our code for moving horizontally off the screen.  When someone is reading your code and they must slide the horizontal scroll bar, well that is just annoying.  It makes your code less readable.  Let’s look at various opportunities that you have to write less annoying code.

Let me start out with rule #1 when it comes to line continuation:
Get your code to work first!!!

What this means is that you write it on one line before trying to break it up.  If you start experimenting with how to break the line up without evening knowing if it works, you will be generating a lot of very difficult to resolve errors. Get your code to work first.

The Backtick
Let’s tackle the big one, the backtick.  It is also known as the grave accent. Depending on who you ask, they either love it or hate it. I personally love it, but only when you use it consistently.  Take a look at this code.

New-ADFineGrainedPasswordPolicy -Name "Group1PSO" -Precedence 10 -ComplexityEnabled $True -Description "PSO for Group 1" -DisplayName"Group1PSO" -LockoutDuration "0.12:00:00" -LockoutObservationWindow "0.00:15:00" -LockoutThreshold 3 -MaxPasswordAge "10.00:00:00" -MinPasswordAge "1.00:00:00" -MinPasswordLength 8 -PasswordHistoryCount 10 -ReversibleEncryptionEnabled $False

This example code is 1 PowerShell cmdlet with its parameters. Just in case you are interested:
PS C:\Users\JASON> 'New-ADFineGrainedPasswordPolicy -Name "Group1PSO" -Precedence 10 -ComplexityEnabled $True -Description "PSO for Group 1" -DisplayName"Group1PSO" -LockoutDuration "0.12:00:00" -LockoutObservationWindow "0.00:15:00" -LockoutThreshold 3 -MaxPasswordAge "10.00:00:00" -MinPasswordAge "1.00:00:00" -MinPasswordLength 8 -PasswordHistoryCount 10 -ReversibleEncryptionEnabled $False'|
Measure-Object -Character -Word -IgnoreWhiteSpace

Lines Words Characters Property
----- ----- ---------- --------
         29        346   

346 characters.  I’m confident that this command will require horizontal scrolling if we do not address the issue.  The Backtick (  `  ) is the PowerShell escape character.  It is not the single quote.  On the US keyboard, it is on the left hand side.  Do not confuse it with the single quote. The 2 draw backs that the backtick has is that it is both very hard to see, and is easily confused with the single quote.  When I introduce its usage in class, I am very clear that no one is to press anything on the right hand side of the keyboard as we perform this exercise together.  Unfortunately 9 times or of 10, someone does.

The backtick is acceptable to use if you use it in a consistent fashion so your colleagues will clearly understand when and why you used it.  Here is an example of what the above sample code looks like when the backtick is used properly:

New-ADFineGrainedPasswordPolicy -Name "Group1PSO" `
 -Precedence 10 `
 -ComplexityEnabled $True `
 -Description "PSO for Group 1" `
 -DisplayName "Group1PSO" `
 -LockoutDuration "0.12:00:00" `
 -LockoutObservationWindow "0.00:15:00" `
 -LockoutThreshold 3 `
 -MaxPasswordAge "10.00:00:00" `
 -MinPasswordAge "1.00:00:00" `
 -MinPasswordLength 8 `
 -PasswordHistoryCount 10 `
 -ReversibleEncryptionEnabled $False

Notice the nice clear alignment of the parameters.  They are slightly indented, but left aligned.  With consistent use, I have no issue with the backtick. Just remember these 4 rules.
1.       Use it in a consistent manor.
2.       Always proceed it with a space.
3.       Always press ENTER right after using it.
4.       Never add a backtick to the last line of a command.

A very common way to avoid the backtick is to use splatting.  Splatting involves the usage of a hash table.  The hash table has two elements.  The first is the name of the parameter and the second is its value.  Take a look:

  $Params = @{
    Name = "Group1PSO"
    Precedence = 10
    ComplexityEnabled = $True
    Description = "PSO for Group 1"
    DisplayName = "Group1PSO"
    LockoutDuration = "0.12:00:00"
    LockoutObservationWindow = "0.00:15:00"
    LockoutThreshold = 3
    MaxPasswordAge = "10.00:00:00"
    MinPasswordAge = "1.00:00:00"
    MinPasswordLength = 8
    PasswordHistoryCount = 10
    ReversibleEncryptionEnabled = $False}

 New-ADFineGrainedPasswordPolicy @Params

We create a variable with a name of our choice.  I called this one $Params.  We set it equal to a hash table.  In PowerShell, we start a hash table with  @{   and end it with  }  .  Each line contains a key/value pair.  We need to use the parameters of the cmdlet that we will be splatting to as our keys or we will have an error. The first key/value pair in the example sets the value of the “Name” parameter to “Group1PSO”.  The second Key/Value pair sets the “Precedence” parameter to a value of “10”
To use the splat, we call the cmdlet New-ADFineGrainPasswordPolicy and then provide the splat.  Notice that we do not use the “$” when providing the splat, only when creating it. We use the “@” and then the variable name.

New-ADFineGrainedPasswordPolicy @Params

The Splat method has the advantage of no backtick!!!  You will not be working with a hard to see character.

The disadvantage, you need to know the parameters of the cmdlet.  When using the backtick, you could still use TAB completion and the intellisense of the ISE.  With splatting, you often need to write the command out normally, and then cut and past the parameter names and values into the hash table and then properly format them.

Using the Pipe Character
A key feature of PowerShell is its ability to chain together a series of cmdlets into a working set of code.  We do this through piping.  With piping, we can significantly reduce our coding requirements.
Take a look at this code:

Get-EventLog -LogName Security -InstanceId 4624 | Select-Object -Property TimeWritten, InstanceID, Message | ConvertoTo-HTML | Out-File -FilePath c:\ps\SecLog.html

This code is utilizing 4 commands.  The problem is that it is requiring me to horizontal scroll in the ISE.  Here is the same code where I pressed the ENTER key after each pipe.

Get-EventLog -LogName Security -InstanceId 4624 |
    Select-Object -Property TimeWritten, InstanceID, Message |
    ConvertoTo-HTML |
    Out-File -FilePath c:\ps\SecLog.html

The code is still considered a single line, just more readable.  I also add a TAB to each line under the start of the first cmdlet in the piped statement.  This is a visual cue to me that these lines of code are part of the Get-EventLog piped statement.  That way I do not need to look at the end of the line to see if they are piped.

Using Commas
Many parameters can have multiple values provided to them.  To discover which parameters can accept multiple values, you need to look at the help file.  Here is the syntax block from the help file of Get-Service.

     Get-Service [[-Name] []] [-ComputerName []] [-DependentServices]
    [-Exclude []] [-Include []] [-InformationAction {SilentlyContinue
    | Stop | Continue | Inquire | Ignore | Suspend}] [-InformationVariable
    []] [-RequiredServices] []

Take notice of the –Name parameter.  Its data type is written as [String[]]  The key element to look for is the “[]”  That extra set of characters of open and close braces right after the data type tells you that this parameter can receive multiple values. Here is the command used without line continuation.

PS C:\> Get-Service -name Bits, WinRM, XboxNetApiSvc, Vmms, Spooler

Status   Name               DisplayName                          
------   ----               -----------                          
Running  Bits               Background Intelligent Transfer Ser...
Running  Spooler            Print Spooler                        
Running  Vmms               Hyper-V Virtual Machine Management   
Running  WinRM              Windows Remote Management (WS-Manag...
Stopped  XboxNetApiSvc      Xbox Live Networking Service  

And here it is with line continuation.

Get-Service -name Bits,

A better example may be made when using Select-Object, which generally produces some very long commands.

      Get-ADComputer –Filter * |
       Select-Object -Property DistinguishedName, SID, SamAccountName, Enable, @{n='ComputerName';e={$PSItem.Name}}

This one is getting out of hand.  We can press ENTER after any of the commas from the argument list of Select-Object’s –Property parameter

      Get-ADComputer –Filter * |
       Select-Object -Property DistinguishedName,

Using Periods
When we are using methods, we can actually separate them on multiple lines.
First off, let’s get some data to work with.
$Data = Get-EventLog -LogName Security -Newest 1 -InstanceId 4634 |
    Select-Object -ExpandProperty Message

This will store the message component of an event in the variable $Data.  Go ahead and look at the contents of this variable.  It is 1 continuous string.  This is actually one of the exercises that I do in my PowerShell class on the final day.  The objective is to get the Account Name listed on the 5th line.  Here is the code written on one line using the methods of the System.String object.

There are much simpler ways to solve this problem. I choose this way because it is long. Here is the same code with the methods separated on different line.


If this is more readable or not is a subject of debate, but it does show you another way to prevent horizontal scrolling.  If you really wanted to push it a bit too far:


No, I would not actually write my code to look like this.

Hash Tables with Select-Object and Format-Table
We can also provide for line continuation when creating custom properties with Select-Object and custom columns with Format-Table.
Take a look at this code:

Get-Process | Format-Table -Property Name, CPU, @{Name = "VMWS"; Expression = {$_.VM + $_.WS}; align="Left"}

You can inject some carriage returns after each element of the hash table.

Get-Process | Format-Table -Property Name, CPU,
        Name = "VMWS"
        Expression = {$_.VM + $_.WS}

You can also remove the semi-colon from the hash table when the elements are on different lines, or leave them in.

Comparison Operators
Comparison operators gives us another opportunity to break our long lines of code into multiple lines.
Consider this code:
Get-Volume |
    Where-Object -filter {$_.SizeRemaining -gt 0 -and $_.SizeRemaining / $_.Size -lt .99 -and $_.FileSystemLabel -ne ""}

Add a carriage return after each logical operator.
Get-Volume |
    Where-Object -filter {$_.SizeRemaining -gt 0 -and
                          $_.SizeRemaining / $_.Size -lt .99 -and
                          $_.FileSystemLabel -ne ""}

It also works after the comparison operators, but it may be a bit unreadable if you do.
Get-Volume |
    Where-Object -filter {$_.SizeRemaining -gt
                          0 -and
                          $_.SizeRemaining / $_.Size -lt 
                          .99 -and
                          $_.FileSystemLabel -ne

As the debate rages on, there is a request on Microsoft Connect to allow for a syntax change to help remove the backtick ( and provide for a different syntax for line continuation using the pipe character.  Below will not work, but it is what is proposed.

Get-EventLog -LogName Security -InstanceId 4624 |
    Select-Object -Property TimeWritten, InstanceID, Message |
    ConvertoTo-HTML |
    Out-File -FilePath c:\ps\SecLog.html


Get-EventLog -LogName Security -InstanceId 4624
    | Select-Object -Property TimeWritten, InstanceID, Message
    | ConvertoTo-HTML
    | Out-File -FilePath c:\ps\SecLog.html

The proposed syntax change would allow for the pipe to be placed at the beginning of the next line, as opposed to the end.

Choose which method you will be using, but make it a team decision. In the end, your goal is to make your code readable.