The question that I took this evening involved a user who has some functional PowerShell code. The needed a way to filter out any duplicate objects. They were looking at using several loops. In my college days, that would have been the answer. As my favorite professor, Dan Matthews, put it, “Never Reinvent the Wheel”
PowerShell has built in functionality to remove duplicate items by using the Sort-Object cmdlet. You need to use the –Property parameter to tell PowerShell which proper of the object you are looking for duplicates on and also you need to use the –Unique parameter to tell PowerShell to only leave unique objects (remove duplicates).
Below is some sample code to generate a set of 11 objects. Two of those objects will have a duplicate value in Prop1.
# Create a dynamic array to hold the test objects.
$Array = @()
# Create 10 objects in the dynamic array.
For ($X = 0;$X -lt 10;$X++)
{
$Obj = New-Object PSObject
$Obj | Add-Member -MemberType NoteProperty -Name "Prop1" -Value $X
$Obj | Add-Member -MemberType NoteProperty -Name "Prop2" -Value $True
$Obj | Add-Member -MemberType NoteProperty -Name "Prop3" -Value $False
# Add the object to the array
$Array += $Obj
}
# Add a duplicate to the array. The duplication will be on the property
# Prop1 with a value of 3
$Obj = New-Object PSObject
$Obj | Add-Member -MemberType NoteProperty -Name "Prop1" -Value 3
$Obj | Add-Member -MemberType NoteProperty -Name "Prop2" -Value $True
$Obj | Add-Member -MemberType NoteProperty -Name "Prop3" -Value $False
# Add the object to the array
$Array += $Obj
# View the contents of the array. Notice the duplicate value in Prop 1
$Array | FT –autosize
# Divider between the two sets of data.
Write-host "-----------------------------------------------------------"
# Use the -Unique parameter of the Sort-Object cmdlet.
# Note the duplicate object is gone.
$Array | Sort-Object -Property Prop1 -Unique | FT -autosize
This output will display two sets of data. One set will show all 11 objects (including the duplicate). The second set was filtered using Select-Object in the last line of the code.
Prop1 Prop2 Prop3
----- ----- -----
0 True False
1 True False
2 True False
3 True False
4 True False
5 True False
6 True False
7 True False
8 True False
9 True False
3 True False
-----------------------------------------------------------
Prop1 Prop2 Prop3
----- ----- -----
0 True False
1 True False
2 True False
3 True False
4 True False
5 True False
6 True False
7 True False
8 True False
9 True False
Notice in the second set that was filtered with Sort-Object, the duplicate value of 3 in Prop1 has been removed.
Comments