That’s not a post, at least in the classical sense 😊 Rather it’s a collection of small scripts, that I will keep updating, for me to find easily. No rocket science, just small everyday stuff that I find myself googling again and again.
Powershell: Basic CATCH block
$ErrorActionPreference='Stop'
Clear-Host
# PLEASE PLEASE PLEASE do not write logs where the program is
# See https://dandraka.com/2021/11/04/dont-write-logs-in-program-files/
$errorLogFolder = "C:\logs\"
$errorFile = Join-Path $errorLogFolder "$([System.IO.Path]::GetFileNameWithoutExtension($PSCommandPath))_errors_$([guid]::NewGuid().ToString().Split("-")[0]).txt"
$errorList = New-Object -TypeName System.Collections.ArrayList
try {
# say you're processing a file or whatever
$fileName = "C:\data\lalala.txt"
# throw an error to show what the log looks like
$i = 3/0
}
catch {
$errorMsg = "Error while processing file '$($fileName)'`nException $($_.Exception.GetType().FullName): $($_.Exception.Message)`nException location:`n$($_.InvocationInfo.ScriptName)`n$($_.InvocationInfo.ScriptLineNumber):$($_.InvocationInfo.Line)"
# you can add the error(s) to a list and report at the end
$errorList.Add($errorMsg) | Out-Null
# or, alternatively, immediately write it into a file
Out-File -FilePath $errorFile -InputObject $errorMsg
Write-Warning $errorMsg
}
The error log produced looks like this:
Error while processing file 'C:\data\lalala.txt'
Exception System.Management.Automation.RuntimeException: Attempted to divide by zero.
Exception location:
C:\code\MyDataProcessingScript.ps1
11: $i = 3/0
An important and often overlooked point, and the reason I updated this cheat sheet, is $_.InvocationInfo.ScriptName. That’s important because sometimes the error doesn’t happen on your main script, but in a script or module that the main script calls. So if all you have is a line number you’re left scratching your head wondering, say, how can an IO error happen in a line that splits a string -because you’re looking at the wrong script.
Powershell: Get the first X bytes of a file.
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, September 2022
#
# ===== Change here =====
$path = 'C:\somepath\somehugelogfile.log'
$pathOut = 'C:\temp\sneakpeak.txt'
$numBytes = 10000
# =======================
$ErrorActionPreference='Stop'
Clear-Host
$bytes = Get-Content -Path $path -Encoding byte -TotalCount $numBytes
$str = [System.Text.Encoding]::UTF8.GetString($bytes)
Out-File -FilePath $pathOut -InputObject $str
Powershell: Find json files with a duplicate attribute (same value in multiple files)
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, January 2025
#
# Modify the path and the attribute (SomeAttribute) as needed in the highlighted lines
$path = 'C:\somepath'
$fileList = Get-ChildItem -Path $path -Filter '*.json'
$valueIndex = [System.Collections.Generic.Dictionary[string,string]]::new(); $fileList | % { $json = (Get-Content -Path $_.FullName | ConvertFrom-Json -AsHashtable); $valueIndex.Add($_.FullName, $json.SomeAttribute) }
$duplicateValueList = [System.Collections.Generic.List[string]]::new(); $valueIndex.Values | Group | Where{$_.Count -gt 1} | % { $duplicateValueList.Add($_.Name) }
$valueIndex.Keys | % { if ($duplicateValueList.Contains($valueIndex[$_])) { Write-Host "File = $($_) `t Value = $($valueIndex[$_])" } }
Powershell: Remove files older than X days recursively.
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, September 2022
#
# ===== Change here =====
$path = 'C:\somepath\'
$filter = '*.xml'
$numDays = 30
# =======================
$ErrorActionPreference='Stop'
Clear-Host
Get-ChildItem -Path $path -Filter $filter -Recurse | Where-Object {($_.LastWriteTime -lt (Get-Date).AddDays($numDays * -1))} | Remove-Item
Powershell: Change information in XML files en masse.
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, September 2022
#
# ===== Change here =====
$dir = 'C:\somepath\'
# =======================
$ErrorActionPreference='Stop'
Clear-Host
$list = Get-ChildItem -Path $dir -Filter '*.xml'
foreach($file in $list) {
[xml]$xml=Get-Content -Path $file.FullName -Encoding UTF8
# customize the XML paths below
$customerName = $xml.PrintJob.DocumentHeader.CustomerName
if ([string]::IsNullOrWhiteSpace($customerName )) {
continue
}
$xml.PrintJob.DocumentBody.RecipientName = $customerName
$xml.Save($file.FullName)
}
Powershell: Change encoding of files.
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, October 2024
#
# !!! Prerequisite: Powershell 7 or later
# Does not work with Powershell 5.x
# ===== Change here =====
$dir = 'C:\somepath\'
$filter = '*.json'
$fromEncoding = 'utf8BOM'
$toEncoding = 'utf8'
# =======================
$ErrorActionPreference='Stop'
Clear-Host
Get-ChildItem -Path $dir -Filter $filter | % { $c = Get-Content -Path $_.FullName -Encoding $fromEncoding; $c | Set-Content -Encoding $toEncoding -Path $_.FullName }
Windows command line: Change permissions of all files and directories in a path recursively.
CD C:\somepath
FOR /D %o IN (*.*) DO echo Y| cacls %o /T /G "NT AUTHORITY\Authenticated Users":F
Here “NT AUTHORITY\Authenticated Users” stands for the authenticated users group of the local machine; “F” stands for Full Permissions.
Linux command line: copy file from one PC to another
# scp -r /path/to/file USERNAME@IP_OF_TARGET:/path/to/dir
scp -r /home/dimitris/Downloads/Win11.iso dimitris@192.168.0.5:/home/dimitris/Downloads
Powershell: Create and use a dictionary
# dictionary definition
$myIndex = New-Object -TypeName 'System.Collections.Generic.Dictionary[string,string]'
# add some values
$myIndex.Add([guid]::NewGuid().ToString(),'lalala')
$myIndex.Add([guid]::NewGuid().ToString(),'hohoho')
# iterate through keys and values
foreach($myId in $myIndex.Keys) {
$myValue = $myIndex[$myId]
Write-Host "Id = '$myId' Value = '$myValue'"
}
Powershell: Get info from multiple XML files and write into CSV
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, October 2022
#
# The file filter here is based on date from-to,
# but obvs you can change Where-Object to match size or name or whatever.
# The CSV separator was chosen to be tab (`t) because this is
# very Excel-friendly.
# ===== Change here =====
$path = 'E:\somepath'
$outPath = "C:\temp\out-$([guid]::NewGuid().ToString().Split('-')[0]).csv"
$strDateTimeFrom = "2022-09-29 18:00:00"
$strDateTimeTo = "2022-09-29 20:00:00"
# =======================
Clear-Host
$ErrorActionPreference='Stop'
[DateTime]$dateTimeFrom = [DateTime]::Parse($strDateTimeFrom)
[DateTime]$dateTimeTo = [DateTime]::Parse($strDateTimeTo)
$filesList = Get-ChildItem -Path $path -Recurse -Filter '*.xml' | Where-Object { ($_.LastWriteTime -gt $dateTimeFrom) -and ($_.LastWriteTime -lt $dateTimeTo) }
$dataList = New-Object -TypeName 'System.Collections.Generic.List[string]'
# change this to match the XML info below
$dataList.Add("CompanyName`tInvoiceId`tTemplateId`tDocumentId`tFileName")
foreach($file in $filesList) {
$fileFullName = $file.FullName
$fileName = $file.Name
[xml]$xml = Get-Content -Path $fileFullName
# that's the part where you specify what info you need from the XML
# my XMLs have multiple Document nodes per file, that's why I need a loop
foreach($document in $xml.PrintJob.Documents.Document) {
$documentId = $document.DocumentHeader.DocumentId
$templateId = $document.DocumentHeader.TemplateId
$invoiceId = $document.ArchiveAttributes.InvoiceId
$custName = $document.DocumentHeader.Addresses.Recipient.CompanyName
$dataList.Add("$custName`t$invoiceId`t$templateId`t$documentId`t$fileName")
}
}
Out-File -FilePath $outPath -Encoding utf8 -InputObject $dataList
Write-Host "Finished, $($filesList.Count) files processed"
Powershell: Copy XML files depending on their data plus data from a database
Clear-Host
$ErrorActionPreference='Stop'
# === Parameters ===
$path = 'D:\dataFilesPath'
$copyPath = 'D:\copyFilesPath'
$dbServer = 'mySqlServer'
$dbName = 'Customers'
$sql = 'SELECT CustomerName FROM Customers WHERE Active=1'
# === Parameters ===
$res = Invoke-Sqlcmd -ServerInstance $dbServer -Database $dbName -Query $sql
$exclusionList = New-Object -TypeName 'System.Collections.Generic.List[string]'
# if a different field of the query is needed, change [0]
$res | % { $exclusionList.Add($_[0].ToLowerInvariant().Trim()) }
if (-not (Test-Path -Path $copyPath)) { New-Item -ItemType Directory -Path $copyPath} $list = Get-ChildItem -Path $path -Filter '*.xml' -Recurse
$cnt=0
foreach($file in $list) {
[xml]$contents = Get-Content -Path $file.FullName -Encoding UTF8
# obvs you'll need to change the XPATH
# to match your XML structure
$customerName = $contents.Data.CustomerData.Customer_Name.'#cdata-section'.ToLowerInvariant().Trim()
if ($exclList.Contains($customerName)) {
continue
}
Write-Host "Found $customerName"
Copy-Item -Path $file.FullName -Destination $copyPath
$cnt++
}
Write-Host "Finished, $cnt files copied to $copyPath"
Chocolatey: My dev machine install list
choco install notepadplusplus
choco install winmerge -y
choco install vscode -y
choco install vscode-powershell -y
choco install vscode-csharp -y
choco install vscode-gitlens -y
choco install git -y
choco install tortoisegit -y
choco install svn -y
choco install tortoisesvn -y
choco install postman -y
choco install soapui -y
choco install sql-server-management-studio -y
choco install intellijidea-community
choco install openjdk8
choco install visualstudio2019professional --package-parameters " --add Microsoft.VisualStudio.Workload.Azure --add Microsoft.VisualStudio.Workload.ManagedDesktop --add Microsoft.VisualStudio.Workload.NetCoreTools --add Microsoft.VisualStudio.Workload.NetWeb --add Microsoft.VisualStudio.Workload.Universal --includeRecommended --includeOptional --passive --locale en-US" -y
choco install visualstudio2022professional --package-parameters " --add Microsoft.VisualStudio.Workload.Azure --add Microsoft.VisualStudio.Workload.ManagedDesktop --add Microsoft.VisualStudio.Workload.NetCoreTools --add Microsoft.VisualStudio.Workload.NetWeb --add Microsoft.VisualStudio.Workload.Universal --includeRecommended --includeOptional --passive --locale en-US" -y
choco install dotnet-5.0-sdk -y
choco install dotnet-6.0-sdk -y
choco install ServiceBusExplorer -y
Install-Package \\fileserver\share\JamsScheduler\SetupClientx64.msi
Powershell: Keep the lights on
Clear-Host
Add-Type -AssemblyName System.Windows.Forms
Write-Host 'Starting...'
$WShell = New-Object -com "Wscript.Shell"
while ($true)
{
$WShell.sendkeys("{F16}")
Start-Sleep -Seconds 180
}
Powershell: Quickly format (“pretty print”) XML files
cd C:\mydata
gci *.xml | % { [xml]$x=get-content $_ ; $x.Save($_.FullName) }
Powershell: Archive files (zip + delete)
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, November 2022
#
# Prerequisite: 7zip is installed in the system.
# =================================
# This script zips everything found in $archiveName that
# has a ModifiedDate after $dateFromStr and matches $filter.
# Inside $archivePath, it creates one dir per run and
# inside this, one 7zip file per month plus one txt file
# that has the 7zip contents.
# E.g.
# C:\OldLogFiles
# Run-LogFileArchive-20221122-112015
# Archive-LogFileArchive-20200301-20200401.7z.001
# Archive-LogFileArchive-20200301-20200401.txt
# Archive-LogFileArchive-20200401-20200501.7z.001
# Archive-LogFileArchive-20200401-20200501.txt
# (etc etc)
# - The names of the run dirs (Run-LogFileArchive-20221122-112015) are
# always Run-[archiveName]-[current date-time].
# - The names of the archives are Archive-[archiveName]-[From date]-[To date].7z.
# - Obviously the script will only generate 7z files for the months
# where it finds files.
# =================================
Clear-Host
$ErrorActionPreference = 'Stop'
try
{
# ===== Parameters - Change here =====
# A short description for the archive. This is added to the filenames.
$archiveName = "LogFileArchive"
# Directory to create the archive in
$archivePath = "C:\OldLogFiles"
# Directory to archive files from
$path = "C:\logs"
# Filter for files to archive, for example *.*, *.log or *.pdf
$filter = "*.log"
# How many months of files to keep (i.e. not archive), for example 12 (1 year).
$monthsToKeep = 1
# From-date to archive, e.g. '2020-12-31'
# If $deleteFiles = $true you don't need to change this ever.
$dateFromStr = "1900-01-01"
# Delete files and empty folders after archiving?
$deleteFiles = $true
# Path of 7zip command line
$zip = "C:\Program Files\7-Zip\7z.exe"
# ===== Parameters =====
if ([string]::IsNullOrWhitespace($filter))
{
$filter = "*.*"
}
if ($monthsToKeep -le 0)
{
throw "Months to keep cannot be 0 or negative"
}
if ([string]::IsNullOrWhitespace($dateFromStr))
{
throw "Date From cannot be empty"
}
$dateToStr = [datetime]::Today.AddMonths($monthsToKeep * -1).ToString("yyyy-MM-01")
Write-Host "Delete files set to $deleteFiles"
# ===== Sanity checks =====
if ([string]::IsNullOrWhitespace($archiveName)) { throw "Parameter archiveName cannot be empty" }
if ([string]::IsNullOrWhitespace($archivePath)) { throw "Parameter archivePath cannot be empty" }
if ([string]::IsNullOrWhitespace($path)) { throw "Parameter path cannot be empty" }
if ([string]::IsNullOrWhitespace($zip)) { throw "Parameter sevenZipPath cannot be empty" }
if (-not(Test-Path -Path $archivePath)) { throw "Archive path $archivePath does not exist" }
if (-not(Test-Path -Path $path)) { throw "Root path $path does not exist" }
if (-not(Test-Path -Path $zip)) { throw "7zip not found in $zip" }
$archivePath = [System.IO.Path]::Combine($archivePath, "Run-$archiveName-$([datetime]::Now.ToString("yyyyMMdd-HHmmss"))")
# Loop through months
$dateFrom = [datetime]::Parse($dateFromStr)
$dateTo = [datetime]::Parse($dateToStr)
$dateFromLoop = $dateFrom
$loop = $true
$haveArchivedFiles = $false
if (Test-Path -Path $archivePath)
{
throw "Directory $archivePath already exists, stopping out of precaution"
}
New-Item -ItemType Directory -Path $archivePath | Out-Null
$fullList = Get-ChildItem -Path $path -Filter $filter -File -Recurse `
| Where-Object { ($_.LastWriteTime -ge $dateFrom) -and ($_.LastWriteTime -lt $dateTo) }
while($loop)
{
$dateToLoop = $dateFromLoop.AddMonths(1)
if ($dateToLoop -gt $dateTo)
{
$dateToLoop = $dateTo
$loop = $false
}
$archiveFile = [System.IO.Path]::Combine($archivePath, "Archive-$archiveName-$($dateFromLoop.ToString("yyyyMMdd"))-$($dateToLoop.ToString("yyyyMMdd")).7z")
#Write-Host $archiveFile
$archiveList = $archiveFile.Replace(".7z", ".txt")
$list = $fullList | Where-Object { ($_.LastWriteTime -ge $dateFromLoop) -and ($_.LastWriteTime -lt $dateToLoop) }
if ($list.Count -gt 0)
{
$haveArchivedFiles = $true
$list | % { Out-File -FilePath $archiveList -Encoding utf8 -Append -InputObject "$($_.FullName)" }
$cmd = $zip
Write-Host "================ Archiving files from $path to $archiveFile ================"
$params = "a $archiveFile -spf -ssw -stl -v2g -mx2 @$archiveList"
& "$cmd" $params.Split(" ")
# 7z parameter -sdel instructs 7zip to delete files after archiving
# $params = "a $archiveFile -sdel -spf -ssw -stl -v2g -mx2 @$archiveList"
# BUUUUUUUUT there's an open 7z bug which is that -sdel doesn't work
# with file lists (which we need here)
# that's why we need to delete the files with powershell after 7z
if ($deleteFiles) {
$list | % { Remove-Item -Force -ErrorAction Continue -Path "$($_.FullName)" }
Write-Host "Deleted $($list.Count) files"
}
}
$dateFromLoop = $dateToLoop
}
if (-not $haveArchivedFiles)
{
Write-Host "================ No files found to archive ================"
}
}
catch
{
Write-Host "================ An error occured ================"
Write-Error $_
}
Git: Add existing code to repo
# See also https://docs.github.com/en/migrations/importing-source-code/using-the-command-line-to-import-source-code/adding-locally-hosted-code-to-github
cd C:\mysourcecode
git init -b main
git add .
git commit -m "initial commit"
git remote add origin https://mycodeprovider/myrepo
git push -u origin --all
Powershell: 1-liner to create azure devops project and repos from zip files
Prerequisite: git config and az login have been completed.
cls; $proj=(Split-Path -Path $pwd -Leaf); az devops project create --name $proj; gci *.zip | % { Expand-Archive $_.FullName }; gci -Directory | % { cd $_.FullName; $dir=(Split-Path -Path $pwd -Leaf); az repos create --project $proj --name $dir; git init -b main ; git add . ; git commit -m "initial commit" ; git remote add origin https://MYUSERNAME@dev.azure.com/MYUSERNAME/$proj/_git/$dir ; git push -u origin --all }
Powershell: Zip directory and generate hash
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, April 2023
#
Clear-Host
$ErrorActionPreference = 'Stop'
# customize here
$releaseId = [guid]::NewGuid().ToString().Split('-')[0]
$SourceDir = "$PSScriptRoot\source\"
$OutputFile = 'mypackage.zip'
$DestinationPath = "$PSScriptRoot\deploy\$($releaseId)"
$tempPath = "$($env:TEMP)\$releaseId"
New-Item -ItemType Directory -Path $DestinationPath
New-Item -ItemType Directory -Path $tempPath
# to avoid file lock issues
Copy-Item -Recurse -Path $SourceDir -Destination $tempPath
$compressParams = @{
Path = "$tempPath\source\*"
CompressionLevel = 'Optimal'
DestinationPath = Join-Path $DestinationPath $OutputFile
Force = $true
}
Compress-Archive @compressParams
(Get-FileHash $compressParams.DestinationPath).Hash | Out-File $(Join-Path $DestinationPath $OutputFile.Replace('.zip','_hash.txt'))
Write-Host "Finished"
Start-Sleep -Seconds 30