Software

My script cheat sheet

1 Comment

That’s not a post, at least in the classical sense 😊 Rather it’s a collection of small scripts, that I will keep updating, for me to find easily. No rocket science, just small everyday stuff that I find myself googling (or, more recently, AI-ing) again and again.

Install dotnet as a user unattended

Installs dotnet (major version given in $dotnetVersion) to C:\app\dotnet as a user, no admin rights needed. For list of versions, see dotnet core relases in github.

PowerShell
$dotnetVersion = '8.0'; $env:DOTNET_ROOT='C:\app\dotnet'; (New-Object System.Net.WebClient).DownloadFile('https://dot.net/v1/dotnet-install.ps1', 'dotnet-install.ps1'); ./dotnet-install.ps1 -Channel $dotnetVersion -InstallDir $env:DOTNET_ROOT

NOTE: without admin rights the PATH variable isn’t updated. Therefore, to use it, you need to run C:\app\dotnet\dotnet.exe instead of just dotnet.exe.

Powershell: Basic CATCH block

PowerShell
$ErrorActionPreference='Stop'
Clear-Host
# PLEASE PLEASE PLEASE do not write logs where the program is
# See https://dandraka.com/2021/11/04/dont-write-logs-in-program-files/
$errorLogFolder = "C:\logs\"
$errorFile = Join-Path $errorLogFolder "$([System.IO.Path]::GetFileNameWithoutExtension($PSCommandPath))_errors_$([guid]::NewGuid().ToString().Split("-")[0]).txt" 
$errorList = New-Object -TypeName System.Collections.ArrayList
try {
    # say you're processing a file or whatever
    $fileName = "C:\data\lalala.txt"
    # throw an error to show what the log looks like
    $i = 3/0
}
catch {
    $errorMsg = "Error while processing file '$($fileName)'`nException $($_.Exception.GetType().FullName): $($_.Exception.Message)`nException location:`n$($_.InvocationInfo.ScriptName)`n$($_.InvocationInfo.ScriptLineNumber):$($_.InvocationInfo.Line)"
    # you can add the error(s) to a list and report at the end
    $errorList.Add($errorMsg) | Out-Null
    # or, alternatively, immediately write it into a file
    Out-File -FilePath $errorFile -InputObject $errorMsg
    Write-Warning $errorMsg      
}

The error log produced looks like this:

Error while processing file 'C:\data\lalala.txt'
Exception System.Management.Automation.RuntimeException: Attempted to divide by zero.
Exception location:
C:\code\MyDataProcessingScript.ps1
11:    $i = 3/0

An important and often overlooked point, and the reason I updated this cheat sheet, is $_.InvocationInfo.ScriptName. That’s important because sometimes the error doesn’t happen on your main script, but in a script or module that the main script calls. So if all you have is a line number you’re left scratching your head wondering, say, how can an I/O error happen in a line that splits a string -because you’re looking at the wrong script.

Powershell: Get the first X bytes of a file.

PowerShell
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, September 2022
#
# ===== Change here =====
$path = 'C:\somepath\somehugelogfile.log'
$pathOut = 'C:\temp\sneakpeak.txt'
$numBytes = 10000
# =======================
$ErrorActionPreference='Stop'
Clear-Host
$bytes = Get-Content -Path $path -Encoding byte -TotalCount $numBytes
$str = [System.Text.Encoding]::UTF8.GetString($bytes)
Out-File -FilePath $pathOut -InputObject $str

Powershell: Find json files with a duplicate attribute (same value in multiple files)

Plain text
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, January 2025
#
# Modify the path and the attribute (SomeAttribute) as needed in the highlighted lines
$path = 'C:\somepath'
$fileList = Get-ChildItem -Path $path -Filter '*.json'
$valueIndex = [System.Collections.Generic.Dictionary[string,string]]::new(); $fileList | % { $json = (Get-Content -Path $_.FullName | ConvertFrom-Json -AsHashtable); $valueIndex.Add($_.FullName, $json.SomeAttribute) }
$duplicateValueList = [System.Collections.Generic.List[string]]::new(); $valueIndex.Values | Group | Where{$_.Count -gt 1} | % { $duplicateValueList.Add($_.Name) }
$valueIndex.Keys | % { if ($duplicateValueList.Contains($valueIndex[$_])) { Write-Host "File = $($_) `t Value = $($valueIndex[$_])" } }

Powershell: Remove files older than X days recursively.

PowerShell
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, September 2022
#
# ===== Change here =====
$path = 'C:\somepath\'
$filter = '*.xml'
$numDays = 30 
# =======================
$ErrorActionPreference='Stop'
Clear-Host
Get-ChildItem -Path $path -Filter $filter -Recurse | Where-Object {($_.LastWriteTime -lt (Get-Date).AddDays($numDays * -1))} | Remove-Item

Powershell: Change information in XML files en masse.

PowerShell
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, September 2022
#
# ===== Change here =====
$dir = 'C:\somepath\'
# =======================
$ErrorActionPreference='Stop'
Clear-Host
$list = Get-ChildItem -Path $dir -Filter '*.xml'
foreach($file in $list) {
    [xml]$xml=Get-Content -Path $file.FullName -Encoding UTF8
    # customize the XML paths below
    $customerName = $xml.PrintJob.DocumentHeader.CustomerName 
    if ([string]::IsNullOrWhiteSpace($customerName )) {
        continue
    }
    $xml.PrintJob.DocumentBody.RecipientName = $customerName 
    $xml.Save($file.FullName)
}

Powershell: Change encoding of files.

Plain text
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, October 2024
#
# !!! Prerequisite: Powershell 7 or later
# Does not work with Powershell 5.x
# ===== Change here =====
$dir = 'C:\somepath\'
$filter = '*.json'
$fromEncoding = 'utf8BOM'
$toEncoding = 'utf8'
# =======================
$ErrorActionPreference='Stop'
Clear-Host
Get-ChildItem -Path $dir -Filter $filter | % { $c = Get-Content -Path $_.FullName -Encoding $fromEncoding; $c | Set-Content -Encoding $toEncoding -Path $_.FullName }

Windows command line: Change permissions of all files and directories in a path recursively.

CD C:\somepath
FOR /D %o IN (*.*) DO echo Y| cacls %o /T /G "NT AUTHORITY\Authenticated Users":F

Here “NT AUTHORITY\Authenticated Users” stands for the authenticated users group of the local machine; “F” stands for Full Permissions.

Linux command line: copy file from one PC to another

# scp -r /path/to/file USERNAME@IP_OF_TARGET:/path/to/dir
scp -r /home/dimitris/Downloads/Win11.iso dimitris@192.168.0.5:/home/dimitris/Downloads

Powershell: Create and use a dictionary

PowerShell
# dictionary definition
$myIndex = New-Object -TypeName 'System.Collections.Generic.Dictionary[string,string]'
# add some values
$myIndex.Add([guid]::NewGuid().ToString(),'lalala')
$myIndex.Add([guid]::NewGuid().ToString(),'hohoho')
# iterate through keys and values
foreach($myId in $myIndex.Keys) {
	$myValue = $myIndex[$myId]
	Write-Host "Id = '$myId' Value = '$myValue'"
}

Powershell: Get info from multiple XML files and write into CSV

PowerShell
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, October 2022
#
# The file filter here is based on date from-to, 
# but obvs you can change Where-Object to match size or name or whatever.
# The CSV separator was chosen to be tab (`t) because this is 
# very Excel-friendly.
# ===== Change here =====
$path = 'E:\somepath'
$outPath = "C:\temp\out-$([guid]::NewGuid().ToString().Split('-')[0]).csv"
$strDateTimeFrom = "2022-09-29 18:00:00"
$strDateTimeTo = "2022-09-29 20:00:00"
# =======================
Clear-Host
$ErrorActionPreference='Stop'
[DateTime]$dateTimeFrom = [DateTime]::Parse($strDateTimeFrom)
[DateTime]$dateTimeTo = [DateTime]::Parse($strDateTimeTo)
$filesList = Get-ChildItem -Path $path -Recurse -Filter '*.xml' | Where-Object { ($_.LastWriteTime -gt $dateTimeFrom) -and ($_.LastWriteTime -lt $dateTimeTo) }
$dataList = New-Object -TypeName 'System.Collections.Generic.List[string]'
# change this to match the XML info below
$dataList.Add("CompanyName`tInvoiceId`tTemplateId`tDocumentId`tFileName")
foreach($file in $filesList) {
    $fileFullName = $file.FullName
    $fileName = $file.Name
    [xml]$xml = Get-Content -Path $fileFullName
    # that's the part where you specify what info you need from the XML
    # my XMLs have multiple Document nodes per file, that's why I need a loop
    foreach($document in $xml.PrintJob.Documents.Document) {
        $documentId = $document.DocumentHeader.DocumentId
        $templateId = $document.DocumentHeader.TemplateId
        $invoiceId = $document.ArchiveAttributes.InvoiceId
        $custName = $document.DocumentHeader.Addresses.Recipient.CompanyName
        $dataList.Add("$custName`t$invoiceId`t$templateId`t$documentId`t$fileName")
    }
}
Out-File -FilePath $outPath -Encoding utf8 -InputObject $dataList
Write-Host "Finished, $($filesList.Count) files processed"

Powershell: Copy XML files depending on their data plus data from a database

PowerShell
Clear-Host
$ErrorActionPreference='Stop'
# === Parameters ===
$path = 'D:\dataFilesPath'
$copyPath = 'D:\copyFilesPath'
$dbServer = 'mySqlServer'
$dbName = 'Customers'
$sql = 'SELECT CustomerName FROM Customers WHERE Active=1'
# === Parameters ===
$res = Invoke-Sqlcmd -ServerInstance $dbServer -Database $dbName -Query $sql
$exclusionList = New-Object -TypeName 'System.Collections.Generic.List[string]'
# if a different field of the query is needed, change [0]
$res | % { $exclusionList.Add($_[0].ToLowerInvariant().Trim()) }
if (-not (Test-Path -Path $copyPath)) { New-Item -ItemType Directory -Path $copyPath} $list = Get-ChildItem -Path $path -Filter '*.xml' -Recurse
$cnt=0
foreach($file in $list) {
    [xml]$contents = Get-Content -Path $file.FullName -Encoding UTF8
    # obvs you'll need to change the XPATH 
    # to match your XML structure
    $customerName = $contents.Data.CustomerData.Customer_Name.'#cdata-section'.ToLowerInvariant().Trim()
    if ($exclList.Contains($customerName)) {
        continue
    }
    Write-Host "Found $customerName"
    Copy-Item -Path $file.FullName -Destination $copyPath
    $cnt++
}
Write-Host "Finished, $cnt files copied to $copyPath"

Chocolatey: My dev machine install list

choco install notepadplusplus
choco install winmerge -y 
choco install vscode -y 
choco install vscode-powershell -y 
choco install vscode-csharp -y 
choco install vscode-gitlens -y 
choco install git -y 
choco install tortoisegit -y 
choco install svn -y 
choco install tortoisesvn -y 
choco install postman -y 
choco install soapui -y 
choco install sql-server-management-studio -y
choco install intellijidea-community
choco install openjdk8
choco install visualstudio2019professional --package-parameters " --add Microsoft.VisualStudio.Workload.Azure --add Microsoft.VisualStudio.Workload.ManagedDesktop --add Microsoft.VisualStudio.Workload.NetCoreTools --add Microsoft.VisualStudio.Workload.NetWeb --add Microsoft.VisualStudio.Workload.Universal --includeRecommended --includeOptional --passive --locale en-US" -y
choco install visualstudio2022professional --package-parameters " --add Microsoft.VisualStudio.Workload.Azure --add Microsoft.VisualStudio.Workload.ManagedDesktop --add Microsoft.VisualStudio.Workload.NetCoreTools --add Microsoft.VisualStudio.Workload.NetWeb --add Microsoft.VisualStudio.Workload.Universal --includeRecommended --includeOptional --passive --locale en-US" -y
choco install dotnet-5.0-sdk -y
choco install dotnet-6.0-sdk -y
choco install ServiceBusExplorer -y
Install-Package \\fileserver\share\JamsScheduler\SetupClientx64.msi

Powershell: Keep the lights on

Plain text
Clear-Host
Add-Type -AssemblyName System.Windows.Forms
Write-Host 'Starting...'
$WShell = New-Object -com "Wscript.Shell"
while ($true)
{
  $WShell.sendkeys("{F16}")
  Start-Sleep -Seconds 180
}

Powershell: Quickly format (“pretty print”) XML files

PowerShell
cd C:\mydata
gci *.xml | % { [xml]$x=get-content $_ ; $x.Save($_.FullName) }

Powershell: Archive files (zip + delete)

PowerShell
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, November 2022
#
# Prerequisite: 7zip is installed in the system.
# =================================
# This script zips everything found in $archiveName that 
# has a ModifiedDate after $dateFromStr and matches $filter.
# Inside $archivePath, it creates one dir per run and 
# inside this, one 7zip file per month plus one txt file
# that has the 7zip contents.
# E.g.
# C:\OldLogFiles
#     Run-LogFileArchive-20221122-112015
#         Archive-LogFileArchive-20200301-20200401.7z.001
#         Archive-LogFileArchive-20200301-20200401.txt
#         Archive-LogFileArchive-20200401-20200501.7z.001
#         Archive-LogFileArchive-20200401-20200501.txt
#         (etc etc)
# - The names of the run dirs (Run-LogFileArchive-20221122-112015) are
#   always Run-[archiveName]-[current date-time].
# - The names of the archives are Archive-[archiveName]-[From date]-[To date].7z.
# - Obviously the script will only generate 7z files for the months 
#   where it finds files.
# =================================
Clear-Host
$ErrorActionPreference = 'Stop'
try
{
    # ===== Parameters - Change here =====
    # A short description for the archive. This is added to the filenames.
    $archiveName = "LogFileArchive"
    # Directory to create the archive in
    $archivePath = "C:\OldLogFiles"
    # Directory to archive files from
    $path = "C:\logs"
    
    # Filter for files to archive, for example *.*, *.log or *.pdf
    $filter = "*.log"
    # How many months of files to keep (i.e. not archive), for example 12 (1 year).
    $monthsToKeep = 1
    # From-date to archive, e.g. '2020-12-31'
    # If $deleteFiles = $true you don't need to change this ever.
    $dateFromStr = "1900-01-01"
    # Delete files and empty folders after archiving?
    $deleteFiles = $true
	# Path of 7zip command line
	$zip = "C:\Program Files\7-Zip\7z.exe"
    # ===== Parameters =====
    if ([string]::IsNullOrWhitespace($filter)) 
    { 
    	$filter = "*.*"
    }   
            
    if ($monthsToKeep -le 0)
    {
    	throw "Months to keep cannot be 0 or negative"
    }
    if ([string]::IsNullOrWhitespace($dateFromStr))
    { 
    	throw "Date From cannot be empty"
    }     
        
    $dateToStr = [datetime]::Today.AddMonths($monthsToKeep * -1).ToString("yyyy-MM-01")
	Write-Host "Delete files set to $deleteFiles"	
    # ===== Sanity checks =====
    if ([string]::IsNullOrWhitespace($archiveName)) { throw "Parameter archiveName cannot be empty" }
    if ([string]::IsNullOrWhitespace($archivePath)) { throw "Parameter archivePath cannot be empty" }
    if ([string]::IsNullOrWhitespace($path)) { throw "Parameter path cannot be empty" }
    if ([string]::IsNullOrWhitespace($zip)) { throw "Parameter sevenZipPath cannot be empty" }
    
    if (-not(Test-Path -Path $archivePath)) { throw "Archive path $archivePath does not exist" }
    if (-not(Test-Path -Path $path)) { throw "Root path $path does not exist" }
    if (-not(Test-Path -Path $zip)) { throw "7zip not found in $zip" }
    
    $archivePath = [System.IO.Path]::Combine($archivePath, "Run-$archiveName-$([datetime]::Now.ToString("yyyyMMdd-HHmmss"))")
    # Loop through months
    $dateFrom = [datetime]::Parse($dateFromStr)
    $dateTo = [datetime]::Parse($dateToStr)
    $dateFromLoop = $dateFrom
    $loop = $true
    $haveArchivedFiles = $false
    if (Test-Path -Path $archivePath)
    {
        throw "Directory $archivePath already exists, stopping out of precaution"
    }
    New-Item -ItemType Directory -Path $archivePath | Out-Null
    
    $fullList = Get-ChildItem -Path $path -Filter $filter -File -Recurse `
    	| Where-Object { ($_.LastWriteTime -ge $dateFrom) -and ($_.LastWriteTime -lt $dateTo) }    
    while($loop)
    {    
        $dateToLoop = $dateFromLoop.AddMonths(1)
        if ($dateToLoop -gt $dateTo)
        {
            $dateToLoop = $dateTo
            $loop = $false
        }
        $archiveFile = [System.IO.Path]::Combine($archivePath, "Archive-$archiveName-$($dateFromLoop.ToString("yyyyMMdd"))-$($dateToLoop.ToString("yyyyMMdd")).7z")
        #Write-Host $archiveFile
        $archiveList = $archiveFile.Replace(".7z", ".txt")
    
        $list = $fullList | Where-Object { ($_.LastWriteTime -ge $dateFromLoop) -and ($_.LastWriteTime -lt $dateToLoop) }
        if ($list.Count -gt 0)
        {
            $haveArchivedFiles = $true
            $list | % { Out-File -FilePath $archiveList -Encoding utf8 -Append -InputObject "$($_.FullName)" }
            $cmd = $zip
            Write-Host "================ Archiving files from $path to $archiveFile ================"
            $params = "a $archiveFile -spf -ssw -stl -v2g -mx2 @$archiveList"
            & "$cmd" $params.Split(" ")
            
            # 7z parameter -sdel instructs 7zip to delete files after archiving
            # $params = "a $archiveFile -sdel -spf -ssw -stl -v2g -mx2 @$archiveList"
            # BUUUUUUUUT there's an open 7z bug which is that -sdel doesn't work
            # with file lists (which we need here)
            # that's why we need to delete the files with powershell after 7z
            if ($deleteFiles) {
				$list | % { Remove-Item -Force -ErrorAction Continue -Path "$($_.FullName)" }
                Write-Host "Deleted $($list.Count) files"
            }         
        }
        $dateFromLoop = $dateToLoop
    }
    if (-not $haveArchivedFiles)
    {
        Write-Host "================ No files found to archive ================"
    }
}
catch
{
    Write-Host "================ An error occured ================"
    Write-Error $_
}

Git: Add existing code to repo

# See also https://docs.github.com/en/migrations/importing-source-code/using-the-command-line-to-import-source-code/adding-locally-hosted-code-to-github
cd C:\mysourcecode
git init -b main
git add . 
git commit -m "initial commit"
git remote add origin https://mycodeprovider/myrepo
git push -u origin --all

Powershell: 1-liner to create azure devops project and repos from zip files

Prerequisite: git config and az login have been completed.

PowerShell
cls; $proj=(Split-Path -Path $pwd -Leaf); az devops project create --name $proj; gci *.zip | % { Expand-Archive $_.FullName }; gci -Directory | % { cd $_.FullName; $dir=(Split-Path -Path $pwd -Leaf); az repos create --project $proj --name $dir; git init -b main ; git add . ; git commit -m "initial commit" ; git remote add origin https://MYUSERNAME@dev.azure.com/MYUSERNAME/$proj/_git/$dir ; git push -u origin --all }

Powershell: Zip directory and generate hash

PowerShell
#
# Source: DotJim blog (http://dandraka.com)
# Jim Andrakakis, April 2023
#
Clear-Host
$ErrorActionPreference = 'Stop'
# customize here
$releaseId = [guid]::NewGuid().ToString().Split('-')[0]
$SourceDir = "$PSScriptRoot\source\" 
$OutputFile = 'mypackage.zip'
$DestinationPath = "$PSScriptRoot\deploy\$($releaseId)"
$tempPath = "$($env:TEMP)\$releaseId"
New-Item -ItemType Directory -Path $DestinationPath
New-Item -ItemType Directory -Path $tempPath
# to avoid file lock issues
Copy-Item -Recurse -Path $SourceDir -Destination $tempPath
$compressParams = @{
    Path = "$tempPath\source\*"
    CompressionLevel = 'Optimal'
    DestinationPath = Join-Path $DestinationPath $OutputFile
    Force = $true
}
Compress-Archive @compressParams
(Get-FileHash $compressParams.DestinationPath).Hash | Out-File $(Join-Path $DestinationPath $OutputFile.Replace('.zip','_hash.txt'))
Write-Host "Finished"
Start-Sleep -Seconds 30

One thought on “My script cheat sheet”

  1. Pingback: My LLM/AI cheat sheet | Dot Jim

Leave a comment