Morning Check SCOM Alerts – Automated using PowerShell Script

One of my clients has a centralised SCOM management group for different segments of the business. The support team's hours are 6 am - 6 pm Monday - Friday. They are also required to perform a morning check Monday – Friday and manually log service desk calls based on the SCOM alerts generated after hours.

I wanted to automate the process (at least to generate a list of alerts that were raised since 6:00 pm previous night). We firstly had a look at the Alerts report from SCOM Generic Reports library but it did not meet our requirements as it does not include the alert resolution state in the report.

We have the following requirements to accomplish:

1. Include any warning and critical alerts generated between 6:00 pm previous business day and 6:00 am present day.
2. Only include alerts that are still open (resolution state not equals to 255)
3. Only include production servers from a specific domain
4. Time Raised is displayed as the local time, not GMT standard time
5. List is emailed out every morning

As my knowledge in SQL reporting is very limited, I achieved the task using below PowerShell script. It is scheduled in Task Scheduler of the SCOM RMS server to run Monday – Friday.

$erroractionpreference = "SilentlyContinue"

$strDomain = "<DNS Domain name where SCOM agents are in>"
$strNetBiosDomain = "<NetBiosDomainName of SCOM agent>"
$RMS = "<FQDN of SCOM RMS>"

##Initiate SCOM PSSnapin
Write-Host "Initiating SCOM PS Snapin..." -ForegroundColor Yellow

#adding SCOM PSSnapin
if ((Get-PSSnapin | where-Object { $_.Name -eq 'Microsoft.EnterpriseManagement.OperationsManager.Client' }) -eq $null) 
{
Add-PSSnapin Microsoft.EnterpriseManagement.OperationsManager.Client -ErrorAction SilentlyContinue -ErrorVariable Err
}
if ((Get-PSDrive | where-Object {$_.Name -eq 'Monitoring'}) -eq $null)
{
New-PSDrive -Name:Monitoring -PSProvider:OperationsManagerMonitoring -Root:\ -ErrorAction SilentlyContinue -ErrorVariable Err | Out-Null
}

#Connect to RMS
Write-Host "Connecting to SCOM RMS $RMS..." -ForegroundColor Yellow
Set-Location "OperationsManagerMonitoring::"
new-managementGroupConnection -ConnectionString:$RMS | Out-Null
Set-Location Monitoring:\$RMS

#Get Current Date and Time
$Date = Get-Date
$TimeBiasInMinutes = ($Date - $Date.ToUniversalTime()).TotalMinutes
$strDate = $Date.ToShortDateString()
$strDate = $strDate -replace("/", "-")


#Output csv file - stored at the same location of this script
$thisScript = $myInvocation.MyCommand.Path
$scriptRoot = Split-Path(Resolve-Path $thisScript)
$OutputFile = Join-Path $scriptRoot "MorningCheckOutput\Morning-Check-$strDate.csv"

#check if output file exists. delete it if it does
if (Test-Path $OutputFile) {Remove-Item $OutputFile -Force}

#Create the folder to store csv file if it does not exist
if (!(Test-Path (Split-Path $OutputFile))) {New-Item (Split-Path $OutputFile) -type Directory -force}

#Set the Start date and time
#StartDate is the previous business day from today
$PreviousBusinessDay = $Date
DO
{
$PreviousBusinessDay = $PreviousBusinessDay.AddDays(-1)
} until ($PreviousBusinessDay.DayofWeek -inotmatch "saturday" -AND $PreviousBusinessDay.DayofWeek -inotmatch "sunday")

$StartDate = Get-Date -Date $PreviousBusinessDay -Hour 18 -Minute 0 -Second 0
$uStartDate = $StartDate.ToUniversalTime()

#Set the end date and time
$EndDate = Get-Date -Date $Date -Hour 6 -Minute 0 -Second 0
$uEndDate = $EndDate.ToUniversalTime()

#If End time is in the future, use current date and time instead
if ($EndDate -gt $Date) {$EndDate = $Date}

#Base URL to retrieve alerts from SCOM web console
$strBaseURL = "<SCOM Web Console URL>/default.aspx?DisplayMode=Pivot&AlertID="

#Get valid monitoring object
$strGroupName = "<SCOM Group that contains production servers>"
foreach ($group in get-monitoringobjectgroup) {if ($group.DisplayName -eq $strGroupName) {$MonitoringObjectGroup = $group}}

$arrGroupMembers = @()
Foreach ($member in $MonitoringObjectGroup.getrelatedmonitoringobjects())
{ $arrGroupMembers += $member.Displayname}

#Get alerts

Write-Host "Getting alerts from SCOM..." -ForegroundColor Yellow
$arrAlerts = Get-Alert | Where-Object {$_.TimeRaised -ge $uStartDate -and $_.TimeRaised -le $uEndDate -and $_.ResolutionState -ne 255 -and $_.NetBiosDomainName -imatch $strNetBiosDomain -and $_.severity -inotmatch "information"}

Write-Host "$iAlertCount alerts found" -ForegroundColor Yellow

#Some of the properties in alerts need to be processed first. therefore creating psobject to store them
$arrPsObjAlerts = @()
Foreach ($alert in $arrAlerts) {
if ($arrGroupMembers -icontains $alert.principalName)
{
$PSObjAlert = New-Object psobject
$TimeRaised = $alert.TimeRaised.AddMinutes($TimeBiasInMinutes)
$LastModified = $alert.LastModified.AddMinutes($TimeBiasInMinutes)
$AlertURL = $strBaseURL+"{"+$alert.ID+"}"

Add-Member -InputObject $PSObjAlert -MemberType NoteProperty -Name ID -Value $alert.Id
Add-Member -InputObject $PSObjAlert -MemberType NoteProperty -Name PrincipalName -Value $alert.PrincipalName
Add-Member -InputObject $PSObjAlert -MemberType NoteProperty -Name AlertName -Value $alert.Name
Add-Member -InputObject $PSObjAlert -MemberType NoteProperty -Name AlertDescription -Value $alert.Description
Add-Member -InputObject $PSObjAlert -MemberType NoteProperty -Name IsMonitorAlert -Value $alert.IsMonitorAlert
Add-Member -InputObject $PSObjAlert -MemberType NoteProperty -Name IsMonitorAlert -Value $alert.IsMonitorAlert
Add-Member -InputObject $PSObjAlert -MemberType NoteProperty -Name ResolutionState -Value $alert.ResolutionState
Add-Member -InputObject $PSObjAlert -MemberType NoteProperty -Name Priority -Value $alert.Priority
Add-Member -InputObject $PSObjAlert -MemberType NoteProperty -Name Severity -Value $alert.Severity
Add-Member -InputObject $PSObjAlert -MemberType NoteProperty -Name TimeRaised -Value $TimeRaised
Add-Member -InputObject $PSObjAlert -MemberType NoteProperty -Name LastModified -Value $LastModified
Add-Member -InputObject $PSObjAlert -MemberType NoteProperty -Name RepeatCount -Value $alert.RepeatCount
Add-Member -InputObject $PSObjAlert -MemberType NoteProperty -Name LastModifiedBy -Value $alert.LastModifiedBy
Add-Member -InputObject $PSObjAlert -MemberType NoteProperty -Name AlertURL -Value $AlertURL
$arrPsObjAlerts += $PSObjAlert

Clear-Variable TimeRaised
Clear-Variable LastModified
Clear-Variable PSObjectAlert
Clear-Variable AlertURL
}
}
$iAlertCount = $arrPsObjAlerts.count
$arrPsObjAlerts | Export-Csv -NoTypeInformation -Path $OutputFile


#email out the output
Write-Host "Emailing the report..." -ForegroundColor Yellow
$strSMTP = "<SMTP Server FQDN>"
$strSubject = "SCOM Morning Check - $strDate" 

$strBody = @"
<html><body>
<p> There are $iAlertCount alerts generated in SCOM between $startDate and $endDate.
<p>
<p>Detailed report attached...
</body></html>
"@

$MailMessage = New-Object System.Net.Mail.MailMessage
$MailMessage.IsBodyHtml = $true
$SMTPClient = New-Object System.Net.Mail.smtpClient
$SMTPClient.host = $strSMTP
$Recipient = New-Object System.Net.Mail.MailAddress("<recipient email address>", "<recipient name>")
$Sender = New-Object System.Net.Mail.MailAddress("<SMTP sender address>", "SCOM Administrator")
$MailMessage.Subject = $strSubject
$MailMessage.Sender = $Sender
$MailMessage.From = $Sender
$MailMessage.Subject = $strSubject
$MailMessage.To.add($Recipient)
$MailMessage.Body = $strBody
$MailMessage.attachments.add($OutPutFile)
$SMTPClient.Send($MailMessage)

Write-Host "Done!" -ForegroundColor Yellow

!!Change the parts indicated in red with your environments information before running the script!!

Note:  A SCOM group is required to include all servers in the scope (in this case, all production servers from the particular domain).

Download script here.

Hope that this post was helpful.