LDAP based discoveries with Powershell in Operations Manager 2007

Compared to MOM 2005 I was pretty disappointed with the discovery wizard for new agents in the SCOM 2007 console. In my opinion it's not flexible enough for large deployments - but that's what Powershell is for, right :)

We keep strict naming conventions in our Active Directory so it is pretty easy for me to create LDAP queries that return only the servers I want to monitor in SCOM. Let's say I want all Exchange Servers and all of their names end with "EXCHSRV". To get started in the Ops Manager shell we have to create an LDAP query object:

$myldap = new-ldapquerydiscoverycriteria -domain MYDOMAIN -ldapquery "(&(objectClass=Computer)(name=*EXCHSRV))"

We then create a Windows discovery object (my Exchange servers run all on Windows....):

$windiscovery = new-windowsdiscoveryconfiguration -ldapquery $myldap

To start the actual discovery and store the result in another object:

$discovery_results = start-discovery -managementserver (get-managementserver) -windowsdiscoveryconfiguration $windiscovery

If necessary you can look at the $discovery_results object to see which machines were discovered. They will be stored in the CustomMonitoringObjects property.

If you are ready to deploy the agents to the discovered machines you can use this command:

foreach ($server in $discovery_results.CustomMonitoringObjects) {
install-agent -AgentManagedComputer $server -confirm
}

This will loop through each discovered machine and try to install the agent. By using the -confirm parameter we tell the script to ask before doing anything. This can be omitted of course, however I'd be careful unless you are sure your LDAP query is correct and matches only desired machines.

Of course you can use a similar approach to query for any given LDAP property. I put this in a script to scan the domain on a regular basis for new machines that might still be missing a OpsMgr agent.

Using Powershell to configure WSUS 3.0 Servers

We have a pretty large WSUS 3.0 hierarchy and use a complicated set of GPOs to configure clients. We use client-side targeting to assign machines to computer groups in WSUS, however this setting needs to be set on each downstream server. Of course over time some of those downstream servers were configured incorrectly, causing patches not being deployed to all machines.

I decided to use a powershell script to identify and fix all downstream servers to switch them to client-side targeting. It's a very simple script with no error handling, but it's a pretty good start to do all kinds of automatic configuration changes. This needs to run on the upstream server and of course requires powershell:

----
[System.Reflection.Assembly]::LoadWithPartialName('microsoft.updateservices.administration')
$ap = new-object 'Microsoft.UpdateServices.Administration.AdminProxy'$local = $ap.GetUpdateServerInstance();
$count = 0
$downstreamservers = $local.GetDownStreamServers();
write-host $downstreamservers.count

foreach ($server in $downstreamservers) {
$wsus = [Microsoft.UpdateServices.Administration.AdminProxy]::GetUpdateServer($server.fulldomainname, $FALSE);
$name = $wsus.Name
$config = $wsus.GetConfiguration()

if ($config.TargetingMode -eq "Client" )
{ write-host "Targeting Mode on $name is Client" -fore Green }
else
{ write-host "Targeting Mode on $name is Server." -fore Red
write-host "Updating config..." -fore Yellow $config.TargetingMode = "Client" $config.Save($FALSE)
$count++
}
}

write-host "Updated $count servers" -fore Yellow
-----

Brief explanation: First we get the local WSUS server and get a collection of all donwstreamservers.

The script then loops thru each WSUS server it found, gets its configuration and checks the "TargetingMode" property. If it's not "Client" we change it to "Server" to client and store the configuration back to the server using the "Save" Method of the configuration object.

Operations Manager 2007 not collecting events

I recently tried to convert one of our simple MOM 2005 rules over to SCOM 2007. It basically just notified us when a administrator account was locked out, a simple but easy way to detect brute force attacks in our environment.

I started with creating a collection rule to get certain events from the Security event logs of all Domain Controllers. I filtered by Event Source, Category and Level ("Success Audit") and targeted it to the AD Domain Controller Role.

I then created a view to check if my events are really recorded to the OpsMgr DB - and there was nothing. Also no errors anywhere. Although the targeting should not have been the problem I tried some options here just to make sure and as expected this didn't help.

It turns out the problem was the filter: I removed Event Category and my events started to come in. My filter was not wrong, it just doesn't seem to work when using the event category! In fact the events in the SCOM view do not show a category at all.

Windows, SNTP and Access Denied

If you have machines in your network that are not part of the domain you might run into an issue with time synchronization. In AD, Domain Controllers are usually good and reliable time servers (if not you are at least in sync with the rest of your network), and domain members sync their automatically (more or less).

Now if you try to sync a non-domain Windows machine with a domain controller by running "net time \\mydc /set" you will get an Access Denied error.

The solution is fairly easy: You can switch your non-domain clients and server to use NTP instead of the Windows standard. Your domain controllers also run native NTP servers on UPD port 123 by default. To change your client edit the registry:

Key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters
Change "Ntpserver" from "time.windows.com" to "mydc"
Change "Type" from "NT5DS" to "NTP"

Restart the Windows Time Service (w32time).
Run "w32time /resync /rediscover" to test your new settings

WSUS 3.0 and Backup Exec - Error 0xe00084af

After upgrading some of our WSUS 2.0 servers also running Backup Exec 10.0 to WSUS 3.0 SP1 our backup jobs started failing.
The error was 0xe00084af Directory was not found, or could not be accessed.None of the files or subdirectories contained within will be backed up. The byte count for the job showed 0.

I messed with a couple of settings like the VSS writers, but eventually just gave up and updated to Backup Exec 10d (10.1). This update is free if you have a valid license for 10.0 and is really fairly simple to do. It only takes a few minutes, however also needs a server reboot. For more details look here: http://seer.support.veritas.com/docs/281044.htm

WSUS 3.0 downloading all patches in all languages...

We have a pretty big organization and run about 100 WSUS servers. I recently started upgrading to WSUS 3.0 SP1, starting with the master upstream server (all other servers are replicas and downstream servers).

All went well first, but after a few weeks of upgrading downstream servers I suddenly saw an increase in the disk space used for content on the master server. After a while the WSUS content exceeded over 50 GB worth of data, although I had only Windows updates and service packs in the English language selected (should be around 10 GB max). Eventually we ran out of space on the content disk.

I double-checked all options, ran the cleanup wizards and all WSUSUtil tools I could think of, nothing helped. I ended up deleting all WSUS content, and while it looked fine first after a few days it downloaded all 50 GBs again. I took a closer look at the files (Windows Explorer and WSUS Admin MMC) and quickly found that our server hosted ALL languages and the express update files, even though I had none of that selected.

I openend a ticket with Microsoft and they quickly gave me the answer: One of the downstream servers was configured incorrectly! I thought it was rather annoying that one server can mess up my master server, especially since some servers are controlled locally and I really can't prevent somebody from doing this. And of course the question is: How do we find the one server causing this problem?

Microsoft told me to download this: http://www.codeplex.com/WSUS/Release/ProjectReleases.aspx?ReleaseId=4640

Extract it using /c and run WSUS3_Basicinfo.exe. This tool is pretty neat, however running this locally against 100 server seemed to be a bit of a hassle to me, and also this would take forever... I came up with an easier way: Find one of the files in the WSUSCONTENT folder you don't need (e. g. a greek language update if you have only English selected). I took the file name and searched my IIS logs for it and BINGO, I had the IP of the server trying to download it.

From there it was pretty straighforward: Reconfigure the downstream server, delete all unneeded revisions and updates (cleanup wizard and wsusutil). Sync with the upstream server. On the upstream server I had to delete all content first, on the downstream server the cleanup wizard did it for me.

This cost me lots of hours of work and downtime for WSUS. Dear Microsoft - never ever should a product that is designed to be used in hierarchy be implemented like that - a downlevel machine can affect the higher levels? That's why it's called a hierarchy; the UPPER levels determine what's going on!