Friday, August 8, 2008

Windows, SNTP and Access Denied

If you have machines in your network that are not part of the domain you might run into an issue with time synchronization. In AD, Domain Controllers are usually good and reliable time servers (if not you are at least in sync with the rest of your network), and domain members sync their automatically (more or less).

Now if you try to sync a non-domain Windows machine with a domain controller by running "net time \\mydc /set" you will get an Access Denied error.

The solution is fairly easy: You can switch your non-domain clients and server to use NTP instead of the Windows standard. Your domain controllers also run native NTP servers on UPD port 123 by default. To change your client edit the registry:

Key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\Parameters
Change "Ntpserver" from "time.windows.com" to "mydc"
Change "Type" from "NT5DS" to "NTP"

Restart the Windows Time Service (w32time).
Run "w32time /resync /rediscover" to test your new settings

WSUS 3.0 and Backup Exec - Error 0xe00084af

After upgrading some of our WSUS 2.0 servers also running Backup Exec 10.0 to WSUS 3.0 SP1 our backup jobs started failing.
The error was 0xe00084af Directory was not found, or could not be accessed.None of the files or subdirectories contained within will be backed up. The byte count for the job showed 0.

I messed with a couple of settings like the VSS writers, but eventually just gave up and updated to Backup Exec 10d (10.1). This update is free if you have a valid license for 10.0 and is really fairly simple to do. It only takes a few minutes, however also needs a server reboot. For more details look here:
http://seer.support.veritas.com/docs/281044.htm

WSUS 3.0 downloading all patches in all languages...

We have a pretty big organization and run about 100 WSUS servers. I recently started upgrading to WSUS 3.0 SP1, starting with the master upstream server (all other servers are replicas and downstream servers).

All went well first, but after a few weeks of upgrading downstream servers I suddenly saw an increase in the disk space used for content on the master server. After a while the WSUS content exceeded over 50 GB worth of data, although I had only Windows updates and service packs in the English language selected (should be around 10 GB max). Eventually we ran out of space on the content disk.

I double-checked all options, ran the cleanup wizards and all WSUSUtil tools I could think of, nothing helped. I ended up deleting all WSUS content, and while it looked fine first after a few days it downloaded all 50 GBs again. I took a closer look at the files (Windows Explorer and WSUS Admin MMC) and quickly found that our server hosted ALL languages and the express update files, even though I had none of that selected.

I openend a ticket with Microsoft and they quickly gave me the answer: One of the downstream servers was configured incorrectly! I thought it was rather annoying that one server can mess up my master server, especially since some servers are controlled locally and I really can't prevent somebody from doing this. And of course the question is: How do we find the one server causing this problem?

Microsoft told me to download this:
http://www.codeplex.com/WSUS/Release/ProjectReleases.aspx?ReleaseId=4640

Extract it using /c and run WSUS3_Basicinfo.exe. This tool is pretty neat, however running this locally against 100 server seemed to be a bit of a hassle to me, and also this would take forever... I came up with an easier way: Find one of the files in the WSUSCONTENT folder you don't need (e. g. a greek language update if you have only English selected). I took the file name and searched my IIS logs for it and BINGO, I had the IP of the server trying to download it.

From there it was pretty straighforward: Reconfigure the downstream server, delete all unneeded revisions and updates (cleanup wizard and wsusutil). Sync with the upstream server. On the upstream server I had to delete all content first, on the downstream server the cleanup wizard did it for me.

This cost me lots of hours of work and downtime for WSUS. Dear Microsoft - never ever should a product that is designed to be used in hierarchy be implemented like that - a downlevel machine can affect the higher levels? That's why it's called a hierarchy; the UPPER levels determine what's going on!