Friday, August 8, 2008

WSUS 3.0 downloading all patches in all languages...

We have a pretty big organization and run about 100 WSUS servers. I recently started upgrading to WSUS 3.0 SP1, starting with the master upstream server (all other servers are replicas and downstream servers).

All went well first, but after a few weeks of upgrading downstream servers I suddenly saw an increase in the disk space used for content on the master server. After a while the WSUS content exceeded over 50 GB worth of data, although I had only Windows updates and service packs in the English language selected (should be around 10 GB max). Eventually we ran out of space on the content disk.

I double-checked all options, ran the cleanup wizards and all WSUSUtil tools I could think of, nothing helped. I ended up deleting all WSUS content, and while it looked fine first after a few days it downloaded all 50 GBs again. I took a closer look at the files (Windows Explorer and WSUS Admin MMC) and quickly found that our server hosted ALL languages and the express update files, even though I had none of that selected.

I openend a ticket with Microsoft and they quickly gave me the answer: One of the downstream servers was configured incorrectly! I thought it was rather annoying that one server can mess up my master server, especially since some servers are controlled locally and I really can't prevent somebody from doing this. And of course the question is: How do we find the one server causing this problem?

Microsoft told me to download this:
http://www.codeplex.com/WSUS/Release/ProjectReleases.aspx?ReleaseId=4640

Extract it using /c and run WSUS3_Basicinfo.exe. This tool is pretty neat, however running this locally against 100 server seemed to be a bit of a hassle to me, and also this would take forever... I came up with an easier way: Find one of the files in the WSUSCONTENT folder you don't need (e. g. a greek language update if you have only English selected). I took the file name and searched my IIS logs for it and BINGO, I had the IP of the server trying to download it.

From there it was pretty straighforward: Reconfigure the downstream server, delete all unneeded revisions and updates (cleanup wizard and wsusutil). Sync with the upstream server. On the upstream server I had to delete all content first, on the downstream server the cleanup wizard did it for me.

This cost me lots of hours of work and downtime for WSUS. Dear Microsoft - never ever should a product that is designed to be used in hierarchy be implemented like that - a downlevel machine can affect the higher levels? That's why it's called a hierarchy; the UPPER levels determine what's going on!

1 comment:

Graeme Hefner said...

Mine is currently doing this, despite not even HAVING a downstream server. Quite a bitch to figure out.