SMS + VirusScan = HELL

We’re experiencing the following symptoms:

  • Very high CPU utilization
  • Vitrually unresponsive systems
  • Violent end-users
  • Vicious finger pointing (not really, but it was a good V-word!)

While doing research, we’ve noted that the 2 things common on systems with these symptoms was that SVCHOST.EXE (Automatic Updates in this case) or CCMEXEC.EXE were the main CPU hogs. Killing CCMEXEC, running ccmclean.exe, or sometimes killing the Automatic Updates service would bring the system back from the brink.

Googling for “ccmexec cpu” and similar phrases came up with a few places to start looking:

SMS Admin newsgroup
MyITForum Discussion

Hopefully we’ll get this nailed down and have:

  • Responsive systems
  • Readiness for update deployments (we’re temporarily suspending them until we get this worked out)
  • Really happy customers
  • Raises (maybe not).

My mind is going… or: Working with HALs

Here are some initial resources as I’m beginning to delve into the area of Hardware Abstraction Layers during our image build cycle. We’re looking at what HAL types can be mapped to others and which are incompatible to begin to consolidate some of our system images. Some Microsoft kb articles:

How to Troubleshoot Windows 2000 Hardware Abstraction Layer Issues

How to force a Hardware Abstraction Layer during an upgrade or an installation of Windows XP

HAL options after Windows XP or Windows Server 2003 Setup

More information to follow once we begin to more fully understand the copncept of Hardware Abstraction Layers.

ACPE 2006 – Day 2

Being my first year attending the ACPE conference, I had no idea what to expect. Would the talks be riveting? Would I get some good food? Will I get a million emails from vendors now that my name is in a database? Since arriving at the Resort at the Mountain, (the location for this year’s conference) I’ve been completely blown away by the level of not only hospitality and fun, but also by the seminar material and speakers. I expect to get a few emails from vendors, but after the party they threw last night (and are throwing again tonight) they can email me all they want.

Some of the more notable seminars:

Jose Dominguez, Senior Network Engineer, UO Network Services
Derrick Lindsay, Mississippi State Department of Education

Jose went over some good information on packet shaping, with the focus being to analyze and understand your environment and create policies for waht you want things to look and act like before throwing dollars at the problem. He also gave some linkst to tools that I haven’t yet heard of but will be testing out in the near future:

http://www.switch.ch/tf-tant/floma/software.html

http://www.ntop.org/overview.html

http://software.uninett.no/stager/

http://www.splintered.net/sw/flow-tools/

http://www.ibm.com/software/tivoli/products/netview/

Derrick gave one of the best talks on business continuity I’ve ever attended. The source of real applicable knowledge for his organization came at an enormous cost however. Most of the information he gave was directly a result of lessons learned during the Rita and Katrina disasters. He talked about abstracting the “smarts” (as he referred to it) of the architecture to a switching cloud that can be routed, changed, and take a beating and keep on switching. He also had some really good advice for dealing with vendors and service providers: Ask them about their redundancy and continuity! All the redundancy money can buy will do your network no good if you can’t get beyond your telco’s equipment. Really great stuff.

http://www.mde.k12.ms.us/

I also sat down today and threw together a quick business card. I haven’t had one made, and I needed to be able to hand something to people when they ask me for a card. That sentence made no sense, but since I’m late for the party, it stays. Anyway, here’s the card if you simply must see it.

Disable Chassis Intrusion detection

After deploying Dell OMCI to about 600 desktops and portables, an alert began displaying upon user logon:

Dell OMCI Chassis Intrusion Alkert

Needless to say, users were somewhat confused by this.

To get rid of the message, we either had to:

  1. Run around to every Dell PC in the organization
  2. Uninstall Dell OMCI
  3. Remotely disable chassis intrusion detection and clear any current detections

Obviously we chose option three. With Dell’s OpenManage IT Assistant software, I was able to build a remote CIM command line to execute on a set of systems (in our case any system that was reporting a status of degraded). Here’s the command we ran:

system cim action=setcim ipaddress=$IP username=$USERNAME password=$PASSWORD authenticationlevel=packet classpropertyvalue=Dell_SMBIOSsettings::ChassisIntrusion:4

To execute the command, I setup a new command line task in ITA, targeted at a query of computers whose status was not “OK.” I set this to run once an hour, since clients were still being discovered and inventoried as this was happening. By setting the query to only hit degraded clents, we avoided running this needlessly on clients already configured properly.