A case of ghost objects in SCOM

At a customer of ours we were investigating a case where an old SCCM server which was removed a while ago kept getting back into SCOM.

What we saw was a server object which was not an agent. Upon investigation we found these were instances of an SCCM roles, such as a distribution point. So first we went into the SQL database and checked for the entries and removed them there. Sure enough the next day they were back in SCOM. So we know it is a discovery somewhere. We checked the entries again and saw them hosted/created by the SCCM primary site server. If you look in the Discovered Inventory view and look for the class “ConfigMgr Site System Information” you will get a list.

So next step was to look in the registry of the SCCM server, because it happens that there can be old entries there. Listed here is an example of that: https://www.windowsmanagementexperts.com/configmgr-remove-orphaned-site-systems/configmgr-remove-orphaned-site-systems.htm However in our case there were no false registry keys.

Time to see what the discovery was doing. I pulled up the SCCM management pack and extracted it to XML and opened it up. It is a bit looking for the discoveries, but there are Hierarchy and Site discoveries listed using scripts. Upon reading the scripts we found Select statements. Looked like SQL at first, but they were WMI queries! These run on the primary site server and look in WMI for the objects in the Site Hierarchy!

We pulled the WMI explorer out and checked the script for where it was looking for the list of servers and roles. This is in Root\SMS\<sitecode> (use your own site-code there) and in the class/table SMS_SystemResourceList we indeed found all the entries we expected + the two entries we did not want to see.

Now with PowerShell we tried to make a query to get those two objects. This went something like this…

$TheObjects = Get-wmiobject –Namespace “ROOT\SMS\sitecode” –Query “SELECT * FROM SMS_SystemResourceList WHERE ServerRemoteName=’theservername.domain’”

We used our own site-code and server name obviously and got our two entries back. So we figured we could simply do a

$TheObjects | Remove-wmiobject

But we received errors that this was not supported. While it seemed like a perfectly normal command to us!

The solution came to us from our friend the PowerShell MVP Jeff Wouters who found that we were looking at a read-only class, so removing stuff from it was not supported. It turns out the SMS_SCI_SysResUse class table is the one we should target to remove those objects.

So we made a query again to get the object with an additional where filter to get 1 of the 2 objects at a time and pipe that to the remove statement.

Get-WMIobject -Namespace “ROOT\SMS\sitecode” -Query “SELECT * FROM SMS_SCI_SysResUse where NetworkOSPath like ‘%theservername.domain%’ and RoleName=’SMS Distribution Point'” | Remove-WMIObject

And repeated that for the other entry for another SCCM role it had before. It looked like this server had come from one site to another site a long time ago during some migration and the two entries that were not cleaned up by the system after removing it from the new site did not get removed. Right after removing the entries from the source class we saw the change being replicated into the class where the SCOM discovery was reading the servers from.

After running the Hierarchy discovery again (there is a task for it in SCOM if you target the primary site server in the SCCM servers state view), the objects disappeared.

Things are not always easy, but by putting some minds together you can achieve it!

Bob Cornelissen