Microsoft released UR5 for System Center 2012 SP1. It contains fixes for DPM, VMM and SCOM.
The KB article can be found here.
Like I said yesterday, be careful to check which version of System Ceter you are downloading the patches for (2012 or 2012 SP1 or 2012 R2; they all have their own set of updates and rollup versions). Be sure to test them in test environments. Read the installation guidance, known issues and so on on the KB article pages and release notes. There may be additional steps required or an installation order!
The SCOM 2012 SP1 specific UR5 can be found here by the way.
Microsoft released Update Rollup 1 for System Center 2012 R2 which has fixes for SCOM, DPM and VMM.
Let me first say a few things:
- Be carefull to first check out which version you are downloading. There are UR releases for SCOM 2012, for 2012 SP1 and for 2012 R2. These can not be mixed around. The story today is about the update on top of 2012 R2.
- Read the KB articles belonging to these updates all the way. They contain info on what this update fixes, but also very important installation information, known issues, installation order and so on. For example for SCOM and VMM there are manual steps like running SQL scripts and importing management packs, and for SCOM the updates are split into the usual SCOM pieces and the SCOM Unix/Linux monitoring pieces.
- Test all updates in a test environment. I know some people might not like me saying this, but given the problems which occurred with some updates coming from Microsoft over the last lets say 14 months you definately want to be testing it in your test environment. Perhaps also wait for 2 weeks or more before doing that to see if people are blogging about problems before even putting it in an official test environment.
For SCOM 2012 R2 UR1 the KB article and information can be found here:
Good luck patching!
Today, 22 January 2014 will be the first WMUG NL user group webinar of the year.
Our good friend and MVP Didier van Hoye is presenting: Troubleshooting Hyper-V Live Migration performance issues. It is going to be a great session and I can't wait
We start at 8 PM Amsterdam time zone.
The session will be hoested on Live Meeting through this link:
If you are connecting directly from the Live Meeting client:
Meeting ID: 643CCK
Attendee Entry Code: xGW9"7&"Q
Come join us at the Windows Management User Group Netherlands for these webinars!
We have several planned and there might be more on the way, depending also on speaker availability. If you are a potential speaker and interested in presenting a technical topic just contact me through bob at thedomainnameofmyblog.
A while back my good friend Marnix Wolf wrote about a list of things you must not forget when adding an additional management server to a SCOM management group. Of course we can all imagine to install SCOM on that server and update it to the same Update Rollup level as the existing management servers, but there are more things to keep in mind. At the time I wrote myself a reminder to add a few items to Marnix's list, and this week while adding a few management servers to an existing management group actually thought about those items. From there Marnix and myself constructed a new list of things to do and check when adding an additional SCOM management server.
You can read the updated list here:
I guess we could think of even more things which could happen or be the case in more rare cases. For instance when not only adding a server, but also removing one of the existing ones after that (so installing servers with the goal of removing others). But those kind of things have more to do with removal of an MS and therefore the other MS having to pick up the workloads of the older MS, such as connectors and workflows pointing to such a server from other tooling, or the Exchange correlation engine in the Exchange 2010 MP. But I think the article mentioned above covers most things we would forget about when just adding a management server.
A good list to jump back to.
Had an issue the other day with an already installed and configured agent refusing to start.
When trying to start the service through command prompt I got this error:
C:\>net start healthservice
The System Center Management service is starting.
The System Center Management service could not be started.
A service specific error occurred: 2164195395.
And in the System event log I found this entry:
The System Center Management service terminated with service-specific error %%-2130771901
Went back to look in the Operations Manager event log entries to see what happened right before the agent service stopped and found at least these two error entries a few days earlier:
Operations Manager has received a bad configuration file for management group XYZ which could not be parsed. Configuration has been re-requested. A copy of the bad configuration file will be placed in C:\Program Files\System Center Operations Manager\Agent\Health Service State\\Connector Configuration Cache\XYZ\OpsMgrConnector.Config.Error.xml
The Management Group XYZ failed to start. The error message is 0xC00CEE04(0xC00CEE04). A previous message with more detail may have been logged.
Right after these the agent had stopped.
Usually first thing to start is to check the registry for the WindowsAccountLockdownSD entry:
This entry should be present. Otherwise copy it from another machine in the same management group with a working agent.
In this case this was all there and good.
Next checked the Registry again, but looking for the State Directory here:
This should have an entry called State Directory and please check if the directory path stated there exists and is correct.
Next I went to this directory and removed the contents.
In my case everything located in:
C:\Program Files\System Center Operations Manager\Agent\Health Service State\
Tried to start the healthservice again through the services.msc or by command prompt net start healthservice
And it started working again.
Very soon it will be the time for the System Center Universe 2014 event again.
This year there is a contest for the SCU Jedi. The one who wins will speak about his topic of interest at the event.
Voting is still possible until the 15th of December, so don't forget to vote!
Here is the playlist with all three remaining contestants:
Play the videos and vote for your favorite by using the like options.
My favorite is Dieter Wijckmans with a great monitoring related subject!!
Jalasoft has been creating great network monitoring and visualization tools on top of SCOM for years now and I have been working with them for a long time. They are quite active in creating new products and new versions of existing products, which evolve with customer demand and SCOM product updates. These weeks we are seeing a number of product updates:
Xian Network Manager 2012 SP2
A little less than 2 weeks ago Jalasoft announced the release of Xian Network Manager 2012 Service Pack 2. This latest version of the product supports SCOM 2012 R2. It features improvements to the Configuration Monitoring feature and adding support for the latest version of VMware Virtual Center. In addition, one of the main features called Netflow also got a lot of upgraded functionality and can handle V9 templates and adds dashboard improvements. Also it can now filter data to a top-N set to send to SCOM in order not to overflow SCOM with too much data which is marginal in importance.
More information on Xian Network Manager can be found here: http://www.jalasoft.com/xian/networkmanager
Xian Wings 2.0
The next thing to get exited about is that Jalasoft is releasing Xian Wings 2.0 this week. It will be released on Wednesday 30 October 2013! Xian Wings brings all kinds of SCOM views to your mobile and tab devices, including state, alerts, performance graphs, tasks and notifications. This is not limited to the Xian Network Manager objects in SCOM, but all objects. Now I am pleased to early-announce (with permission) some of the improvements in Wings 2.0:
Xian Wings HD for iPad
- Remote Shell: it is now possible to run PowerShell and cmd prompt commands directly from your iPad against the SCOM server or another computer. This means that additional tasks or actions can be directly done as a response to some alerts received in SCOM. I will surely be showing you what this looks like in one my upcoming blog posts.
- Notifications Widgets: besides alerts, states and performance graphs, it is now possible to add notifications widgets to any Wings dashboard. This widget will display a new notification object for each alert generated or modified on SCOM associated to any selected object within the scope of this particular widget.
Xian Wings for Windows Phone
- Full support for Windows phone 8: a new Xian Wings client for Windows phone 8 has been created in order to fully support this new version. Besides this latest version, Wings still supports phone 7 as well. It is very nice to see both versions of the Windows Phone OS are now supported.
Xian Wings for Android and iPhone
- Class icon for each monitored object: each monitored object now displays its associated type icon along with its status, making it easier to know which object has problems and what kind of component it is; for instance a database, computer or network device.
Xian Wings 2.0 Server
- The Xian Wings server now fully supports .Net Frameworks 4.5 in order to improve performance and compatibility with the latest Microsoft platforms and technologies.
- The Xian Wings server and/or console can now be installed on any computer running Windows 7 or 8.
More information on Xian Wings can be found here: http://www.jalasoft.com/xian/wings
Like I said this calls for an additional blog post on the possibilities of the latest Xian Wings 2.0 on my iPad mini. Stay tuned for that one to come in the near future!
The case of not being able to uninstall SCOM agent due to Failed to load Intercept Scripting LibrarySCOM, System Center, SCOM 2012 Send feedback »
Had a strange case today. On some servers in a DMZ we wanted to upgrade the existing SCOM 2007 R2 agents to SCOM 2012 SP1. However something strange happened during upgrade and somehow it looked like both the old agent and the new agent were installed, but not working. Upon removing of the old agent entry from the control panel we still found the SCOM 2012 agent unable to connect and even to start the service (we had both cases actually on a few machines). We also had one machine where we first removed the old agent followed by a reboot and a clean install of the new agent and still had below problems.
Next course of action is to go to Control Panel and uninstall the SCOM 2012 agent. Only to receive an error like the one below. Failed to load Intercept Library.
Troubleshooting led to all kinds of things
First because I couldn't remove the agent, I thought lets try to heal it first.
Lets go through a few items:
- The system environment variable Path still contained the path to the 2007 agent folder.
- The state directory of the agent found in the registry was pointing to the 2007 folder at hklm\system\currentcontrolset\services\healthservice\parameters\
- There were a few old entries in the registry pointing to wrong folder names. However looking at these they seemed not to be that important. Cleaned them anyway.
Try to start the SCOM agent.
Results in two errors seen below. Especially the 2147221164 error seemed to be the main one and we got that one on all the machines.
The error 2147221164 can be caused by different things and I have some links at the bottom of this post that have some relation. However in this case it was something different. We were missing a registry entry for a connector.
We ran the following registry file to merge into the registry:
Windows Registry Editor Version 5.00
@="C:\\Program Files\\System Center Operations Manager\\Agent\\MOMConnector.dll"
Well, at least the SCOM agent service starts after this. However we are immediately greeted in the Operations Manager event log by hundreds of 4507 errors. They talk about not being able to register a class ID in the registry (the agent tries to register some entries in the HKEY_classes_root\CLSID for different types of workflows). And with hundreds I mean hundreds of these errors.
So I guess we get back to uninstalling the agent again, because this can not be good.
This time I went out and got the log file where we uninstall the agent. I see the following entries:
CExecutorHelper::GetPath: FAILED: Executor not registered.. Error Code: 0x80070057.
UninstallAPMMonitoringConfiguration: Failed to load Intercept Scripting Library. Error Code: 0x80004005.
MSI (s) (B8!D8) [14:19:26:606]: Product: System Center 2012 - Operations Manager Agent -- Failed to load Intercept Scripting Library
Well, I have looked and could not find anything about this error "Failed to load Intercept Scripting Library" or anything in these lines basically.
What we can see in here is what I expected when I saw this error. Intercept Studio is the old name for the APM piece of the SCOM agent. And we can see it is trying to uninstall that piece (or at least gather all related info).
At this point I decided to call out to my fellow MVP's and MS Product Team members. Somebody must have an idea
And sure enough, out of the Product Group there came a suggestion. He actually looked at the first line in the error (Executor not registered). So he suggested to register an APM dll and see if that got it moving.
Alright here goes:
In written form:
- In a (elevated) command prompt on the server go to the directory where the SCOM agent APM component is installed.
C:\Program Files\System Center Operations Manager\Agent\APMDOTNETAgent
- Next register the DLL
Now try to uninstall the SCOM agent again.
Next we close all open programs and clean out the SCOM agent related directories in "C:\Program Files". I am guessing they were left over after the issues.
Next we could proceed to install the SCOm 2012 SP1 agent and UR3 and run momcertimport.exe to import the already existing certificate. And everytyhing was running again!
A very big thank you to Vlad, Daniele and Vitaly for getting me in touch with Dmitry, who gave the suggestions pointing to the solution!! And of course a very big thank you to Dmitry himself for trying to think of ways how this could happen and what might be done to get it rolling again!
Some of the articles I have used in troubleshooting and which I freely interpreted during troubleshooting and trying to solve this:
AT least some of the things mentioned in these linked articles pointed me to somewhat similar situations for smaller pieces of the puzzle and where to look for troubleshooting.
Hope this helps the very few that might run into a similar issue.
As many of you know the last version of the SharePoint 2013 management pack had an issue with its reports. They couldn't be published to the report server and thus caused the health state of the management server to turn to a red state. Yesterday an updated version of the SharePoint 2013 packs have been published that contain the fix to that issue.
You can find the Packs here:
- System Center Management Pack for SharePoint Server 2013
- System Center Management Pack for SharePoint Foundation 2013
Very happy this got fixed now, so we can get to monitoring again
Last week I installed a SCOM agent on a Windows 2008 R2 print server, only to discover that the SCOM agent went wild and kept crashing. Furthermore I found out it was generating a lot of data (about 18 milion entries in a few hours). So what had happened? Well the reason I found in this blog post from Daniel Mueller dating from April 2013.
In short the Printer Server 2008 mp (version 6.0.7004.0) has a mistake in a few performance collection rules. They are targetted at "Printer" in stead of print server role. Because it collects data from all instances, this meant in my case it saw about 500 printer queues and multiplied that by 500 times, because every printer queue monitored all instances of all other printer queues as well. As you can see with a high number of printer queues this thing will go wild.
The Printer Server 2008 management pack dates from December 2012 and it looks like even after the blog post from Daniel nothing has happened with it. So I decided a few days ago that I will post a hotfix management pack which will disable the original faulty collection rules and create new rules with the same settings but correct targetting.
The "Hotfix Management Pack: Print Server 2008" can be found in the TechNet Gallery.
It is specifically to override the 6.0.7004.0 version of the Print Server 2008 management pack from Microsoft, so you do not run into the performance and agent issues.
And thanks to Daniel Mueller for recognizing this issue in the first place.