During the past year I have seen several requests from people in the community on how they can check if their servers have successfully applied a certain Cumulative Update. Lately I had a requirement to create some test scripts to determine if CU5 has been applied for the RMS (including SQL scripts and management pack import), Management servers, gateways and web console servers. So I set out to create scripts for this and make them general enough to share.
As you probably know its much easier to have a quick check for agent updates from the SCOM console. I did not include the ACS server check for the moment. This post and set of scripts is specifically for CU5, but I will post the same for CU6 today in another post. Idea is the same, scripts are on a different link. The scripts can not cover a 100% of all possible things to check, but I think we get far enough this way
Thus far I have tested the scripts in a few environments and have generalized them as much as possible, finding information we need from the system itself. I am sure others with more PowerShell skills can do this in fewer lines of code, but I have tried my best. Any comments are welcome of course.
- You download the zip file and extract it somewhere on the machine you want to check (RMS, MS, GW, Web).
- Log onto the machine with the account you used to install the CU. Reason is that the script is trying to find the CU install log files in the temp folder in your profile, if it can not find it there it will give an error and skip to the next checking steps.
- Open an elevated command prompt en go to the folder where the scripts are located.
- Run one of the .cmd files belonging to the server role you have installed:
TestSCOMRootMSCU5.cmd for the RMS (This also checks the SQL scripts and MP import you did right after RMS upgrade)
TestSCOMMSCU5.cmd for a Management Server (not the RMS)
TestSCOMGWCU5.cmd for a Gateway
TestSCOMWebCU5.cmd for a Web Console server
If you run the Web Console on a management server or the RMS than you simply run both scripts needed for that machine.
- Check the output for Errors. I have made the scripts stop after it found an error in some stage, except if it can not find the install log files. Errors are marked with [ERROR] and are in red. Am listing some examples below.
I tried to document the steps and checks we do in the script itself. Different roles have some different checks associated with them.
Below is an example where first a RMS check was run and finished with success. Next piece was what happens if you run two of the scripts on the box for a role it doesn't have (it gives an error ). Next we run the web console checking script which starts out and finishes successfully.
And yes, this happens to be the screenshot for CU6.
So that is it. It should cover most scenarios.
The next post will be the same for CU6 and point back to this one on how to run it.
Also I want to extend a word of thanks to Marius van de Ven, whom I bothered a few times with some specific questions on how to do stuff in PowerShell.
Enjoy and good luck!
At last MMS 2012 we got the opportunity to take the beta versions of the Private Cloud exams 70-246 and 70-247 for System Center 2012. I must say they were tough exams and I and many others were not sure about the results. Usually I have a feeling about an exam I took, but this time it was very mixed. The thing is with these exams that they are filled with questions on different technologies in the System Center 2012 suite and stuff around it.
Yesterday I got confirmation on the first exam and today on the second one. I am pleased to report that I passed both of them.
Very happy with that. It means I have all the prerequisites for the MCSE Private Cloud certification!
I will soon write a post about all the new Windows Server, desktop and System Center exams and certifications which are here or coming up within this year. It will probably not contain the same about the SQL 2012 and programming/Visual Studio related exams for 2012, but know there will be similar new exams for those as well.
Had a case where we used the SCOM 2012 network device discovery and found our devices to be in Pending Management, with the notification "No Response Ping". Even with network devices where we only ping and ping does work from the command prompt.
Thing is that it is not the same when you ping from a command prompt and when it happens programatically.
As several prople run into this the short way to fix it:
On the Windows machine of your management server(s) open the firewall settings and find rules starting with "Operations Manager" in both inbound and outbound rules. There are a few that seem to be default disabled. Those are highlighted below. And guess what... They are all about Ping and SNMP. Jsut turn on whatever rules you want there and re-run the discovery rule (or wait for it to happen on your schedule) and the network devices will appear.
A view of inbound rules and outbound rules:
If you are running SCOM 2007 R2 or SCOM 2012, there are new updates available for you! See below for both. We start with SCOM 2012 and will name SCOM 2007 below that.
SCOM 2012 UR1
A number of days ago Microsoft Released System Center 2012 UR1 (Update Rollup 1) of SYstem Center 2012 with update for SCOM and VMM mainly (and App Controller I think). The list of updates for SCOM 2012 is an interesting one which addresses a number of issues we have seen. SO that is recommended to be installed. The update is not very large in download size and there is no more bootstrapper. After the update the files that were replaced will have version 7.0.8560.1021. The following are resources you will need to go for SCOM 2012 UR1:
- Description of Update Rollup 1 for System Center 2012
- Download Update Rollup 1 for System Center 2012 - Operations Manager
- Kevin Holman's page: OpsMgr 2012: Cumulative Update Rollup 1 (UR1) ships–and my experience installing it
SCOM 2007 R2 CU6
A few hours ago Microsoft released Cumulative update 6 for SCOM 2007 R2 (CU6). It has a number of fixes in the list. It looks like the installation order is roughly the same as previous CU's, but be sure to first read through the whole KB article. Experience has learned that reading closely avoids a number of issues. Be sure to test it first in a test environment, or perhaps wait for a few days until some of the more well known names have posted about their experiences installing it and if they found any issues during install or right after it. The version of the files after the upgrade will be 6.1.7221.99 after applying CU6 (the files that get replaced of course).
- Description: Cumulative Update 6 for System Center Operations Manager 2007 R2 is available
- Download: Cumulative Update 6 for System Center Operations Manager 2007 R2 (KB2626076)
For sure I will do both updates in several environments (both test and production) and if I run into anything I will surely post it back here.
Happy upgrading and monitoring!
While I was playing around with some SCOM 2012 web console items in a test environment I ran into another small issue. Probably because we have been playing around with it too much. But than again I guess somebody else might run into this issue as well.
The case is this one:
We see some parts of the web console, but when we click on for instance a computer state view the monitoring pane shows an error.
The error says:
HTTP Error 500.19 - Internal Server Error
The requested page cannot be accessed because the related configuration data for the page is invalid.
You can see the situation and the rest of the info in the screenshot.
Because I happened to be e-mailing with a product team member on another issue anyway, I mentioned it. Turns out to be something simple.
Open IIS Manager, open the server and go to Application Pools.
Check if the OperationsManagerMonitoringView application pool is set to use .Net Framework 2 and not 4. As you see in the screenshot above it was using version 4 for that application pool.
If it is set to .Net 4 than change it back to .Net 2 and recycle the application pool.
The web console worked like a charm after that.
Thanks for your pointer Nathan!
Had an issue today in a test environment with a freshly installed SCOM 2012 all-in-one machine. SCOM was working fine. Started the SCOM Web Console and suddenly got an error. The details of the error looked like this:
The rest is here in a screenshot:
So clicked around a little bit and came to the conclusion that I must have missed something during prerequisite installation or installation of the SCOM Web Console had some issue. Not wanting to loose too much time I went for removing the Web Console feature and re-doing some prerequisite installation stuff.
- Start SCOM 2012 installation and select to remove a feature. Remove Web Console feature.
- Run some powershell commands to install the prerequisites:
Add-WindowsFeature NET-Framework-Core,Web-Static-Content,Web-Default-Doc,Web-Dir-Browsing,Web-Http-Errors,Web-Http-Logging,Web-Request-Monitor,Web-Filtering,Web-Stat-Compression,AS-Web-Support,Web-Metabase,Web-Asp-Net,Web-Windows-Auth –restart
- Run the following command from elevated command prompt to register the ASP .NET 4 with IIS.
- Next the command to allow the Asp .Net 4 ISAPI/CGI.
c:\windows\system32\inetsrv\appcmd set config /section:isapiCgiRestriction /[path=`'C:\Windows\Microsoft.NET\Framework64\v4.0.30319\aspnet_isapi.dll`'].allowed:True
- Restart the server
- Run SCOM 2012 setup and select to add a feature and add the Web Console feature.
Started up the web console link again from internet explorer and got a logon prompt and a few seconds later had a working Web Console.
I guess the principle of "when in doubt why it is not working -> re-install the Web Console" still works.
Last week I got a question from a friend in the System Center community, Adrian Chirtoc, about how to remove a management group entry from a multi-homed SCOM 2012 agent by using a script.
The case at hand was a parallel side-by-side installation of SCOM 2012 next to a SCOM 2007 R2 management group. By this process all SCOM agents got upgraded to SCOM 2012 version. Next step in this case would be to remove the management group entry of the old SCOM management group from the new agent.
Now one would at first think this is as easy as using the method previously used for SCOM 2007 agents. We have discussed a few of those over here in SCOM Trick 19 – Remove SCOM agent command.
Another much heard remark would be to use the control panel, where in the SCOM 2012 version agent you will find this place where you can add and remove and edit management group entries in an existing SCOM 2012 agent.
But in this case the requirement is to do it by using a script. This will also be applicable if you are in a multi-homing scenario where you are doing agent management by some other means, such as Configuration Manager or any other deployment tool. This is where the scripting comes into play.
So, as far as I could find out in short time I could only find an uninstall command from the momagent.msi which in this version should completely remove the agent and where you will not name a management group to remove (which you could do in SCOM 2007). I could not find any way to interface with the agent management applet as shown above. SO next step would be to check out either PowerShell or scripting against the SDK.
So, we end up talking to the Operations Manager Agent Configuration library, which is available on each SCOM 2012 agent and management server as AgentConfigManager.dll . From there we find a method to use to remove a management group entry from an agent.
I uploaded a simple vbscript to remove a management group entry from a SCOM 2012 agent in the TechNet Gallery you can find by clicking this sentence.
From there I figured we might want to be able to also add management group entries to an existing agent in order to multi-home it by using a script. So that is simply another method which we can use to manipulate the agent. The example script to add a management group to an existing SCOM 2012 agent by script is also uploaded to the TechNet Gallery. There is also a possibility to add an action account to the agent in the same command, but I left it at three entries (management group + management server + agent port).
To get to these I have used information from the Operations Manager 2012 SDK reference.
Have fun scripting the existing SCOM 2012 agents!
A few days ago Microsoft released a KB article which explains how to add a product key to an evaluation version of SCOM 2012. It is a simple procedure. You can find the KB article here:
You can check if you are using an eval version of the product by opening the SCOM Console and going to Help -> About.
Good luck with that!
Lately I have been working to tune a BizTalk 2010 management pack, because it was creating a lot of alerts whenever something went wrong (hundreds and sometimes thousands). I have found in the past with a BT 2006 MP that that one has the same behavior and how to get around it. While I was working on this noise reduction this week, we also got some questions and remarks from others in the community who were running into the same thing. So I thought I can generalize it a bit for this mp and post it for others to use. I did it the easy way, so anybody will see which rules were disabled and re-created.
Below is a description and an important note (which states to first test it!!) and a download link from the TechNet Gallery:
BICTT BTNR Biztalk 2010 management pack
This management pack is created in order to reduce the noise in a multitude of alerts some people see after importing the Biztalk Server 2010 management packs.
This management pack has the same version as the management pack it is meant to reduce noise from (which is 7.0.389.0 for this release).
The Microsoft management pack for Biztalk Server 2010 can be found here:
The BICTT BTNR management pack is depending on those management packs for their use.
What happened was that a number of rules check for events in the event logs of these BizTalk Servers for messages that got stuck. This would create one SCOM alert for every message that got stuck in a channel, because the message ID was specified in the alert and thus seen as unique. In multiple cases this caused hundreds or thousands of alerts in a very short time period if this happened on large volume systems. This thus caused the SCOM console to become very slow and operators could not scroll through the long list of alerts anymore to see what was going on. In some cases this resulted in a select-all and close action of operators in trying to clean up and thus throwing away other alerts as well which happened to be in their screen.
What this management pack does is to override the original rules which seem to cause this behavior and it holds exactly the same rules with one small change. It doesn’t check the contents of what we see as the Alert Description to see if it is the same. So if one of these rules is triggered multiple times on the same machine on the same channel it will give you one alert and increase the repeat counter on it.
One additional note:
Always test management packs in a test environment and see if they work as expected. This management pack is made available as-is because we got a few requests for it, but we do not provide any special support for it. Just make sure you test it yourself.
Download the BICTT BTNR Biztalk 2010 Management pack here from the TechNet Gallery.
Good luck and enjoy your monitoring!
So after playing around on test machines, I found it time to upgrade our (small) production DPM 2010 installation to DPM 2012. With the RTM bits of course. Yeah! We have a few domain controllers, Exchange servers, file shares, web servers, a SharePoint server, SQL databases, System States on it.
This is a kind of step by step of what I encountered during the upgrade and you will find that I did hit some snags causing me to do something first before continuing. If you are trying to replicate these steps it is best to first read ahead to see if there is something you need to do first!
Alright, so first things I checked again was to see if protection groups are working. This seemed to be fine. Next to check the agents. I had two old agents in the list which were not protected either, so I removed them. There were no agents waiting for upgrades or unavailable.
Next thing to check is if we have the latest QFE for DPM 2010 (this is a QFE from March 2011):
If you still need to run this QFE, remember to upgrade the server components, the agents and I would advise a consistency check on the protection groups.
So the next step is to of course download the DPM 2012 iso file from wherever you get your Microsoft bits. In my case the machine is a physical server, so I used another box to unpack the files from the iso to a local path, in my case “D:\DPM12”. Its is a 3 GB file containing about 2954 files so had to give it a few minutes. And a few minutes more. Alright, my hardware is not the fastest!
While we are waiting for this it might also be a good time to backup the DPM database!
So let’s start rolling with the upgrade by starting the setup from the DPM 2012 media!
Click the Data Protection Manager to install. Accept the following agreement.
After waiting a while for the installer to unpack we get an error as follows:
Alright, so we need to download that fix from here first:
After installation of this hotfix and restarting the DPM server and updating all the agents, we can go for the next step which was the previous step… Running the installer for DPM 2012 again!
And that brings us up to the welcome page of the DPM 2012 setup.
This brings us to the prerequisite check and the question what SQL to use.
I will go for the first option and press the Check and Install button.
The check went fine, so let’s go to the next step.
Alright we need the product key from the back of the DVD. In my case that will be the back of the downloaded ISO file. After entering the product key we move on to the next step:
I had all components on the C drive already and there is sufficient space available there, so we can move ahead.
DPM creates a local user for you and you need to enter a password for this account.
We are asked whether to opt-in to Microsoft Update. So at the moment of this writing Microsoft is slowly adding the System Center 2012 products into Microsoft Update (and WSUS). I will use the opt-in setting here to connect to Microsoft Update, in order to be ready for future updates coming through that channel.
We are asked whether we want to enter into the CEIP. I click Yes and see the magic Upgrade button at the bottom of the screen. Let’s go for it.
This starts to install SQL 2008 R2. This took a while.
After this step it went into installing DPM 2012 and it gave a fail.
Alright, it looks like it had a problem copying over de old DPM 2010 database. Probably had a locked file due to SQL services stopping a bit too slow. So I went back and ran the installer again. And the result was positive!
Next on the agenda is the reboot, so did that immediately.
After the reboot I open the DPM 2012 console. Great!
So first thing I want to do is update the agents.
On the left side menu go to Management and find the agents with the following status:
We can select these agents and use the Update Available link to start updating them.
After all agents have been updated we move on to the Protection pane.
You can select a protection group and click the Consistency check button in the top menu.
All of them need to get a consistency check.
After some time you should be seeing those nice green icons again!
Of course check everything that doesn’t turn green in the console. And we now need to update DPM monitoring in SCOM as well, but that is out of scope for today!
So in all we needed to check the latest DPM 2010 update was installed and we needed another update as well before we could move ahead with the full upgrade. In this case we ran into another snag during the upgrade, but we went for it again and everything was alright.
Another update done. On to the next!