I have written and talked about Jalasofts great tool called Device Simulator. Many people in the monitoring space have been using this tool in previous versions. Today they released a fresh version of this tool to the 5.5 version.
The main additions to this version are:
- An OID editor meant to customize the behavior of certain OID's within the simulation file of your to be simulated device.
- The SNMP Dump Manager to more easily work with al the simulation files you are using
- The number of simulated devices out of the box has been expanded to over 100!
This tool can be used for free provided you only simulate a few devices at a time. And for a few dollars an expanded license can be bought to do a lot more testing with this.
For demo and test environments this can be used a lot as a primairy tool to simulate network devices without needing to have all of that infrastructure in production. Also you could take a recording of an existing device in your production and even including real time data and play it back through the simulator.
Everybody needs a copy of this piece of software!
As you may be aware I have been very involved with an animal charity. It is called House of Tails (www.houseoftails.org ) and is about taking care of abandoned dogs in Thailand. This is also the reason why the volume of my tech blog posts went down for a while. I will however pick up the pace again, I promise! Lots of ideas here. Because the dog sanctuary has a very hard time to make things happen this has taken a lot of time. We have setup an official Foundation (Stichting in Dutch language) with bank and email and PayPal and so on and we are working on a fresh website.
Yesterday a dear friend Pete Zerger wrote a blog posting about our story and he has taken it upon himself (and his friends and family) to bring a challenge. Any donation given to our House of Tails charity within a 72 hour period (starting last evening, so for the coming 52 hours or so still) will be DOUBLED. So any donation you make will get a double effect for the animal charity and all of it will go to the wellbeing of these dogs!
Please read how you can help too:
Also I will include a link here to the PayPal/Creditcard possibility of the charity Stichting House of Tails.
Take a minute to check it out through that post and through the House of Tails website.
Seeing how my geeky family and my online families are willing to help out a good cause is truely heart warming. It means soo much for us and for the dogs!
a truely gratefull Bob Cornelissen
This is an update to the page describing a solution to the "Web Console Configuration Required" message for SCOM 2012 and SCOM 2012 R2.
If you get this message on your workstations and you do not have the rights to simply run the configuration wizard it asks you to run, it has to be configured through another way.
What the configuration wizard does is set a few registry keys and add a code signing certificate (can also be done through registry). For instance a customer of mine this needs to be entered into a VDI image to make it work for most people.
At SCOM 2012 R2 UR3 a new code signing certificate was introduced which was valid until 22 July 2015. ANd now at SCOM 2012 R2 UR7 a new code signing certificate is introduced. It is now valid until somewhere in September 2016. Bit unfortunate there was a lack of overlap there.
The article I made last year SCOM 2012 Web Console Configuration Required describes the fix and how to find the registry keys. AT the bottom of the article there are updates for a registry key we found as an extra potential blocker of access to the SCOM web Console and there is another update created today for UR7. It has the registry files you need for either UR3/4/5/6 or for the UR7 now.
Good luck and enjoy!
I wish to direct your attention to a blog post made by my friend and fellow System Center MVP Tao Yang. He is talking about community efforts and contributions and on how some of them get turned into commercial products. If you are working with SCOM for instance you will find many management packs and scripts and solutions and thoughts right from the community. You will also find some great paid solutions written by serious software vendors who add value. There are also cases where both options are available and this is a point where you have to think on how to spend your time and money wisely. (By the way a great way is supporting a dog charity, look to the right of this post!). Power to the community. Please read his blog post:
Regards, Bob Cornelissen
Recently I received a request to be able to monitor all Windows Services running under a specific user account (service account) one certain machines, and to bring them together in a overview. I will show you how I did this.
This blog post will show you a complete way to monitor a list of services running using the same account.
I will show you at the bottom of the post how to monitor a list of services with a similar name (which is just a few words different from this whole story).
First of all we have of course multiple ways to monitor services as I have explained in a blog post series about the subject: SCOM - Monitoring a Service - Part 1 intro. However all of this is geared towards monitoring a single service on a single machine or a group of machines for instance.
But sometimes I get requests to:
- Monitor multiple services with somewhat similar names (for instance "System Center *").
- Monitor multiple services running under a certain account.
- Alert if any service on any server is running using a certain account (which they dont want).
The first item has been blogged about years ago. The second one is what I was asked to do in this case, so I will pick that one up first. And the last one is something we did in a completely diffeent manner and I might blog about that as well later if you like that (the question was actually not nonly for Windows Services, but also for Windows Scheduled Tasks). I will get back to that in a minute.
The next thing is to bring these items for the first two bullets together in a dashboard or console. As example in the case I ran into at a certain customer the easy way would be to open up a Savision Live Map and add them through an automatic population rule. That way they are of course added to a group and are visible and have a rollup, all happy. In this case we were limited in the amount of maps we could use. So a next likely alternative is to create a distributed application (DA) in SCOM. From there we can add the found services into the DA's. Problem is this does not populate automatically by default. But we can change that behavior to enable automatic population of items on a DA. Think it is best to put that in a separate post how to do that. But the idea was to: find these services + monitor them + put them in a DA + automatically. So also if a service gets added which uses the same criteria it should automatically be brought into monitoring and should be added to the DA! And in his specific case we added the resulting DA's as items on an existing Live Map dashboard.
You know what, I can blog about all these things These kind of workarounds are not always easy to find, and sometimes it is an adjustment of what has been described by one or two people before.
We need to have a discovery and monitoring of Windows Services running on a certain machine, for all services which are using the same service account. And we also wanted to do this for both Automatic and Manual services.
Now we need to create a similar use case to do this blog post with. Lets go and find all Windows Services on the SCOM management server using the SDK service account (On my test machine im running a few services with that account to make it easy for me).
I will write this down mostly step by step and expect a little knowledge on how to do stuff in the SCOM Console and there will be XML editting involved, but still no crazy stuff.
Let's do this!
I am going to use a method posted about by Brian Wren wayyyyy back in 2008 for using wildcards with the Windows Service template, and adjust it to my needs. By the way the method in the post mentioned is the way to monitor multiple services with a similar name and a wildcard, using the same template as we use below. Lucky we can keep using the same methods as before and even modify them for other similar needs!
I want to have a new fresh management pack, and I want to have a group of servers to target this service monitoring to. So I start with creating a New Group in SCOM and meanwhile when it asks me where I want to save it I will have it create a New Management Pack for me.
The name is easy to recognize and I am using a versioning format as 1.year.month.anumber. This way with the year and month placed there when I create updates to a pack it is always a higher version than the last time I editted it (if it is in the same month I just move the last digit up a version number). Just one of the things I use for a versioning strategy.
And now we have a group in the same management pack. From this point I walk through the wizard for the group and just add the SCOM Management server to it. So this is my group of servers where I want to monitor the services on.
Moving on, now we will go to the Authoring pane in the SCOM console.
In the Authoring pane find the Management pack templates -> Windows Service. Click the Add Monitoring Wizard. Select Windows Service to start the correct wizard.
Enter a name and description and of course place this in the newly created management pack.
Now on the Service Details page we enter a temporary service name. We will change this later, but we need to have something we can find back quickly enough. So I entered test123 as service name. Next we select the targeted group, which is the group we created earlier and contains the application servers. Because we want to monitor these services both when they are in automatic state and manual startup state we de-select the checkbox at "Monitor only automatic service". I am not that interested in performance data for each service so I just walk through the rest of the wizard in a Next Next Next Finish kind of way.
So now we have a service monitor created in the new management pack and targetted at a group of servers.
Now go to the SCOM Admin pane and go to the Management Packs list. Find this management pack and export it to for instance the desktop.
First we need to do something else before we start editting the code within the exported XML of the management pack.
Now the default Windows Service monitoring template uses a different method to find the specific service (discover it!) than we would use to collect several services in one query. FOr this we want to use WMI to query windows for all the services we are interested in and put that in the Discovery item in the management pack we just created. For this to work of course we first need to find the correct WMI query and test if this gives us the desired result. For this we can use a tool already installed on your machine.
Go to the server where there are these different services running using the same service account. Open a command prompt. Type wbemtest and press enter.
Press the Connect button, make sure it says "root\cimv2" near the top and press Connect button again.
We should now be connected and all the buttons enabled. Find the Query button and click it.
Now the trick is to make the right query to get your data returned to you. In my case I had to search a little bit. Also because I wanted to have the extra filter not only for account name but also wanting to see manual and automatically starting services.
select * from win32_service where (Startname like '%sdk%') AND (Startmode != 'Disabled')
I came up with that query. So what does it do. Well it requests the list of windows services and filters it (where) to find any service with a Startname containing sdk. The percent % signs around it are the wildcards. In WMI they use a percent sign as wildcard and not the star. And I added a second filter where I filtered out all disabled services. That should give me what I want. Lets run the query by clicking the Apply button.
Alright. These are the internal names for the services and indeed there are two SCOM related services and 3 SQL related services which are running with an account with "sdk" in the name somewhere. So after some playing and trying to find how to write WMI queries we finally have a query which will return what we want. Now please copy the query somewhere, so we can use this later.
Now the management pack Authoring part! We will now edit the XML of the management pack we saved earlier. You can do this in any editor and even in Notepad, but I suggest to use at least an editor with some code highlighting. For instance I use Notepad++ for this.
And of course I found out that I did not have it installed on this test server, so I downloaded it (free !) and installed it.
Open up the XML file in the preferred tekst editor. We want to find the Discovery for our custom service we created. So find "test123" and you will have found the discovery.
This looks like a lot of stuff and to some it is not Always too clear what it all means, but stay with me. I will highlight the pieces of code within this block we want to change:
I am sorry to report that this blogging software messes up the XML code blocks and the codeblock option makes it unreadable. So I put the whole thing in MS Word and made two screenshots of the whole discovery. The red parts of the code is what we are looking for (in three places).
We are going to replace the first red part with:
(I left the Frequency tag in there so you recognize it).
The next red piece was where it said: 'BinaryPathName'
We replace that by: 'PathName'
And the third red piece was for the DisplayName and in my case it said: SDK Services
We will replace that by: $Data/Property[@Name='Name']$
Next I scroll to the top of the code and find the version number and give the last digit a higher number:
Now I found out that the template can also make a strange management pack reference. We are using discovery modules living in the Windows Library and the reference has an alias. We need to change something to make this work. Go to near the top of the management pack code (first 30 lines or so) and you will see a list of references. Look for the one listing the Microsoft.Windows.Library as such:
See the here purple colors alias? We need to copy that alias name (so everyting between the quotes and not including those quotes). Next go to the first block of tekst you replaced.
Now the bit where I put the red line under is a reference, we can recognize it by the ! exclamation mark ! Now replace the part before the exclamation mark (windows) with the alias you had in your reference.
Now it should look like the above, and referencing the reference alias you found in your code.
Next I save the file using the exact same name (do not rename management pack files!). And I go to SCOM Admin pane - Management Packs and I import this management pack from disk.
Now in the SCOM Console go to the Monitoring pane. On the left hand side scroll all the way down until you find "Windows Service and Process Monitoring" and go to the state view called "Windows Service State". Within minute I saw these 5 services appear and a few minutes later they were in a monitored state as well. The two at the top just happened to be in there already.
Alright, there we have it! We are now monitoring all windows services - running under a service account with sdk in the name of the account - for both automatic and manual start services - on all machines in a custom group we created.
How to monitor multiple services with a similar name in SCOM
Alright as promissed how to monitor a group of services with a similar display name, such as this example:
You see the group of Jalasoft services there. I want to find all of them and monitor them all. Well you have seen that we can use wbemtest tool to create a WMI query to get that data. In this case we will filter on the name (this is the displayname as you see it in the screenshot above).
select * from win32_service where (name like '%Jalasoft%')
There is our string and the result while querying WMI for this. Exactly what we want.
Well in the story above we know where the query ended up, so we can just insert this WMI query in the same place to have it monitor all services with Jalasoft in the name.
In this case I want to add this monitoring set to the existing pack I created above. So in short the order of actions:
- Create a Service monitoring template entry and name it Jalasoft Services.
- Store it in the Custom Appllication Monitoring management pack.
- Complete the wizard and put in a recognizable name in the what service to monitor field.
- Export the management pack
- Verify through wbemtest that our WMI query gives the desired result and copy the correct query
- Open up the exported management pack in an XML editor or text editor and find the discovery relating to this in the way shown above and replace the three blocks of code for the three new ones and of course insert the right WMI query.
- Find the management pack reference Alias pointing to Microsoft.Windows.Library and replace the reference in the new code block for that one.
- Raise the number of the management pack version and save the management pack.
- Import the management pack
- Inspect the results in the monitoring pane
And there we go:
A group of services with something similar in the name, running on a group of servers we defined. Any service with similar name being added to this group of machines will get picked up within a few minutes and be monitored.
SO now we have had the story on how to add either a group of services with a similar name or running under a similar account.
Brian Wren's post was all we need to cover the second example and also put us on the right path for the first example. I just thought it was time to blog about this in an extensive post to show how this is done and what things we can run into. Also using the SCOM 2012 R2 interface of course and showing the wbemtest interface.
Next on the agenda:
Well I said in the beginning that I wanted these objects placed in a distributed application in SCOM and have it pull in these services automatically. So I will create a simple Distributed Application and I will add all Jalasoft services to it.
Of course this will be in my next post coming very soon!
It has been a while since I wrote about Veeam and their management pack for monitoring VMware products (did you know they also added Hyper-V to that now?). Lately I got a chance to install this pack in a reasonable size environment, so this is a good chance to write more about the product and what it can do for us.
This environment has two VMware infrastructures. One for the servers based on 60+ hosts and one for the VDI desktops environment also roughly based on 60+ hosts. This situation also requires that we have different monitoring for each environment, because we are not really interested in VDI desktops which get destroyed and rebuilt every day. But in the server infra we are interested in the guests as those are the ones we also monitor with SCOM agents. I will discuss as well how this was achieved, because Veeam has a way of doing this which not every organization will use, but it is very useful in the scenario we are dealing with.
When designing your Veeam-SCOM monitoring infrastructure there are a few things to keep in mind which affected our design in this scenario.
- Numbers. A bit of an open door, but the amounts of hosts/datastores/VM’s you monitor has an effect on the sizing of your Veeam Collectors. Deciding whether or not to monitor VM’s makes a big difference.
- Wanting two separate methods of monitoring for the two environments is best done with the use of Monitoring Groups. It is a feature included in the Enterprise Plus license of Veeam and gives possibility to create in our case two Monitoring Groups. Each with their own collection policy (one not interested in collecting VM information for instance).
- A Collector can only be a member of one Monitoring Group.
- High Availability, even though a collector might be able to handle the load it might be desired to have an additional collector in each monitoring group for high availability purposes and during normal situations a way to spread the load.
One good piece of advice is to download the resource kit and use it. It has an excel sheet in it which is a calculator for the estimated number of collectors you need for a certain environment!
By the way you will find that a collector can handle a very nice load. The stability of this platform has been strengthened throughout the years (And I have been stress testing previous versions of the pack for years in some big environments just to see when it would break down under the load and give the feedback to the developers).
In our case we went with 3 collectors for the server VMware environment and for 2 collectors for the VDI environment. Because we were not interested in the VDI VM’s this was enough to also provide high availability for the collectors within the monitoring groups.
Because not many are using the monitoring groups feature I will show you what it looks like.
This picture shows on the left hand side the monitoring groups. The top one is the default and the other two were created manually. The example shows the VDI monitoring group with 2 collectors and some stats. In the middle of the picture there are two important things we adjusted as a start. First of all to change the collection interval from the default 5 minutes to 10 minutes (this basically gives a bit less data flow simply because it collects for instance performance data less frequently). The second important setting is setting the Collect VM Data to false for this monitoring group. With thousands of VDI VMs this makes quite a difference.
If you open the monitoring group collector settings you see first of all the possibility to stop the inheritance from the settings of the top level to flow down to this monitoring group. Next we set the EnableVMDiscovery option to False to stop the discovery of VMs and their data from going towards SCOM.
Something else to note in these settings is the CollectionInterval which in this case we set to 10 minutes. And as an example the DataStoreScanInterval that is set to a number, which is actually a multiplier for the CollectionInterval. So when I changed the CollectionInterval from 5 minutes to 10 minutes, also other collections which happen less often are dependent on this setting. In this case the Datastores will be scanned every 12x10 minutes, so every 2 hours instead of every 1 hour. Keep in mind that when tinkering with the collector settings there might be multiple things happening in the background. In case you want to play with the default settings make sure you read up on what these settings do. Veeam MP is very well documented from design to install to adjusting to monitoring.
Now, when starting out with the installation and connecting the collectors to the VCenter servers we deliberately set the collection interval to 10 minutes and also the VM discovery to disabled on both monitoring groups. We figured the amount of data coming in from only the hosts, VCenter, clusters, datastores and hardware and so on and the resulting alerts from that would be enough to start with in the first few days. This was a good decision as there was enough to start with all these layers without already putting the VMs layer on top of it. For larger environments I think it is useful to phase in the deployment this way to give opportunity to do tuning and tweaking and discuss the resulting alerts and views with the operators. It also gives you opportunity to check on the performance and dataflow of the collectors and their agents and the SCOM infrastructure. Check also for most common alerts and health state changes and the amount of performance data coming in and see if this suits your needs. After that continue to the next phase and bring in more monitored stuff.
Hope this was useful to show a case where monitoring groups within the Veeam Management Pack can be used and why. Actually I wrote this post a while back and did not get to posting it until now. Meanwhile we have done a lot of tuning and dashboarding for instance.
Our friends at Infront Consulting Group are organizing another great set of sessions on SCOM MP Authoring. Watch below for an announcement and registration information. The event is capped to 100 attendees for get there on time:
Silect Software and Infront Consulting are hosting a FREE MP University on August 12th. Registration is capped at 100 people so we wanted to ask you to spread the word to your many followers. Brian Wren and Kevin Holman will both be presenting.
Jump to the registration at This Link
A big thank you to Infront Consulting and Silect Software for organizing this and for the presenters such as Brian Wren and Kevin Holman for being there!
I am sure many of you know about BlueStripe solutions and how they connect to SCOM for application discovery and so on. Today we hear that Microsoft has aquired BlueStripe. I think this is a very good move indeed! Read the announcement here:
We will probably soon see the effects of this in several areas around monitoring and management
For those of you wanting to provide the SCOM full console, possibly augmented with Savision Live Maps extensions, to desktops which are locked down, as a package. It might help
We want to create a software package for the SCOM Console with Savision Live maps extensions so it can be run on managed locked down desktops.
Used are Windows 7 desktops and ThinApp and AppSense.
The desktops are locked down for users.
We started with the prerequisites and installation files which needed to be added to the package (or not):
- .Net Framework 4, was already available in the system, so not included in building this package
- SQL CLR Types, was already present in the image as package, so not included
- ReportViewer 2012, was already present in the image as package, so not included
+ SCOM 2012 R2 Console installation
+ The at that time current Update Rollup 5 for SCOM 2012 R2
+ Savision Live maps extension for the at time current version 7.3
After the package was created and installed there were problems with SCOM and specifically with the Savision extension for viewing Savision maps directly from the full SCOM console.
It looked strongly like a rights issue, because the problem would occur on the restricted desktops with normal users.
The issue were the below mentioned two dll files, which do not get created at installation, but at login. The first user to login seems to create these two files in the ProgramData path. The creation of dll files in that path was blocked by AppSense.
The solution is to WhiteList the specific path in AppSense Application Manager so a user can write the dll there.
It seems that the subdirectory where these are created are dependent on the version of Savision used and when updating it to a newer version an additional subdirectory is created and the new two files placed there. So a wildcard was used for the folder name.
There was also an exclusion made for the SCOM directory:
As soon as it was made possible for a user to create these files upon first login to SCOM Console and opening a Live Map these were created and everything worked fine.
A big thanks to Jerry Chen for checking and fixing this.
I had some problems with a State view in SCOM for a certain class, which used to work and now produced an error saying:
Object reference not set to an instance of an object
So I opened up the Discovered Inventory view and selected the appropriate class and waited for the screen to populate.
The same error appeared.
Just for fun I thought I would pick up the instances of the class with powershell by using the get-scomclass and get-scomclassinstance combination to get a table of all objects of the class and that did turn out a list of data.
So I suspected a faulty entry somewhere in the database for one of the discovered objects. Actually I had 1 top level class created and 3 child classes and this happened to two of the child classes.
Updating the pack to a newer version did not work and it was confirmed in the test environment that the pack should not be the problem.
I created a state view of the not monitored top level class and added columns for the other classes to it and that one looked fine.
So I wanted the discovered objects to be all gone from SCOM
The way you usually speed up the process is by setting overrides to the discovery rules for these classes and disabling them for all objects.
Next you run the command remove-scomdisabledclassinstance
And wait for it to remove all the entries.
But while running this command I got an error (see below in text and picture form):
PS C:\> remove-scomdisabledclassinstance
**WARNING** This operation WILL result in class instances and relationships being permanently deleted. This operation i
s irreversible and will likely result in a significant amount of activity in the operational database. Are you sure you
want to do this? [Y/N]
Started. This operation may take a very long time to complete.
remove-scomdisabledclassinstance : Discovery data has been received from a rule targeted at a non-existent object ID.
Object ID: 8375a67f-5750-53e1-bd9f-7468152934b0
Rule ID: 3147a861-3d91-74a6-675e-e41be8f92658
At line:1 char:1
+ CategoryInfo : InvalidOperation: (Microsoft.Syste...nstancesCommand:RemoveSCDisabledClassInstancesComma
nd) [Remove-SCOMDisabledClassInstance], DiscoveryDataFromRuleTargetedToDeletedObjectException
+ FullyQualifiedErrorId : ExecutionError,Microsoft.SystemCenter.OperationsManagerV10.Commands.RemoveSCDisabledClas
If you get that error the best you can first do is in SCOM open up a Discovered Inventory view and check how many objects are still there (if it does not throw the same exception as the one I started with!).
Check the number of entries there.
Now run the remove-scomdisabledclassinstance command again.
If it throws the error refresh the screen to check if discovered inventory shows a lower number of discovered instances.
Keep doing that until it finishes. Took me 25 times
Next I waited for a grooming interval and let it populate the classes again one by one by turning the discoveries back on.