Category: "Windows 2008"
I was working with an old Windows 2008 R2 server last night. It needed a "few" updates!
So I will first admit to several of my own mistakes. I did not give myself time to update this machine regularly enough in the past and of course we always have to install the Windows Updates on time. If you figure you wait for an extra month for any fixes introduced one month to be fixed the next its something we all understand. But this was many months worth of updates. I went the lazy way, which bit me as you will see below.
I was first interested on getting 1 specific update on the machine. So I selected that update and a random few other smaller updates. Now this is a mistake! It installed the updates and wanted a reboot. OK. Next thing which happens is that the machine starts up in an immediate Blue Screen with code STOP 0x00000050 PAGE_FAULT_IN_NONPAGED_AREA or in short a code 0x50. There was no way around this into for instance safe mode or whatever. The only thing which popped up was the System Recovery Options shown below:
By the way, before you get to this screen it asks you for the Local Administrator password. Turns out even I did not remember, but I got it in the end. Managing admin accounts, including local administrator accounts is important to do. Watch Paula JanuszKiewicz give you an example why it is important here at one of the CQURE academy sessions about passing the hash.
Felt a little panic coming up at that point, because data loss or at least a lot of time fixing things can follow this action. Did not look like I could do much from here either. I did have backups of the data, so in time I would have restored it.
Another rerason for the panic is that I was doing two systems at the same time and in the same way.... and you guessed it... both with the same result!
A lot of googling open and there are a lot of videos explaining how to fix this FROM Windows! Problem is I am stuck in this System Recovery Options Screen. The memory check did not show anything by the way.
Well somewhere hidden in a comment of one of the threads (I can not find it!) was the suggestion that some previous hotfix might have hit one file and removing that file solved it for a few people.
In the picture above you can see a command prompt. Open that.
Next you need to find out which drive letter contains your Windows Installation. The System Recovery just uses a drive letter for itself and throws the other drives into other drive letters. So I did a C: Enter. DIR and knew this was not the drive. So I went to D: and did DIR again. Nope.. Continued until I got it.
The file I am looking for is fntcache.dat
this is the font cache file. Do NOT touch the DLL file there. The DAT file is a cache and will be re-built by Windows after restart.
Now I exited the command prompt and restarted the server. It started again into Windows where I hoped it would go.
Next I still needed to do a select-all on the rest of the updates and install them all the same
So keep in mind to update regularly + do not select half the updates but go for them all because there are fixes in there which fix issues created (or surfaced) by other fixes.
Now I can continue with actually replacing these servers, which was the plan to start with!
This week I added an Orchestrator 2012 Runbook server to an existing one for scale-out and high availability reasons. Very soon it was ready to go and I was making some additional runbooks to use together with SCOM. In these runbooks were Run .Net Script activities with PowerShell scripts in there. And I noticed the script activities would refuse to run. Except when I ran them separately as a normal PowerShell script. SO I went in the history and checked what had happened:
File C:\Program Files\System Center Operations Manager 2012\Powershell\OperationsManager\OperationsManager.psm1 cannot be loaded because the execution of scripts is disabled on this system. Please see "get-help about_signing" for more details.
O right! So I opened up a PowerShell prompt using Run As Administrator and I typed "Set-ExecutionPolicy Unrestricted".
And the script failed to run again! Wait a second perhaps it is because I forgot to run the same command on the other runbook server. Oops. OK running that command again and....
I went searching for it and I saw a comment in a thread somewhere saying it could be that the same command needs to be done for the 64 bit version of PowerShell as well!
Did this on all Runbook servers this time as well
And try again. Working fine!
Was monitoring an Exchange machine today and got a message that the Log disk contained less than x percent of storage. First thing to check if backups have run on the machine, because if those fail the log disk tends to fill up quickly. But the backups were OK and there was only a few hours of data on the disk. After short investigation it turned out to be the System Volume Information directory causing this and it had a lot of big files in it with a number of several GB each. Thing was most of these files were older than a year. So hereby a few quick commands:
First one to list the shadow copy space reserved and used for each disk:
vssadmin list shadowstorage
This got me some output and lets list here some stuff for the Log disk:
Used Shadow Copy Storage space: 52.481 GB (52%)
Allocated Shadow Copy Storage space: 52.983 GB (52%)
Maximum Shadow Copy Storage space: UNBOUNDED (100%)
Aha, so it had no limit and was already occupying over half the disk space and 52 GB. By the way the data disk was a factor 10 bigger in size and also had well over half the disk size used by only this stuff.
You can also open the Explorer and take the properties of the disk and look for the Shadow Copies tab. Also to make alterations.
In this case I used the available commands though. First I set the amount of disk space to use as maximum to only a few GB, which causes the old files to be deleted. Next I used the same command to raise the maximum space to be used again to a more suitable number.
vssadmin resize shadowstorage /on=E: /For=E: /Maxsize=19GB
So the above command sets the maximum size on the E drive to 19GB.
Sure got rid of the problem of Exchange management pack telling us during each day the Log disk was running below 50% free space already.
Very good, on to the next isue to be solved!
Last week I installed a SCOM agent on a Windows 2008 R2 print server, only to discover that the SCOM agent went wild and kept crashing. Furthermore I found out it was generating a lot of data (about 18 milion entries in a few hours). So what had happened? Well the reason I found in this blog post from Daniel Mueller dating from April 2013.
In short the Printer Server 2008 mp (version 6.0.7004.0) has a mistake in a few performance collection rules. They are targetted at "Printer" in stead of print server role. Because it collects data from all instances, this meant in my case it saw about 500 printer queues and multiplied that by 500 times, because every printer queue monitored all instances of all other printer queues as well. As you can see with a high number of printer queues this thing will go wild.
The Printer Server 2008 management pack dates from December 2012 and it looks like even after the blog post from Daniel nothing has happened with it. So I decided a few days ago that I will post a hotfix management pack which will disable the original faulty collection rules and create new rules with the same settings but correct targetting.
The "Hotfix Management Pack: Print Server 2008" can be found in the TechNet Gallery.
It is specifically to override the 6.0.7004.0 version of the Print Server 2008 management pack from Microsoft, so you do not run into the performance and agent issues.
And thanks to Daniel Mueller for recognizing this issue in the first place.
Yesterday the Windows Server Management Pack got a new update to version 6.0.7026.0 .
Changes in This Update
The April 2013 update (version 6.0.7026.0) of the Windows Server Operating System Management Pack contains the following changes:
- Fixed a bug in Microsoft.Windows.Server.2008.Monitoring.mp where the performance information for Processor was not getting collected.
- Made monitoring of Cluster Shared Volume consistent with monitoring of Logical Disks by adding performance collection rules. (“Cluster Shared Volume - Free space / MB”,”Cluster Shared Volume - Total size / MB”,”Cluster Shared Volume - Free space / %”,”Cluster Disk - Total size / MB”,”Cluster Disk - Free space / MB”,”Cluster Disk - Free space / %”)
- Fixed bug in Microsoft.Windows.Server.ClusterSharedVolumeMonitoring.mp where the Cluster disks running on Windows Server 2008 (non R2) were not discovered.
- Fixed bug 'Cluster Disk Free Space Percent' and Cluster Disk Free Space MB' monitors generate alerts with bad descriptions when the volume label of a cluster disk is empty.
- Added feature to raise event when NTLM requests time out and customers are unable to use mailboxes, outlook stops responding, due to the low default value for Max Concurrent API registry Key (HLM\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters) , which is a ceiling for the maximum NTLM or Kerberos PAC password validations a server can take care of at a time. It uses the “Netlogon” performance counter to check for the issue.
A few months ago Microsoft released newer versions of the Windows Server base OS management pack. This version had a good number of improvements, especially in the monitoring of cluster disk and cluster shared volumes. However the pack was still not complete enough in the sense that it missed the performance collection rules for Cluster Disks (the new class, replacing the Logical Disk pieces especially for CLustered instances).
My highly respected colleague MVP Daniele Grandini helped out by creating an addendum management pack for Cluster Disks which would take care of this. Explanation can be found here: Cluster Disks management pack addendum and he posted the management pack on the Technet Gallery over here: http://gallery.technet.microsoft.com/Cluster-Disks-management-99f43845.
This week I was loading new management packs - Windows Base OS and SQL and of course this addendum mp because they had a lot of clusters in this monitored environment. And discovered that the pack Daniele had authored was authored on a SCOM 2012, while I was loading it into a SCOM 2007 R2 at the moment and that did not work for two reasons. So I changed the management pack a bit to make it work with SCOM 2007 R2 as well and sent it back to Daniele and he immediately placed the changed MP on the TechNet Gallery entry mentioned above. Its great to get these adjustments out this fast and really proves how much we can do in the community by joining forces to make things work and share!
If you are into monitoring Clusters than take a good look at the latest WIndows Server mp and this addendum mp to catch a bit more (and more reliable) ifnormation about your high available systems.
Big thanks to Daniele Grandini!
Because of questions from two customers about high availability for Hyper-V virtual machines and NOT similar hardware between the hosts I was clicking around and found an interesting checking tool for processor compatibility for hosts that you want to add into a hyper-v cluster. This is because if virtual machines fail over and they discover they cant run the same processor feature set the failover will fail. And you might find out too late (like when it needs to failover!). Of course the ideal route would be to buy two or more exactly identical hosts and connect those to storage to create the cluster with. But in this case I was asked to check if I could find a combination with not similar machines and see if we can somehow make the virtuals higher available.
So first of all the technet page on hyper-v clustering (classic failover clustering but a page further the CSV live migration is also discussed).
And I found this tool the "Virtual Machine Migration Test Wizard":
With this one you can check in existing clusters, but also adding a machine to an existing cluster or just checking which machines you have running could be placed together in a cluster. This is all based at first on processor feature compatibility and is probably not the only thing you should check for
And in the comments of that page somebody also made a link to a short video explaining the why and how of this tool:
So, now lets see if I can play around with this a little.
Re-blogging the announcement as is:
Today we are pleased to announce the availability of the Release Candidate for Windows Small Business Server 2011 Essentials. Ideal as a first server for small businesses with up to 25 users, Windows Small Business Server 2011 Essentials provides a cost-effective and easy-to-use solution to help protect data, organize and access business information from virtually anywhere, support the applications needed to run a business, and quickly connect to online services for e-mail, collaboration and CRM.
The Release Candidate (RC) signifies our last mile efforts for releasing this product to our customers, and prepares our large network of software and solution partners for its release.
To fast track your learning, we have also launched our new SBS 2011 Essentials Learning Bites – fifteen 3-5 minute videos focusing on core features and functionality. Further videos, plus many other resources can be found on the official SBS website.
I almost feel stupid to say but i never fully understood how this really works. I read many articles and examples but still i never got the picture.
Until i read a blog of Kudrat Sapaev where it is easy and crystal clear explained!
The blog can be found at http://kudratsapaev.blogspot.com/2009/07/loopback-processing-of-group-policy.html
Bastiaan van Onselen
From the Microsoft Deployment Toolkit Team good news on an update to the MDT 2010:
Deploy Windows 7 and Office 2010 quickly and reliably—while boosting user satisfaction
Microsoft® Deployment Toolkit (MDT) 2010 Update 1 is now available! Download MDT 2010 Update 1 at: http://go.microsoft.com/fwlink/?LinkId=159061
As you prepare to deploy Windows® 7, Office 2010, and Windows Server® 2008 R2, get a jump start with MDT 2010 Update 1. Use this Solution Accelerator to achieve efficient, cost-effective deployment of Windows 7, Office 2010, and Windows Server 2008 R2.
This latest release offers something for everyone. Benefits include:
For System Center Configuration Manager 2007 customers:
New “User Driven Installation” deployment method. An easy-to-use UDI Wizard allows users to initiate and customize operating system and application deployments to their PCs that are tailored to their individual needs.
Support for Configuration Manager R3 “Prestaged Media.” For those deploying Windows 7 and Office 2010 along with new PCs, a custom operating system image can easily be preloaded and then customized once deployed.
For Lite Touch Installation:
Support for Office 2010. Easily configure Office 2010 installation and deployment settings through the Deployment Workbench and integration with the Office Customization Tool.
Improved driver import process. All drivers are inspected during the import process to accurately determine what platforms they really support, avoiding common inaccuracies that can cause deployment issues.
For all existing customers:
A smooth and simple upgrade process. Installing MDT 2010 Update 1 will preserve your existing MDT configuration, with simple wizards to upgrade existing deployment shares and Configuration Manager installations.
Many small enhancements and bug fixes. Made in direct response to feedback received from customers and partners all around the world, MDT 2010 Update 1 is an indispensible upgrade for those currently using MDT (as well as a great starting point for those just starting).
Continued support for older products. MDT 2010 Update 1 still supports deployment of Windows XP, Windows Server 2003, Windows Vista®, Windows Server 2008, and Office 2007, for those customers who need to be able to support these products during the deployment of Windows 7 and Office 2010.
Download Microsoft Deployment Toolkit 2010: http://go.microsoft.com/fwlink/?LinkId=159061.
Learn more by visiting the MDT site on Microsoft TechNet: www.microsoft.com/mdt.
Get the latest news by visiting the Microsoft Deployment Toolkit Team blog: http://blogs.technet.com/msdeployment/default.aspx.
Provide us with feedback at firstname.lastname@example.org.
If you have used a Solution Accelerator within your organization, please share your experience with us by completing this short survey: http://go.microsoft.com/fwlink/?LinkID=132579.
Microsoft Deployment Toolkit Team