Monday, March 11, 2013

Microsoft System Center Operations Manager (SCOM) 2012 - Event 29181 for Store Index Optimize work item - How to find such rules running

Hi Guru's,

I recently worked on another SCOM 2012 case and below is a summary of it.

Problem:

Event 29181 generating in OM log for SQL exception related to below two workflows.

• Configuration Store index optimization work item
• Configuration Store statistics update work item 

So, How to find 2 workflow which thrown an exception with event 29181, a question came up in mind that how to check scheduled rule that is running the indexing and stats update
These rules running every hourly and creating huge DB traffic however unable to view from console.


Error Message :
Description: Management Configuration Service group failed to perform Configuration Store index optimization work item for a period of time. Last error message (if available): Microsoft.EnterpriseManagement.ManagementConfiguration.DataAccessLayer.DataAccessException: Data access operation failed    at Microsoft.EnterpriseManagement.ManagementConfiguration.DataAccessLayer.DataAccessOperation.ExecuteSynchronously(Int32 timeoutSeconds, WaitHandle stopWaitHandle)    at Microsoft.EnterpriseManagement.ManagementConfiguration.SqlConfigurationStore.IndexOptimizeWorkItem.ExecuteSharedWorkItem()    at Microsoft.EnterpriseManagement.ManagementConfiguration.Interop.SharedWorkItem.ExecuteWorkItem() ----------------------------------- System.Data.SqlClient.SqlException (0x80131904): Sql execution failed. Error 2801, Level 16, State 1, Procedure ServerAuditTrigger, Line 34, Message: The definition of object ServerAuditTrigger has changed since it was compiled.    at System.Data.SqlClient.SqlConnection.OnError(SqlException


Description: Management Configuration Service group failed to perform Configuration Store statistics update work item for a period of time. Last error message (if available): Microsoft.EnterpriseManagement.ManagementConfiguration.DataAccessLayer.DataAccessException: Data access operation failed    at Microsoft.EnterpriseManagement.ManagementConfiguration.DataAccessLayer.DataAccessOperation.ExecuteSynchronously(Int32 timeoutSeconds, WaitHandle stopWaitHandle)    at Microsoft.EnterpriseManagement.ManagementConfiguration.SqlConfigurationStore.StatisticsUpdateWorkItem.ExecuteSharedWorkItem()    at Microsoft.EnterpriseManagement.ManagementConfiguration.Interop.SharedWorkItem.ExecuteWorkItem() ----------------------------------- System.Data.SqlClient.SqlException (0x80131904): Sql execution failed. Error 2801, Level 16, State 1, Procedure ServerAuditTrigger, Line 34, Message: The definition of object ServerAuditTrigger has changed since it was compiled.    at System.Data.SqlClient.SqlConnection.OnError(SqlException



Here, Followed step by step action plan as below.

Action plan1 :
First thing can look at powershell, lets see example of Heartbeat Health service,  can use below command to get results.

can use PowerShell or read MP files xml and dig through the file to find the monitor or alert’s ID.  
Open Operations Manager Shell and run this command. 
get-alert -criteria 'Name = ''Health Service Heartbeat Failure''' 
This will work if you have alerts that are available in the management group. 
The data returned will have a line for MonitoringRuleId.  This is the value you can specify in the AlertSource section for Id. 
MonitoringRuleId : b59f78ce-c42a-8995-f099-e705dbb34fd4  
What if do not have any recent alerts that can query with PowerShell to find the Id?  get-monitor -criteria 'DisplayName = ''Health Service Heartbeat Failure''' 
Id: b59f78ce-c42a-8995-f099-e705dbb34fd4  You can use similar command line for rules by using get-rule command.  To get the GroupIdFilter you can use PowerShell to get the Group ID similarly. 
get-monitoringobjectgroup  Find the group you want and find the associated Id. 
Id: 9e249559-e166-0e92-1bfc-fea90a63f843 
There is an article for 2007 version,   http://blogs.technet.com/b/stefan_stranger/archive/2010/11/30/what-monitors-rules-and-discoveries-are-running-on-an-opsmgr-agent.aspx  

Then observed it shows all monitors and rules running but still not those 2 trying to find on which SQL exception thrown.

Action Plan 2:
Then tried to look at this article to get the info for all the things running on server or an agent. (Use simply reporting feature) 
http://technet.microsoft.com/en-us/library/hh916937.aspx  

however it will display all the rules or monitors running executed by MP or from console, not a system hard coded.

Action Plan 3:
Then try to look from console itself by following below steps.
To view all rules and monitors running on a computer 
1. Open the Operations console and click Monitoring. 
2. For an agent-managed computer, navigate to the Operations Manager\Agent Details\Agent Health State view.  For a management server, navigate to Operations Manager\Management Server\Management Servers State view. 
3. Click the agent you want to see rules and monitors for. 
4. In the Tasks pane, select the task Show Running Rules and Monitors for this Health Service. 
5. The Run Task – Show Running Rules and Monitors for this Health Service dialog box appears. Click Run.  
6. The Task Status dialog box appears. When the task is completed, you can click Copy Text or Copy HTML and paste the task output in the appropriate tool for analysis

Resolution:

Finally conclusion is, below 2 rules are hard coded into system and by default it will execute every hour to run against each Database.

• Configuration Store index optimization work item
• Configuration Store statistics update work item 

This is the default configuration.   
The work item ones just optimize the configuration and the “OptimizeIndex” one optimizes the dataset in OperationsManager DB. 

When we start the System Center Management Configuration service, the config service engine work item will be loaded. (CS.WorkItemDescriptorList)   
They will be processed one by one on the dirty agents (config file out-of-date).
When the former completed, the next will be got by CS.WorkItemGetNext.   
All the interval time for the jobs are hard coded. We don’t support to modify them.  CS.IndexOptimize and CS.IndexOptimize are the exact ones in this case.   
You can find the details under OperationsManager database. 


So, to Summurize,

Problem

How to find the work items from configuration service 

Solution

CS.StatisticsUpdate and CS.IndexOptimize are the configuration store work items.
If the work items are not processed successfully, it will still exist in the CS.workitem table and workitemstateID =1. Other management server will pick this item up and process again until it is successfully processed and workitemstateID marked as 20. 


If the work items will be finally processed, we can ignore the failure events.   

More Info

Tune the configurations of work items  http://blogs.technet.com/b/momteam/archive/2013/01/29/support-tip-config-service-deltasynchronization-process-fails-with-timeout-exception.aspx 

In fact, we can actually have a configuration file to set the frequency of the work items executed.   
Its path is C:\Program Files\System Center 2012\Operations Manager\Server\ConfigService.config   
You can modify the value of the setting in this file. (You may need to stop the System Center Management Configuration service before the modification).   


 








Wednesday, January 23, 2013

Microsoft System Center Operation Manager 2012 - Unable to view DW perf data in SCOM Console

Hi Guys, I am posting another article for SCOM 2012 product on which I worked recently and resolved one of my customer issue.

OS - Microsoft Windows 2008 R2 SP1
Apps - Microsoft System Center Operation Manager (SCOM) 2012
DB - Microsoft SQL server 2012

While created rule to view performance data for DW Synchronization, it is not viewable in SCOM console however same data visible in 'Perfmon' tool.

(DW Synchronization server & DW Connection server)

Solution
While checked, found attached are 2 screenshot for the rule defined in SCOM 2012 in order to view performance data for DW server.
Here, found that these rules are target to the Data Warehouse Synchronization Server instead of Data Warehouse Connection Server whereas both are diffrent rules defined.
So, either need to target to correct class for existing rule or need to create new rule targeting to DW connection server instead of DW synchronization server in order to get the perf data for DW server in SCOM console.
After targetting to correct class, i.e to DW connection server, issue got resolved.


So, by summarize this,

Problem:
========= 

Unable to view perf data from “Synchronize Performance” view   

Analysis:
========== 

No collection rule collects the performance data from Data Warehouse Connection Server.  After manually created rules targeted to performance counters in the object “OpsMgr DW Synchronization module”, this issue got resolved and able to see the performance data correctly. 

Wednesday, October 10, 2012

SCOM 2012 problem description changes not getting saved

Hi Guys, I am back. I am posting one more SCOM case here.

OS - Microsoft Windows 2008 R2
App - Microsoft SCOM 2012

Issue - Whenever creating any rule and once created, tried to edit he alert description, all modification changes on problem description not getting saved.

Tried repro in internal LAB as below.

1. Created a NT Event Log (Alert) rule.

2. The description is default.
3. Modified the description and saved.
4. Re-checked and found no issues.


So it worked at my end but tested on SCOM 2012 with CU1 whereas in this case there is no CU installed. Here can perform more test to isolate the issue like,
• Delete the whole ENU language pack section and see what happens. 
• Create a custom view in monitoring pane and save in the same MP to check if we can see the changes in MP. 

Solution :-

Imported customer management pack into LAB for further testing and test outcome as below.   
1.When my computer’s locale is set to ENU. I can modify the rule properties, alert names and etc. smoothly without any issue. 
2.When my computer’s locale is set to ENA. I can modify them once and this modification is under ENU. But it is interesting that the next time I modify it, it will reflect in ENA language pack.   
I keep the change of LanguagePack ID="ENA" IsDefault="true" in MP languagepack section.   

So the solution was to change the system locale (format) to ENU.

To conclude, the SCOM console will always display the name under ENU language pack. This is why we always cannot see the change while the system locale is ENA. 

Monday, September 24, 2012

Hi Guys,

I got one more SCOM issue on which I spend many hours to troubleshoot and finally identify the cause area and resolved it hence thought to share here for SCOM 2007 R2 product.

Microsoft SCOM 2007 R2 Enterprise - Randomly entire Management Group becomes greyed out

Setup - All nodes are part of VM
OS - Microsoft Windows Server 2008 R2 Enterprise
App - Microsoft System Center Operation Manager 2007 R2 Enterprise
DB - Microsoft SQL Server 2008 R2

Issue:-
Randomly, the management group becomes greyed out in SCOM 2007 R2 environment. however health service on each monitored system still healthy.

In this case, followed step by step troubleshooting in order to identify the area of cause and necessary action towards resolution.

1. Start perfmon capturing with necessory counters or use command below.
Logman.exe create counter Perf-1Sec -f bincirc -max 500 -c "\LogicalDisk(*)\*" "\Memory\*" "\Network Interface(*)\*" "\Paging File(*)\*" "\PhysicalDisk(*)\*"  "\Server\*" "\System\*" "\Process(*)\*" "\Processor(*)\*"  "\Cache\*" -si 00:00:01 -o C:\PerfMonLogs\Perf-1Sec.blg
Logman.exe start Perf-1Sec   

2. Wait for problem to report again and then stop the perfmon log. Here can either use Perfmon console or “Logman stop xxxx”

3. create a dump file for health service on RMS first.     
A.Open task manager, right click health service.exe to create a dump file.  
B.After dump file is created, please go to the temp folder and copy this dump file to a safe location. After OS restart, these dump file could be clean up.     

4. get a SCOM trace.
A. Stop the healthservice (tried to stop the service from services.msc, if the process hang during the stopping process, can create another dump for it and then terminate using Task Manager)  
B. After stop healthservice, open a command line and go to “c:\program files\System Center Operations Manager 2007\Tools” folder. Try the following command:      
Stoptracing
Del c:\windows\temp\OpsMgrTrace\*.*  
StartTracing VER     

NOTE: VER is case sensitive.      
C.       Start healthservice and wait for 10 minutes to check if the service is recovered  
D.       If not, please stop and capture the trace:      
Stoptracing  
formattracing     
E.       Capture all the log file under c:\windows\temp\OpsMgrTrace     

5. once all the data is captured and you are ready to reboot the system to recover, instead of restart the system, please use http://download.sysinternals.com/files/NotMyFault.zip to trigger a blue screen on the system. At that time, the system will crash and start to dump all the memory to C:\Windows\memory.dmp. this file will record the entire OS status.  

Note - If dump cannot be captured using bug check, please help to check if this method can work? 
Click HERE

After over with analysis, it was suspected issue not with SCOM side but SQL performance and conflicting with Backup job running on same time.
Hence captured perfmon log on all Database servers as well where found Disk Latency issue on logs.

Conclusion:
When look at the system, even though the management group becomes Gray out, it is the Health Service Watcher object is grey, the health service running on each monitored system is still healthy. Since the watcher objects is running on RMS, the problem is more related to the RMS status.
By looking at the problem history, although the issue is reported randomly, it is found the most likely, the problem is reported during midnight. Usually, that is the time for backup tasks.
Before RMS report error, we can see that SCOM SDK service always reports error for connecting to the remote DB. Thus, we checked the task on SQL side. In most cases, the RMS error time matches with the DB backup schedule. For a test, we disabled the backup job during midnight, the RMS problem disappears.     Further check the SQL server, it is found some disk latency happens. Thus, we believe the RMS issue is not caused by SCOM configuration. Actually, it is a victim of SQL performance.
After fix the SQL latency problem, SCOM has been stable.  

Reference - Here is the process to promote an MS to RMS:  Click HERE

Friday, September 14, 2012

How to check which GPO applied and which registry changing by GPO

Hi Guys,
I am adding one more article here because I feel it would be more benificial for all of us who worked on Microsoft platform under Administrative task, many of us worked or working with Group Policy, even I worked for many years but intresting is, I never saw which registry being changed by applying Group Policy Objects on server.

You can open RUN box from start menu, enter " RSOP.MSC" which will open a seperate window for Resultant set of Policies and you can see all policy applied to box.



Once the console opens you will be able to see which settings have been applied to your PC.
Note: Only settings that have been applied to your machine and user account will show up.



You can use command prompt as many are lover of it, When using the command line, it should be noted that you have to specify the scope of the results. To find all the policies that are applied to your user account, you would use the following command:
"gpresult /Scope User /v"  (Here you can save it to text file by adding >filename.txt)

Then if you scroll down, you will see the the Resultant Set Of Policies for User section.


If you are looking for all policies applied to your Computer, all you need to do is change the scope:
"gpresult /Scope Computer /v"

If you scroll down, now you will now see that there is a Resultant Set Of Policies for Computer section.



Now, question is, how do we check which registry settings added by modified group policy object. so we can use one of fantastic tool, Process Monitor here.

You can download it from MS Sys-Internal.
http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx

Then extract and run it locally.
When Proc Mon opens, you will need to add a condition as follows:
"Process Name is mmc.exe then Include"
Then click the add button.



To get only the registry keys that are changed, we need add another one:
"Operation is RegSetValue then Include"
Then again click the add button.



Once the two rules have been added, you can go ahead and click ok.



Now go and open the Group Policy setting that you wish to edit.



Before you actually change the setting, switch back over to Proc Mon and clear the log.



Then go and change the GPO and click apply.



If you switch over to Proc Mon you will see that you have a registry key(s) there. Right-click on it and select the Jump To… option from the context menu.

That will fire up Regedit and take you to the exact key which was modified

Friday, August 17, 2012

SCOM 2012 secondary server installation failed

Hi,

I came up with another article for the issue of Microsoft System Center Operation Manager 2012 secondary server installation issue. So sharing my experience here how I dealt with this situation.

OS - Microsoft Windows 2008 R2
Apps - Microsoft System Center Operation Manager 2012

Issue - While try to install SCOM 2012 secondary server into existing Management Group by selecting custom port for Data-Ware house, installation gets failed.



Here I have analyzed the set up log and found below error.


Error:  :StackTrace:   at System.Net.Dns.InternalGetHostByName(String hostName, Boolean includeIPv6)
at System.Net.Dns.GetHostEntry(String hostNameOrAddress)
at Microsoft.EnterpriseManagement.OperationsManager.SetupCommon.SetupUtils.CheckIsRemoteSqlServer(String sqlInstanceName)
at Microsoft.EnterpriseManagement.OperationsManager.Setup.Common.SetupHelpers.GetAccountForDB(Account dasAccount, String sqlServerInstance, String& dasAccountForDB)
   at Microsoft.SystemCenter.Essentials.SetupFramework.InstallItemsDelegates.OMDataWarehouseProcessor.SetDataWarehouseSecurity()
[16:17:36]:     Error:  :FATAL ACTION: SetDataWarehouseSecurity
[16:17:36]:     Error:  :FATAL ACTION: DWInstallActionsPostProcessor
[16:17:36]:     Error:  :ProcessInstalls: Running the PostProcessDelegate returned false.

Cause:
The setup failed because we use 1443 as the SQL service port for data warehouse server, there’s a known issue that when GUI passed the port number to the setup, the format is not correct and causing exception.

The work around is to install the management server with command line, however the SDK and data write account have special characters in the password and could not be used in CMD prompt.

Resolution:
We change the port back to default 1433 port from SQL server configuration manager and use following steps to update the databases.

1.     On the server hosting the operational database, update the OperationsManager database table.
a.   Open SQL Server Management Studio.
b.   Expand Databases, OperationsManager, and Tables.
c.   Right-click dbo. MT_Microsoft$SystemCenter$DataWarehouse, and then click Edit Top 200 Rows.
d.   Change the value in the MainDatabaseServerName_2C77AA48_DB0A_5D69_F8FF_20E48F3AED0F column from server name,1433 to server name.
e.   Close SQL Server Management Studio.

2.     On the data warehouse server, update the member database.
a.   Open SQL Server Management Studio.
b.   Expand Databases, OperationsManagerDW, and Tables.
c.   Right-click dbo. MemberDatabase, and then click Edit Top 200 Rows.
d.   Change the value in the ServerName column from server name,1433 to server name
e.   Close SQL Server Management Studio.

After the changes had been made, we rerun the setup and the management server was successfully installed. So using another port will not work by design and treated as a Bug,

Reference-

Installing Operations Manager by Using the Command Prompt Window - See Here

How to install Operations Manager 2012 single server configuration from the command prompt - See Here

How to Install Additional Management Servers - See Here



Friday, August 3, 2012

Microsoft System Center Configuration Manager 2007 R3 - unable to download patches - 2. Failed to find updates with error code 800B0001

Hi Guys,

I recently worked upon SCCM case for failing downloading windows updates with below error therefore sharing it here in my blog with solution implemented so that it will help to whoever facing same issue.

Failed to find updates with error code 800B0001

OS- Microsoft Windows 2008 R2
Application - Microsoft SCCM 2007 R3

As Microsoft does not support with SP1 - See Here 
Check Windowsupdate.log & Patchdownloader.log file for error details.
Windowsupdate.log Finding:

2012-07-17          15:46:38:808       816        2168       Misc       WARNING: Error: 0x800b0001 when verifying trust for C:\Windows\SoftwareDistribution\SelfUpdate\wuident.cab
2012-07-17          15:46:38:808       816        2168       Misc       WARNING: Digital Signatures on file C:\Windows\SoftwareDistribution\SelfUpdate\wuident.cab are not trusted: Error 0x800b0001
2012-07-17          15:46:38:808       816        2168       Setup    WARNING: Self Update check failed to download package information, error = 0x800B0001
2012-07-17          15:46:38:809       816        2168       Setup    FATAL: Self Update check failed, err = 0x800B0001
2012-07-17          15:46:38:867       816        2168       Agent      * WARNING: Skipping scan, self-update check returned 0x800B0001
2012-07-17          15:46:38:922       816        2168       Agent      * WARNING: Exit code = 0x800B0001
2012-07-17          15:46:38:922       816        2168       Agent    WARNING: WU client failed Searching for update with error 0x800b0001
2012-07-17          15:46:38:938       816        11e4      AU          >>##  RESUMED  ## AU: Search for updates [CallId = {C4B3D200-5463-4446-9432-EB74507F131E}]
2012-07-17          15:46:38:938       816        11e4      AU            # WARNING: Search callback failed, result = 0x800B0001
2012-07-17          15:46:38:939       816        11e4      AU            # WARNING: Failed to find updates with error code 800B0001

Patchdownloader.log finding

Contentsource = http://download.windowsupdate.com/msdownload/update/software/secu/2012/06/windows6.1-kb2698365-x64_bf20bb36fc73c0d1f53ea1e635b8aa46c71d7b1f.cab . Software Updates Patch Downloader 7/25/2012 5:55:17 AM 10344 (0x2868)
Downloading content for ContentID = 9349,  FileName = windows6.1-kb2698365-x64.cab. Software Updates Patch Downloader 7/25/2012 5:55:17 AM 10344 (0x2868)
Download
http://download.windowsupdate.com/msdownload/update/software/secu/2012/06/windows6.1-kb2698365-x64_bf20bb36fc73c0d1f53ea1e635b8aa46c71d7b1f.cab in progress: 10 percent complete Software Updates Patch Downloader 7/25/2012 5:55:18 AM 9484 (0x250C)http://download.windowsupdate.com/msdownload/update/software/secu/2012/06/windows6.1-kb2698365-x64_bf20bb36fc73c0d1f53ea1e635b8aa46c71d7b1f.cab in progress: 61 percent complete Software Updates Patch Downloader 7/25/2012 5:55:18 AM 9484 (0x250C)Download http://download.windowsupdate.com/msdownload/update/software/secu/2012/06/windows6.1-kb2698365-x64_bf20bb36fc73c0d1f53ea1e635b8aa46c71d7b1f.cab in progress: 81 percent complete Software Updates Patch Downloader 7/25/2012 5:55:18 AM 9484 (0x250C)
Download
http://download.windowsupdate.com/msdownload/update/software/secu/2012/06/windows6.1-kb2698365-x64_bf20bb36fc73c0d1f53ea1e635b8aa46c71d7b1f.cab in progress: 91 percent complete Software Updates Patch Downloader 7/25/2012 5:55:18 AM 9484 (0x250C)
Download
http://download.windowsupdate.com/msdownload/update/software/secu/2012/06/windows6.1-kb2698365-x64_bf20bb36fc73c0d1f53ea1e635b8aa46c71d7b1f.cab to C:\Users\svc_sccm\AppData\Local\Temp\CAB1D33.tmp returns 0 Software Updates Patch Downloader 7/25/2012 5:55:19 AM 9484 (0x250C)
Checking machine config Software Updates Patch Downloader 7/25/2012 5:55:19 AM 9484 (0x250C)
Cert revocation check is disabled so cert revocation list will not be checked. Software Updates Patch Downloader 7/25/2012 5:55:19 AM 9484 (0x250C)
To enable cert revocation check use: UpdDwnldCfg.exe /checkrevocation Software Updates Patch Downloader 7/25/2012 5:55:19 AM 9484 (0x250C)
Authentication of file C:\Users\svc_sccm\AppData\Local\Temp\CAB1D33.tmp failed, error 0x800b0004 Software Updates Patch Downloader 7/25/2012 5:55:19 AM 9484 (0x250C)
ERROR: DownloadContentFiles() failed with hr=0x80073633 Software Updates Patch Downloader 7/25/2012 5:55:19 AM 10344 (0x2868)

it was happening for all patches and not to specific.

I was getting the same error message in windows update log file as stated above and as Microsoft strengthened the WSUS communication channels in the last month or so, which may explain why older patches worked but newer ones, are not working hence see here.

Additionally you can refer this article as well - See Here

However in my case, issue was still exist. I searched for KB 272011 but they havn't instaled it. So suggested to
install 2718704 and then 2720211

Also during my remote session, I noticed that any of Microsoft update link like http://gva1swparis.hq.intra.who.int:8530/SelfUpdate/wuident.cab downloading sucesfully However, the Digital Signature tab is missing from the file (go to properties, next to general TAB, it should show TAB for 'Digital Signature'. So Looks like there's a problem with the server itself. Since we move the same file to a different machine, we're able to see the Digital signature tab.

Also tried by changing security authentication mode but no sucess.

So, finally I resolved the issue. I have specified details below.

Cause:-

This problem may occur if one or more of the following conditions are true:
  • Log file or database corruption exists in the %Systemroot%\System32\Catroot2 folder.
  • Cryptographic Services is set to disabled.
  • Other Windows files are corrupted or missing.
  • The timestamp signature or certificate could not be verified or is malformed.
  • The hidden attribute is set for the %Windir% folder or one of its subfolders.
  • The Unsigned non-driver installation behavior Group Policy setting (Windows 2000 only) is set to Do not allow installation or Warn but allow installation, or the Policy binary value is not set to 0 in the following registry key:
    HKEY_LOCAL_MACHINE\Software\Microsoft\Non-Driver Signing
  • The Enable trusted publisher lockdown Group Policy setting is turned on, and you do not have the appropriate certificate in your Trusted Publishers certificate store. This Group Policy setting is located under User Configuration, under Windows Settings, under Internet Explorer Maintenance, under Security, under Authenticode Settings in the Group Policy MMC snap-in.
    Resolution:-
    1. Set Cryptographic Services to automatic, it was set to Manual earlier
    2. Rename catroot2 folder by stopping cryptsvc service, rename to oldcatroot2, start service again & then removed tmp *.cat files from
    %systemroot%\system32\CatRoot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE}

    Note - If no files that start with tmp exist in this folder, do not remove any other files. The .cat files in this folder are necessary for installing hotfixes and service packs.
    3. Reregister the DLL files that are associated with Cryptographic Services
    Go to command prompt by 'Run as Administrator'
    regsvr32 /u softpub.dll
    regsvr32 /u wintrust.dll
    regsvr32 /u initpki.dll
    regsvr32 /u dssenh.dll
    regsvr32 /u rsaenh.dll
    regsvr32 /u gpkcsp.dll
    regsvr32 /u sccbase.dll
    regsvr32 /u slbcsp.dll
    regsvr32 /u mssip32.dll
    regsvr32 /u cryptdlg.dll
     if some files failed, ignore it for next try and restart the server
    Once done, again do same excercise and re-register following
    regsvr32 softpub.dll
    regsvr32 wintrust.dll
    regsvr32 initpki.dll
    regsvr32 dssenh.dll
    regsvr32 rsaenh.dll
    regsvr32 gpkcsp.dll
    regsvr32 sccbase.dll
    regsvr32 slbcsp.dll
    regsvr32 mssip32.dll
    regsvr32 cryptdlg.dll
    then restart box again.
    4. Remove the hidden attribute from %Windir% and from its subfolders
    Launch command prompt again with Administrator access and type following
attrib -s -h %windir%
attrib -s -h %windir%\system32
attrib -s -h %windir%\system32\catroot2
exit
    5. Rename EDB.Log file by launching command prompt
ren %systemroot%\system32\catroot2\Edb.log *.tst
    6. Temporarily turn off Trusted Publishers Lockdown and install the appropriate certificates to your trusted publishers certificate storeYou can continue to use the Enable trusted publisher lockdown Group Policy setting, but you must first add the appropriate certificates to your Trusted Publishers certificate store. To do this, turn off the Enable trusted publisher lockdown Group Policy setting, install the appropriate certificates in your Trusted Publishers certificate store, and then turn the Enable trusted publisher lockdown Group Policy setting back on. To install the appropriate certificate for Microsoft Windows and Microsoft Internet Explorer product updates, follow these steps:
    1. Download the Microsoft product update that you want to install from the Microsoft Download Center, from the Windows Update Catalog, or from the Microsoft Update Catalog. For more information about how to download product updates from the Microsoft Download Center, click the following article number to view the article in the Microsoft Knowledge Base:
      119591 How to obtain Microsoft support files from Online Services
      For more information about how to download product updates from the Windows Update Catalog, click the following article number to view the article in the Microsoft Knowledge Base:
      323166 How to download updates that include drivers and hotfixes from the Windows Update Catalog
    2. Extract the product update package to a temporary folder. The command-line command that you use to do this depends on the update that you are trying to install. View the Microsoft Knowledge Base article that is associated with the update to determine the appropriate command-line switches that you will use to extract the package. For example, to extract the 824146 security update for Windows XP to the C:\824146 folder, run Windowsxp-kb824146-x86-enu -x:c:\824146. To extract the 828750 security update for Windows XP to the C:\828750 folder, run q828750.exe /c /t:c:\828750.
    3. Right-click the KBNumber.cat file from the product update package in the temporary folder you created in step 2, and then click Properties.

      Note The KBNumber.cat file may be in a subfolder. For example, the file may be in the C:\824146\sp1\update folder or in the C:\824146\sp2\update folder.
    4. On the Digital Signatures tab, click the digital signature and then click Details.
    5. Click View Certificate, and then click Install Certificate.
    6. Click Next to start the Certificate Import Wizard.
    7. Click Place all certificates in the following store, and then click Browse.
    8. Click Trusted Publishers, and then click OK.
    9. Click Next, click Finish, and then click OK.

    7. Verify the status of all certificates in the certification path and import missing or damaged certificates from another computerTo verify certificates in the certificate path for a Windows or Internet Explorer product update, follow these steps:

    Step 1: Verify Microsoft certificates

    1. In Internet Explorer, click Tools, and then click Internet Options.
    2. On the Content tab, click Certificates.
    3. On the Trusted Root Certification Authorities tab, double-click Microsoft Root Authority. If this certificate is missing, go on to step 2.
    4. On the General tab, make sure that the Valid from dates are 1/10/1997 to 12/31/2020.
    5. On the Certification Path tab, verify that This certificate is OK appears under Certificate Status.
    6. Click OK, and then double-click the NO LIABILITY ACCEPTED certificate.
    7. On the General tab, make sure that the Valid from dates are 5/11/1997 to 1/7/2004.
    8. On the Certification Path tab, verify that either This certificate has expired or is not yet valid or This certificate is OK appears under Certificate Status.

      Note Although this certificate is expired, the certificate will continue to work. The operating system may not work correctly if the certificate is missing or revoked.
    9. Click OK, and then double-click the GTE CyberTrust Root certificate. You may have more than one of these certificates with the same name. Check the certificate that has an expiration date of 2/23/2006.
    10. On the General tab, make sure that the Valid from dates are "2/23/1996 to 2/23/2006."
    11. On the Certification Path tab, verify that This certificate is OK appears under Certificate Status.
      Click OK, and then double-click Thawte Timestamping CA.
    12. On the General tab, make sure that the Valid from dates are "12/31/1996 to 12/31/2020."
    13. On the Certification Path tab, verify that This certificate is OK appears under Certificate Status.

    Step 2: Import missing or damaged certificates

    If one or more of these certificates are missing or corrupted, export the missing or corrupted certificates to another computer, and then install the certificates on your computer. To export certificates on another computer, follow these steps:
    1. In Internet Explorer, click Tools, and then click Internet Options.
    2. On the Content tab, click Certificates.
    3. On the Trusted Root Certification Authorities tab, click the certificate that you want to export.
    4. Click Export, and then follow the instructions to export the certificate as a DER encoded Binary x.509(.CER) file.
    5. After the certificate file has been exported, copy it to the computer where you want to import it.
    6. On the computer where you want to import the certificate, double-click the certificate.
    7. Click Install certificate, and then click Next.
    8. Click Finish, and then click OK.
   8. Clear the temporary file and restart the hotfix installation or the service pack installation
To clear the temporary file and restart the hotfix installation or the service pack installation, follow these steps:

  1. Delete all the tmp*.cat files in the following folders:

    %systemroot%\system32\CatRoot\{127D0A1D-4EF2-11D1-8608-00C04FC295EE}

    %systemroot%\system32\CatRoot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE}
  2. Delete all the kb*.cat files in the following folders:
    %systemroot%\System32\CatRoot\{F750E6C3-38EE-11D1-85E5-00C04FC295EE}
    %systemroot%\System32\CatRoot\{127D0A1D-4EF2-11D1-8608-00C04FC295EE}
  3. Delete all the oem*.* files from the %systemroot%\inf folder.
  4. At the command prompt, type the following commands. Press ENTER after each command.
    net stop cryptsvc
    ren %systemroot%\System32\Catroot2 oldcatroot2
    net start cryptsvc
    exit
  5. Restart the failed hotfix installation or service pack installation.
9. Empty the software distribution folder
  1. Click Start, click Run, type services.msc, and then click OK.

    Note On a Windows Vista-based computer, click Start, type services.msc in the Start Search box, right-click services.msc, and then click Run as administrator.
  2. In the Services (Local) pane, right-click Automatic Updates, and then click Stop.
  3. Minimize the Services (local) window.
  4. Select all the contents of the Windows distribution folder, and then delete them.

    Note By default, the Windows distribution folder is located in the drive:\Windows\SoftwareDistribution folder. In this location, drive is a placeholder for the drive where Windows is installed.
  5. Make sure that the Windows distribution folder is empty, and then maximize the Services (local) window.
  6. In the Services (Local) pane, right-click Automatic Updates, and then click Start.
  7. Restart the computer, and then run Windows Update again.
By following all above step by step, it helped me to resolved my issue.