Issues connecting SvcMgr 2012 to OpsMgr 2012 (Part 2)

Following the post Issues connecting SvcMgr 2012 to OpsMgr 2012 (Part 1) in part 2 I will detail debugging steps I have taken and a possible workaround.

Since I can repro the issue out of SM I started attaching a WinDbg to my sample application and I loaded psscor2 to enable Managed Code debug. I know that the problem is an InvalidCastException, so I put a breakpoint on that exception using the command “!soe –create System.InvalidCastException 1”, the following is the dump of the stack after the breakpoint was hit :

0:013> !ClrStack -a
OS Thread Id: 0x8d4 (13)
Child-SP         RetAddr          Call Site
RSP/REG          Object           Name
RSP/REG          Object           Name
000000001ce1dcf0 0000000005310da0 System.InvalidCastException
000000001ce1ddb0 0000000005310da0 System.InvalidCastException
RSP/REG          Object           Name
000000001ce1ddc8 0000000005310da0 System.InvalidCastException
RSP/REG          Object           Name
000000001ce1deb0 0000000005310da0 System.InvalidCastException
RSP/REG          Object           Name
000000001ce1df48 00000000024df268 System.DBNull
000000001ce1e000 000000000299adf0 Microsoft.EnterpriseManagement.ManagementGroup
000000001ce1e010 00000000031c3408 Microsoft.EnterpriseManagement.Common.Result
000000001ce1e020 0000000005310c80 Microsoft.EnterpriseManagement.Common.Internal.ImageReferenceGenerated

I found the System.DBNull on the stack that caused the System.InvalidCastException as seen in the Error found in EventLog. I dumped the object 00000000031c3408 Microsoft.EnterpriseManagement.Common.Result to see if the null value comes from there, confirmed :

0:013> !do 00000000031c3408
Name: Microsoft.EnterpriseManagement.Common.Result
MethodTable: 000007ff00239fd0
EEClass: 000007ff00219eb8

MT            Field           Offset                 Type VT             Attr            Value Name
0000000000000000  40002e3        8                       0 instance 00000000031c3420 valueList

next the 00000000031c3420 valueList :

0:013> !do 00000000031c3420
Name: System.Collections.Generic.List`1[[System.Object, mscorlib]]
MethodTable: 000007fef2dbb418
EEClass: 000007fef29ccd38
MT            Field           Offset                 Type VT             Attr            Value Name
000007fef2db58f8  4000a29        8      System.Object[]  0 instance 00000000031c34d8 _items
000007fef2dced78  4000a2a       18         System.Int32  1 instance                8 _size
000007fef2dced78  4000a2b       1c         System.Int32  1 instance                8 _version
000007fef2dc73f8  4000a2c       10        System.Object  0 instance 0000000000000000 _syncRoot
000007fef2db58f8  4000a2d        0      System.Object[]  0   shared           static _emptyArray

ant then 00000000031c34d8 _items using sosex debugger extension :

0:013> !mdt 00000000031c34d8
00000000031c34d8 (System.Object[], Elements: 8)  expand
0:013> !mdt -e:1 00000000031c34d8
00000000031c34d8 (System.Object[], Elements: 8)  expand
[0] 00000000031c3448 (BOXED System.Guid) {f8a35325-6412-4020-32de-926bac141887} VALTYPE (MT=000007fef2dc2410, ADDR=00000000031c3450)
[1] 00000000031c3468 (BOXED System.Guid) {627f4dff-40f7-89d6-6d0d-71102b6e8cea} VALTYPE (MT=000007fef2dc2410, ADDR=00000000031c3470)
[2] 00000000031c3488 (BOXED System.Guid) {496e66b8-3b96-0e46-c49f-d04480280952} VALTYPE (MT=000007fef2dc2410, ADDR=00000000031c3490)
[3] 00000000031c34a8 (BOXED System.DateTime) 2010/01/19 18:38:48.657 VALTYPE (MT=000007fef2e08040, ADDR=00000000031c34b0)
[4] 00000000031c34c0 (BOXED System.DateTime) 2010/01/19 18:38:40.907 VALTYPE (MT=000007fef2e08040, ADDR=00000000031c34c8)
[5] 00000000024df268 (System.DBNull)
<— This is the DBNull object found on the stack, it is confirmed that it comes from the Result object
[6] 00000000031c3538 “Microsoft.Windows.Cluster”
[7] 00000000031c3588 (BOXED System.Boolean) BOXEDVAL=true

then I searched another object of type result on the stack to see a valid value for the 5th item :

0:013> !mdt -e:1 00000000031b9510
00000000031b9510 (System.Object[], Elements: 8)  expand
[0] 00000000031b9480 (BOXED System.Guid) {04717ff4-a2e3-a078-d3b2-c9e8bbb2d463} VALTYPE (MT=000007fef2dc2410, ADDR=00000000031b9488)
[1] 00000000031b94a0 (BOXED System.Guid) {55270a70-ac47-c853-c617-236b0cff9b4c} VALTYPE (MT=000007fef2dc2410, ADDR=00000000031b94a8)
[2] 00000000031b94c0 (BOXED System.Guid) {2e9ceaf4-334b-2357-2514-9628dba09b91} VALTYPE (MT=000007fef2dc2410, ADDR=00000000031b94c8)
[3] 00000000031b94e0 (BOXED System.DateTime) 2008/11/07 15:40:58.690 VALTYPE (MT=000007fef2e08040, ADDR=00000000031b94e8)
[4] 00000000031b94f8 (BOXED System.DateTime) 2008/11/07 15:40:58.490 VALTYPE (MT=000007fef2e08040, ADDR=00000000031b9500)
[5] 00000000031b9570 “u16x16Icon”

[6] 00000000031b95a0 “System.Entity”
[7] 00000000031b95d8 (BOXED System.Boolean) BOXEDVAL=true

it seems that the “Microsoft.Windows.Cluster” managed type has an invalid icon. I searched the stack for the ResultSet object that contains the Result object, to retrieve the name of the columns, I found the following in the output of ClrStack :

000000001ce1e390 00000000031b8c58 Microsoft.EnterpriseManagement.Common.ResultSet
000000001ce1e3a0 000007ff001c71db Microsoft.EnterpriseManagement.Configuration.TypeSpaceCache.RefreshMonitoringClassCache(System.Collections.ObjectModel.ReadOnlyCollection`1<Microsoft.EnterpriseManagement.Common.ResultSet>)

the ResultSet dump, contains ResultList and columnDefinitions

0:013> !mdt  00000000031b8c58
00000000031b8c58 (Microsoft.EnterpriseManagement.Common.ResultSet)
resultList:00000000031fbd50 (Microsoft.EnterpriseManagement.Common.Result[], Elements: 579)  expand
columnDefinitions:00000000031b93e8 (Microsoft.EnterpriseManagement.Common.ColumnDefinition[], Elements: 7)  expand

I expanded the resultList, element at index 87 is the Result found in the output of ClrStack with the System.DBNull :

[85] 00000000031c3048 (Microsoft.EnterpriseManagement.Common.Result)
[86] 00000000031c3228 (Microsoft.EnterpriseManagement.Common.Result)
[87] 00000000031c3408 (Microsoft.EnterpriseManagement.Common.Result) <— Result set with System.DBNull
[88] 00000000031c35a0 (Microsoft.EnterpriseManagement.Common.Result)
[89] 00000000031c3738 (Microsoft.EnterpriseManagement.Common.Result)

finally I dumped columnDefinitions to retrieve the name of the column with the DBNull value :

0:013> !mdt -e:2 00000000031b93e8
00000000031b93e8 (Microsoft.EnterpriseManagement.Common.ColumnDefinition[], Elements: 7)  expand
[0] 00000000031b8c78 (Microsoft.EnterpriseManagement.Common.ColumnDefinition)
columnName:00000000031b8cb8 (System.String: “ImageId”)
sourceColumnName:00000000031b8e60 (System.String: “ImageId“)
aliased:false (System.Boolean)
isExpression:false (System.Boolean)
columnSource:00000000031b8ce0 (System.String: “ImageReference”)
columnType:00000000024ea038 (System.RuntimeType)
columnDatabaseTypeAsInt:0xe (System.Int32)
columnMaxLength:0xffffffff (System.Int32)
isNullable:false (System.Boolean)
[1] 00000000031b8e88 (Microsoft.EnterpriseManagement.Common.ColumnDefinition)
columnName:00000000031b8ec8 (System.String: “ReferenceId”)
sourceColumnName:00000000031b8f30 (System.String: “ReferenceId“)
aliased:false (System.Boolean)
isExpression:false (System.Boolean)
columnSource:00000000031b8ef8 (System.String: “ImageReference”)
columnType:00000000024ea038 (System.RuntimeType)
columnDatabaseTypeAsInt:0xe (System.Int32)
columnMaxLength:0xffffffff (System.Int32)
isNullable:false (System.Boolean)
[2] 00000000031b8f60 (Microsoft.EnterpriseManagement.Common.ColumnDefinition)
columnName:00000000031b8fa0 (System.String: “ManagementPackId”)
sourceColumnName:00000000031b9018 (System.String: “ManagementPackId“)
aliased:false (System.Boolean)
isExpression:false (System.Boolean)
columnSource:00000000031b8fe0 (System.String: “ImageReference”)
columnType:00000000024ea038 (System.RuntimeType)
columnDatabaseTypeAsInt:0xe (System.Int32)
columnMaxLength:0xffffffff (System.Int32)
isNullable:false (System.Boolean)
[3] 00000000031b9058 (Microsoft.EnterpriseManagement.Common.ColumnDefinition)
columnName:00000000031b9098 (System.String: “TimeAdded”)
sourceColumnName:00000000031b9100 (System.String: “TimeAdded“)
aliased:false (System.Boolean)
isExpression:false (System.Boolean)
columnSource:00000000031b90c8 (System.String: “ImageReference”)
columnType:00000000024df1a0 (System.RuntimeType)
columnDatabaseTypeAsInt:0x4 (System.Int32)
columnMaxLength:0xffffffff (System.Int32)
isNullable:false (System.Boolean)
[4] 00000000031b9130 (Microsoft.EnterpriseManagement.Common.ColumnDefinition)
columnName:00000000031b9170 (System.String: “LastModified”)
sourceColumnName:00000000031b91e0 (System.String: “LastModified“)
aliased:false (System.Boolean)
isExpression:false (System.Boolean)
columnSource:00000000031b91a8 (System.String: “ImageReference”)
columnType:00000000024df1a0 (System.RuntimeType)
columnDatabaseTypeAsInt:0x4 (System.Int32)
columnMaxLength:0xffffffff (System.Int32)
isNullable:false (System.Boolean)
[5] 00000000031b9218 (Microsoft.EnterpriseManagement.Common.ColumnDefinition)
columnName:00000000031b9258 (System.String: “ImageCategory”)
sourceColumnName:00000000031b92c8 (System.String: “ImageCategory”) <—- The Column with DBNull value
aliased:false (System.Boolean)
isExpression:false (System.Boolean)
columnSource:00000000031b9290 (System.String: “ImageReference”)
columnType:00000000024ca3c8 (System.RuntimeType)
columnDatabaseTypeAsInt:0xc (System.Int32)
columnMaxLength:0x64 (System.Int32)
isNullable:false (System.Boolean)
[6] 00000000031b9300 (Microsoft.EnterpriseManagement.Common.ColumnDefinition)
columnName:00000000031b9340 (System.String: “MPElementName”)
sourceColumnName:00000000031b93b0 (System.String: “MPElementName“)
aliased:false (System.Boolean)
isExpression:false (System.Boolean)
columnSource:00000000031b9378 (System.String: “ImageReference”)
columnType:00000000024ca3c8 (System.RuntimeType)
columnDatabaseTypeAsInt:0xc (System.Int32)
columnMaxLength:0x100 (System.Int32)
isNullable:false (System.Boolean)

Checking the queries against the OpsMgr db I found the following query :

exec dbo.p_SelectForTypeCache @LanguageCode1=‘ENU’,@LanguageCode2=NULL,@LastKnownMPLastModified=‘1753-01-01 00:00:00’,@LastKnownMPCacheRefreshTimestamp=‘1753-01-01 00:00:00’

The Exception occurs in a SDK method named TypeSpaceCache.RefreshMonitoringClassCache so the SP named SelectForTypeCache is a good candidate. The output of the SP is the following, columns have the same name as the ones found in the ResultSet :


Now inspecting the TSQL code we can find the SP has been changed with OM2012 (and an intriguing comment) :


SELECT [ImageReference].[ImageId], [ImageReference].[ReferenceId], [ImageReference].[ManagementPackId], [ImageReference].TimeAdded, [ImageReference].LastModified, SUBSTRING([EnumType].EnumTypeName, 39, 100) AS ImageCategory, [MPElementView].[MPElementName],
FROM dbo.ImageReference
JOIN dbo.[Resource] on ([Resource].ResourceId = ImageReference.ImageId)
LEFT JOIN dbo.[Category] ON [Category].CategoryTarget = [Resource].ResourceId
LEFT JOIN dbo.[EnumType] ON [EnumType].EnumTypeId = [Category].CategoryValue
JOIN dbo.MPElementView ON (ImageReference.ReferenceId = MPElementView.MPElementId)
INNER JOIN dbo.[ManagementPack] ON dbo.ManagementPack.ManagementPackId = [ImageReference].ManagementPackId AND dbo.ManagementPack.ContentReadable = 1
WHERE dbo.[ImageReference].[ReferenceId] IN (select dbo.ManagedType.ManagedTypeId from dbo.ManagedType)
and dbo.[Resource].ResourceType = 4;

This is the analogous on 2007 :

SELECT [ImageReference].[ImageId], [ImageReference].[ReferenceId], [ImageReference].[ManagementPackId], [ImageEntity].TimeAdded, [ImageEntity].LastModified, [ImageEntity].[ImageCategory], [MPElementView].[MPElementName],
FROM dbo.ImageReference
JOIN dbo.ImageEntity on (ImageEntity.ImageId = ImageReference.ImageId)
JOIN dbo.MPElementView ON (ImageReference.ReferenceId = MPElementView.MPElementId)
INNER JOIN dbo.[ManagementPack] ON dbo.ManagementPack.ManagementPackId = [ImageReference].ManagementPackId
WHERE dbo.[ImageReference].[ReferenceId] IN (select dbo.ManagedType.ManagedTypeId from dbo.ManagedType)

The point here is the 2012 query returns a lot of null values in ImageCategory, while the 2007 query doesn’t. Here is the result of the query (2012):


We get Null values because of missing records in the table Category that is new in OpsMgr 2012. OpsMgr DB has been upgraded from 2007 to 2012, it’s not a green field. I suspect that the upgrade process for some reason does not create an entry in the table Category for every entry in the OpsMgr 2007 table ImageEntity.

A possible workaround is to identify ManagementPacks with missing entries in the table category and for each of them delete and reimport the MP. I tried with a couple of them and now the category table is filled with the missing entries.

Italian version of this post is available here

– Fabrizio

  1. #1 by Andy Williams on October 2, 2015 - 10:29 am

    Thank you so much for this post. I had spent a good many hours searching the net trying to work out why my SCSM 2012 R2 system wouldn’t connect to SCOM 2012 R2, both were running the latest URs and have been built from scratch as opposed to upgraded from previous versions. When I fired the 2012 TSQL code you listed above at my OppsMgr DB I found one row with Null under Image Category. This related to an MP for our NetApp Storage system, I removed the MP deleted the failed connector and recreated the connector and it worked first time. Great post!

  1. Issues connecting SvcMgr 2012 to OpsMgr 2012 (Part 1) « Quae Nocent Docent

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: