Issues connecting SvcMgr 2012 to OpsMgr 2012 (Part 2)


Following the post Issues connecting SvcMgr 2012 to OpsMgr 2012 (Part 1) in part 2 I will detail debugging steps I have taken and a possible workaround.

Since I can repro the issue out of SM I started attaching a WinDbg to my sample application and I loaded psscor2 to enable Managed Code debug. I know that the problem is an InvalidCastException, so I put a breakpoint on that exception using the command “!soe –create System.InvalidCastException 1”, the following is the dump of the stack after the breakpoint was hit :

0:013> !ClrStack -a
OS Thread Id: 0x8d4 (13)
Child-SP         RetAddr          Call Site
RSP/REG          Object           Name
RSP/REG          Object           Name
000000001ce1dcf0 0000000005310da0 System.InvalidCastException
000000001ce1ddb0 0000000005310da0 System.InvalidCastException
RSP/REG          Object           Name
000000001ce1ddc8 0000000005310da0 System.InvalidCastException
RSP/REG          Object           Name
000000001ce1deb0 0000000005310da0 System.InvalidCastException
RSP/REG          Object           Name
000000001ce1df48 00000000024df268 System.DBNull
000000001ce1e000 000000000299adf0 Microsoft.EnterpriseManagement.ManagementGroup
000000001ce1e010 00000000031c3408 Microsoft.EnterpriseManagement.Common.Result
000000001ce1e020 0000000005310c80 Microsoft.EnterpriseManagement.Common.Internal.ImageReferenceGenerated

I found the System.DBNull on the stack that caused the System.InvalidCastException as seen in the Error found in EventLog. I dumped the object 00000000031c3408 Microsoft.EnterpriseManagement.Common.Result to see if the null value comes from there, confirmed :

0:013> !do 00000000031c3408
Name: Microsoft.EnterpriseManagement.Common.Result
MethodTable: 000007ff00239fd0
EEClass: 000007ff00219eb8

Fields:
MT            Field           Offset                 Type VT             Attr            Value Name
0000000000000000  40002e3        8                       0 instance 00000000031c3420 valueList

next the 00000000031c3420 valueList :

0:013> !do 00000000031c3420
Name: System.Collections.Generic.List`1[[System.Object, mscorlib]]
MethodTable: 000007fef2dbb418
EEClass: 000007fef29ccd38
Fields:
MT            Field           Offset                 Type VT             Attr            Value Name
000007fef2db58f8  4000a29        8      System.Object[]  0 instance 00000000031c34d8 _items
000007fef2dced78  4000a2a       18         System.Int32  1 instance                8 _size
000007fef2dced78  4000a2b       1c         System.Int32  1 instance                8 _version
000007fef2dc73f8  4000a2c       10        System.Object  0 instance 0000000000000000 _syncRoot
000007fef2db58f8  4000a2d        0      System.Object[]  0   shared           static _emptyArray

ant then 00000000031c34d8 _items using sosex debugger extension :

0:013> !mdt 00000000031c34d8
00000000031c34d8 (System.Object[], Elements: 8)  expand
0:013> !mdt -e:1 00000000031c34d8
00000000031c34d8 (System.Object[], Elements: 8)  expand
[0] 00000000031c3448 (BOXED System.Guid) {f8a35325-6412-4020-32de-926bac141887} VALTYPE (MT=000007fef2dc2410, ADDR=00000000031c3450)
[1] 00000000031c3468 (BOXED System.Guid) {627f4dff-40f7-89d6-6d0d-71102b6e8cea} VALTYPE (MT=000007fef2dc2410, ADDR=00000000031c3470)
[2] 00000000031c3488 (BOXED System.Guid) {496e66b8-3b96-0e46-c49f-d04480280952} VALTYPE (MT=000007fef2dc2410, ADDR=00000000031c3490)
[3] 00000000031c34a8 (BOXED System.DateTime) 2010/01/19 18:38:48.657 VALTYPE (MT=000007fef2e08040, ADDR=00000000031c34b0)
[4] 00000000031c34c0 (BOXED System.DateTime) 2010/01/19 18:38:40.907 VALTYPE (MT=000007fef2e08040, ADDR=00000000031c34c8)
[5] 00000000024df268 (System.DBNull)
<— This is the DBNull object found on the stack, it is confirmed that it comes from the Result object
[6] 00000000031c3538 “Microsoft.Windows.Cluster”
[7] 00000000031c3588 (BOXED System.Boolean) BOXEDVAL=true

then I searched another object of type result on the stack to see a valid value for the 5th item :

0:013> !mdt -e:1 00000000031b9510
00000000031b9510 (System.Object[], Elements: 8)  expand
[0] 00000000031b9480 (BOXED System.Guid) {04717ff4-a2e3-a078-d3b2-c9e8bbb2d463} VALTYPE (MT=000007fef2dc2410, ADDR=00000000031b9488)
[1] 00000000031b94a0 (BOXED System.Guid) {55270a70-ac47-c853-c617-236b0cff9b4c} VALTYPE (MT=000007fef2dc2410, ADDR=00000000031b94a8)
[2] 00000000031b94c0 (BOXED System.Guid) {2e9ceaf4-334b-2357-2514-9628dba09b91} VALTYPE (MT=000007fef2dc2410, ADDR=00000000031b94c8)
[3] 00000000031b94e0 (BOXED System.DateTime) 2008/11/07 15:40:58.690 VALTYPE (MT=000007fef2e08040, ADDR=00000000031b94e8)
[4] 00000000031b94f8 (BOXED System.DateTime) 2008/11/07 15:40:58.490 VALTYPE (MT=000007fef2e08040, ADDR=00000000031b9500)
[5] 00000000031b9570 “u16x16Icon”

[6] 00000000031b95a0 “System.Entity”
[7] 00000000031b95d8 (BOXED System.Boolean) BOXEDVAL=true

it seems that the “Microsoft.Windows.Cluster” managed type has an invalid icon. I searched the stack for the ResultSet object that contains the Result object, to retrieve the name of the columns, I found the following in the output of ClrStack :

000000001ce1e390 00000000031b8c58 Microsoft.EnterpriseManagement.Common.ResultSet
000000001ce1e3a0 000007ff001c71db Microsoft.EnterpriseManagement.Configuration.TypeSpaceCache.RefreshMonitoringClassCache(System.Collections.ObjectModel.ReadOnlyCollection`1<Microsoft.EnterpriseManagement.Common.ResultSet>)

the ResultSet dump, contains ResultList and columnDefinitions

0:013> !mdt  00000000031b8c58
00000000031b8c58 (Microsoft.EnterpriseManagement.Common.ResultSet)
resultList:00000000031fbd50 (Microsoft.EnterpriseManagement.Common.Result[], Elements: 579)  expand
columnDefinitions:00000000031b93e8 (Microsoft.EnterpriseManagement.Common.ColumnDefinition[], Elements: 7)  expand

I expanded the resultList, element at index 87 is the Result found in the output of ClrStack with the System.DBNull :

[85] 00000000031c3048 (Microsoft.EnterpriseManagement.Common.Result)
[86] 00000000031c3228 (Microsoft.EnterpriseManagement.Common.Result)
[87] 00000000031c3408 (Microsoft.EnterpriseManagement.Common.Result) <— Result set with System.DBNull
[88] 00000000031c35a0 (Microsoft.EnterpriseManagement.Common.Result)
[89] 00000000031c3738 (Microsoft.EnterpriseManagement.Common.Result)

finally I dumped columnDefinitions to retrieve the name of the column with the DBNull value :

0:013> !mdt -e:2 00000000031b93e8
00000000031b93e8 (Microsoft.EnterpriseManagement.Common.ColumnDefinition[], Elements: 7)  expand
[0] 00000000031b8c78 (Microsoft.EnterpriseManagement.Common.ColumnDefinition)
columnName:00000000031b8cb8 (System.String: “ImageId”)
sourceColumnName:00000000031b8e60 (System.String: “ImageId“)
aliased:false (System.Boolean)
isExpression:false (System.Boolean)
columnSource:00000000031b8ce0 (System.String: “ImageReference”)
columnType:00000000024ea038 (System.RuntimeType)
columnDatabaseTypeAsInt:0xe (System.Int32)
columnMaxLength:0xffffffff (System.Int32)
isNullable:false (System.Boolean)
[1] 00000000031b8e88 (Microsoft.EnterpriseManagement.Common.ColumnDefinition)
columnName:00000000031b8ec8 (System.String: “ReferenceId”)
sourceColumnName:00000000031b8f30 (System.String: “ReferenceId“)
aliased:false (System.Boolean)
isExpression:false (System.Boolean)
columnSource:00000000031b8ef8 (System.String: “ImageReference”)
columnType:00000000024ea038 (System.RuntimeType)
columnDatabaseTypeAsInt:0xe (System.Int32)
columnMaxLength:0xffffffff (System.Int32)
isNullable:false (System.Boolean)
[2] 00000000031b8f60 (Microsoft.EnterpriseManagement.Common.ColumnDefinition)
columnName:00000000031b8fa0 (System.String: “ManagementPackId”)
sourceColumnName:00000000031b9018 (System.String: “ManagementPackId“)
aliased:false (System.Boolean)
isExpression:false (System.Boolean)
columnSource:00000000031b8fe0 (System.String: “ImageReference”)
columnType:00000000024ea038 (System.RuntimeType)
columnDatabaseTypeAsInt:0xe (System.Int32)
columnMaxLength:0xffffffff (System.Int32)
isNullable:false (System.Boolean)
[3] 00000000031b9058 (Microsoft.EnterpriseManagement.Common.ColumnDefinition)
columnName:00000000031b9098 (System.String: “TimeAdded”)
sourceColumnName:00000000031b9100 (System.String: “TimeAdded“)
aliased:false (System.Boolean)
isExpression:false (System.Boolean)
columnSource:00000000031b90c8 (System.String: “ImageReference”)
columnType:00000000024df1a0 (System.RuntimeType)
columnDatabaseTypeAsInt:0x4 (System.Int32)
columnMaxLength:0xffffffff (System.Int32)
isNullable:false (System.Boolean)
[4] 00000000031b9130 (Microsoft.EnterpriseManagement.Common.ColumnDefinition)
columnName:00000000031b9170 (System.String: “LastModified”)
sourceColumnName:00000000031b91e0 (System.String: “LastModified“)
aliased:false (System.Boolean)
isExpression:false (System.Boolean)
columnSource:00000000031b91a8 (System.String: “ImageReference”)
columnType:00000000024df1a0 (System.RuntimeType)
columnDatabaseTypeAsInt:0x4 (System.Int32)
columnMaxLength:0xffffffff (System.Int32)
isNullable:false (System.Boolean)
[5] 00000000031b9218 (Microsoft.EnterpriseManagement.Common.ColumnDefinition)
columnName:00000000031b9258 (System.String: “ImageCategory”)
sourceColumnName:00000000031b92c8 (System.String: “ImageCategory”) <—- The Column with DBNull value
aliased:false (System.Boolean)
isExpression:false (System.Boolean)
columnSource:00000000031b9290 (System.String: “ImageReference”)
columnType:00000000024ca3c8 (System.RuntimeType)
columnDatabaseTypeAsInt:0xc (System.Int32)
columnMaxLength:0x64 (System.Int32)
isNullable:false (System.Boolean)
[6] 00000000031b9300 (Microsoft.EnterpriseManagement.Common.ColumnDefinition)
columnName:00000000031b9340 (System.String: “MPElementName”)
sourceColumnName:00000000031b93b0 (System.String: “MPElementName“)
aliased:false (System.Boolean)
isExpression:false (System.Boolean)
columnSource:00000000031b9378 (System.String: “ImageReference”)
columnType:00000000024ca3c8 (System.RuntimeType)
columnDatabaseTypeAsInt:0xc (System.Int32)
columnMaxLength:0x100 (System.Int32)
isNullable:false (System.Boolean)

Checking the queries against the OpsMgr db I found the following query :

exec dbo.p_SelectForTypeCache @LanguageCode1=‘ENU’,@LanguageCode2=NULL,@LastKnownMPLastModified=‘1753-01-01 00:00:00′,@LastKnownMPCacheRefreshTimestamp=‘1753-01-01 00:00:00′

The Exception occurs in a SDK method named TypeSpaceCache.RefreshMonitoringClassCache so the SP named SelectForTypeCache is a good candidate. The output of the SP is the following, columns have the same name as the ones found in the ResultSet :

image

Now inspecting the TSQL code we can find the SP has been changed with OM2012 (and an intriguing comment) :

– OMG IS THIS WRONG!!!
— BUGBUG
— HACK

SELECT [ImageReference].[ImageId], [ImageReference].[ReferenceId], [ImageReference].[ManagementPackId], [ImageReference].TimeAdded, [ImageReference].LastModified, SUBSTRING([EnumType].EnumTypeName, 39, 100) AS ImageCategory, [MPElementView].[MPElementName],
[ManagementPack].[ContentReadable]
FROM dbo.ImageReference
JOIN dbo.[Resource] on ([Resource].ResourceId = ImageReference.ImageId)
LEFT JOIN dbo.[Category] ON [Category].CategoryTarget = [Resource].ResourceId
LEFT JOIN dbo.[EnumType] ON [EnumType].EnumTypeId = [Category].CategoryValue
JOIN dbo.MPElementView ON (ImageReference.ReferenceId = MPElementView.MPElementId)
INNER JOIN dbo.[ManagementPack] ON dbo.ManagementPack.ManagementPackId = [ImageReference].ManagementPackId AND dbo.ManagementPack.ContentReadable = 1
WHERE dbo.[ImageReference].[ReferenceId] IN (select dbo.ManagedType.ManagedTypeId from dbo.ManagedType)
and dbo.[Resource].ResourceType = 4;

This is the analogous on 2007 :

SELECT [ImageReference].[ImageId], [ImageReference].[ReferenceId], [ImageReference].[ManagementPackId], [ImageEntity].TimeAdded, [ImageEntity].LastModified, [ImageEntity].[ImageCategory], [MPElementView].[MPElementName],
[ManagementPack].[ContentReadable]
FROM dbo.ImageReference
JOIN dbo.ImageEntity on (ImageEntity.ImageId = ImageReference.ImageId)
JOIN dbo.MPElementView ON (ImageReference.ReferenceId = MPElementView.MPElementId)
INNER JOIN dbo.[ManagementPack] ON dbo.ManagementPack.ManagementPackId = [ImageReference].ManagementPackId
WHERE dbo.[ImageReference].[ReferenceId] IN (select dbo.ManagedType.ManagedTypeId from dbo.ManagedType)

The point here is the 2012 query returns a lot of null values in ImageCategory, while the 2007 query doesn’t. Here is the result of the query (2012):

clip_image001

We get Null values because of missing records in the table Category that is new in OpsMgr 2012. OpsMgr DB has been upgraded from 2007 to 2012, it’s not a green field. I suspect that the upgrade process for some reason does not create an entry in the table Category for every entry in the OpsMgr 2007 table ImageEntity.

A possible workaround is to identify ManagementPacks with missing entries in the table category and for each of them delete and reimport the MP. I tried with a couple of them and now the category table is filled with the missing entries.

Italian version of this post is available here

- Fabrizio

About these ads
  1. Issues connecting SvcMgr 2012 to OpsMgr 2012 (Part 1) « Quae Nocent Docent

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 348 other followers

%d bloggers like this: