One module to rule them all – a custom #scom managed module #sysctr


One module to rule them all – a custom #scom managed module

During many years of MP development I faced cases where I would have loved the chance to pass a complete DataItem to a powershell module. There are many real world cases when you’d want to do that:

          You want to leverage the native OLEDB provider and you want to correlate multiple returned rows

          You need to use internal data sources that do not return property bags, but rather custom DataItems (for example the Microsoft.SystemCenter.GetWorkflowsReportProbe that can be used to query the agent for running and failed workflows)

          you need to take care of an entire walk from the standard SNMP modules and add logic to create your own data

          You don’t know how many properties the DataItem has in its payload and you want dynamically process them all

Today this is not possible for a couple of reasons:

1)      You cannot reference the whole DataItem from the previous module, for example writing $Data$

2)      The POSH module strips away any XML tag, so even if you find a way to pass an XML fragment to a POSH module all the XML tags are stripped away and you end up with a useless sequence of values.

For those who are not MP developers (yet) and are interested a DataItem is the way OpsMgr modules can pass data one to each other. It’s an XML document with DataItem as the root element:

Simple DataItem

<?xml version=”1.0″?><DataItem type=”BaseData” time=”1900-01-01T00:00:00″ sourceHealthServiceId=”00000000-0000-0000-0000-000000000000″></DataItem>

A more complex one from the Microsoft.SystemCenter.GetWorkflowsReportProbe module is

WorkflowsReport DataItem

<DataItem type=”WorkflowsReport” time=”2014-07-05T10:36:52.1304369+02:00″ sourceHealthServiceId=”00000000-0000-0000-0000-000000000000″><Status>Failed</Status><Count>1</Count><Details><Instance Id=”{760DB2AB-8591-FF63-3660-8165664B0B11}”><Workflow>Microsoft.Windows.Server.2008.AppServer.InstancesGroup.Discovery</Workflow></Instance></Details></DataItem>

And this is an example of a property bag DataItem

PropertyBag dataItem

<DataItem type=”System.PropertyBagData” time=”2014-07-04T15:23:46.4343432Z” sourceHealthServiceId=”0ebf800d-fc70-ddbc-58ee-565ae0e76db8″><Property Name=”Index” VariantType=”3″>0</Property><Property Name=”Description” VariantType=”8″>Propert bag 0</Property></DataItem>

The goal is to pass the entire DataItem as-is to a powershell module to be able to use all the POSH power to manipulate the data.

I’m proud to announce that now we can achieve this goal, I just released a custom managed module that make it possible. The current version is a working version, but still not finished. I need to add tracing (hoping some help from the team since it’s not documented how to integrate in standard tracing) and add some polishing. But if you need and dare you can use it today.

You can find the module on codeplex (and once polished on Technet Gallery): https://scommm.codeplex.com

The basics

The module is a custom ConditionDetection batching enabled (more on this later) that takes any DataItem in input and outputs the encoded DataItem embedded  in a parent DataItem. Returning to the previous sample DataItem:

Original

<DataItem type=”WorkflowsReport” time=”2014-07-05T10:36:52.1304369+02:00″ sourceHealthServiceId=”00000000-0000-0000-0000-000000000000″><Status>Failed</Status><Count>1</Count><Details><Instance Id=”{760DB2AB-8591-FF63-3660-8165664B0B11}”><Workflow>Microsoft.Windows.Server.2008.AppServer.InstancesGroup.Discovery</Workflow></Instance></Details></DataItem>

Becomes

Encoded

<DataItem type=”QND.EncodedDataItem” time=”2014-07-05T08:36:53.2084844Z” sourceHealthServiceId=”0ebf800d-fc70-ddbc-58ee-565ae0e76db8″><EncodedItem>&amp;lt;DataItem type=”WorkflowsReport” time=”2014-07-05T10:36:52.1304369+02:00″ sourceHealthServiceId=”00000000-0000-0000-0000-000000000000″&amp;gt;&amp;lt;Status&amp;gt;Failed&amp;lt;/Status&amp;gt;&amp;lt;Count&amp;gt;1&amp;lt;/Count&amp;gt;&amp;lt;Details&amp;gt;&amp;lt;Instance Id=”{760DB2AB-8591-FF63-3660-8165664B0B11}”&amp;gt;&amp;lt;Workflow&amp;gt;Microsoft.Windows.Server.2008.AppServer.InstancesGroup.Discovery&amp;lt;/Workflow&amp;gt;&amp;lt;/Instance&amp;gt;&amp;lt;/Details&amp;gt;&amp;lt;/DataItem&amp;gt;</EncodedItem></DataItem>

And

Original

<DataItem type=”System.PropertyBagData” time=”2014-07-04T15:31:33.6367941Z” sourceHealthServiceId=”0ebf800d-fc70-ddbc-58ee-565ae0e76db8″><Property Name=”Index” VariantType=”3″>9</Property><Property Name=”Description” VariantType=”8″>Propert bag 9</Property></DataItem>

Becomes

Encoded

<DataItem type=”QND.EncodedDataItem” time=”2014-07-04T15:31:34.0438042Z” sourceHealthServiceId=”0ebf800d-fc70-ddbc-58ee-565ae0e76db8″><EncodedItem>&amp;lt;DataItem type=”System.PropertyBagData” time=”2014-07-04T15:31:33.6367941Z” sourceHealthServiceId=”0ebf800d-fc70-ddbc-58ee-565ae0e76db8″&amp;gt;&amp;lt;Property Name=”Index” VariantType=”3″&amp;gt;9&amp;lt;/Property&amp;gt;&amp;lt;Property Name=”Description” VariantType=”8″&amp;gt;Propert bag 9&amp;lt;/Property&amp;gt;&amp;lt;/DataItem&amp;gt;</EncodedItem></DataItem>

Basically the new dataItem has a single tag <EncodedItem> that contains the encoded and XML friendly version of the original dataItem. The encoding is achieved using the standard System.Xml.XmlWriter .NET class.

The receiving POSH module just needs to use System.Xml.XmlReader to un-ecode the payload and get the original XML document.

The condition detection module accept one parameter: ReturnMultipleItems of type Boolean. This is allow to manage batching. When a module is batch enabled it takes all the dataItem on its input ports before writing anything in output.

Example. You have a datasource who returns multiple objects you want to filter and then discover. Pretty common scenario many native e modules implements. The workflow is composed like this: datasource->filter->discovery mapper. This composition can work only if the filter module implements batching. It takes all the DataItem, discards the ones that do not match the filter and post all the remaining ones in a single batch. If the filter module were not batch enabled then you would have just the first object discovered.

The encoder condition detection implements batching and more. If the ReturnMultipleItems is true it will output (in batch) all the input dataItem as it received them. So if it gets 10 dataItem it will return a batch of 10 encoded DataItem. If ReturnMultipleItems is false it will always return a single DataItem where the encoded payload is the sum of all the input DataItem.

Why? you would ask. A couple of reasons:

1)      POSH modules are not batch enabled so to make everything work batching must be implemented

2)      Its way more efficient to call a single POSH module for all dataItems instead of calling a POSH module for each returned DataItem

In the end this parameter put control in your hands.

How to use it

The first step is to import QND.DataItemEncoder.Library.mpb, once imported you have access to the following condition detection:

XML

      <ConditionDetectionModuleType ID=”QND.Library.DataItemEncoder” Accessibility=”Public” Batching=”true”>

        <Configuration>

          <xsd:element name=”ReturnMultipleItems” type=”xsd:boolean” />

        </Configuration>

        <ModuleImplementation>

          <Managed>

            <Assembly>Res.QND.DataItemEncoder</Assembly>

            <Type>QND.DataItemEncoder</Type>

          </Managed>

        </ModuleImplementation>

        <OutputType>QND.EncodedDataItem</OutputType>

        <InputTypes>

          <InputType>System!System.BaseData</InputType>

        </InputTypes>

      </ConditionDetectionModuleType>

Then yyou just add the ConditionDetection module between the datasource and the POSH module where you want to process the DataItem:

Module Composition

<ModuleImplementation>

  <Composite>

    <MemberModules>

      <ProbeAction ID=WF TypeID=MSI!Microsoft.SystemCenter.GetWorkflowsReportProbe>

        <ManagementGroupId>$Config/ManagementGroupId$</ManagementGroupId>

        <ManagedEntityId>$Config/ManagedEntityId$</ManagedEntityId>

        <Status>Failed</Status>

      </ProbeAction>

      <ProbeAction ID=Test TypeID=Windows!Microsoft.Windows.PowerShellProbe>

        <ScriptName>QND.DecodeInput.ps1</ScriptName>

        <ScriptBody>

          $IncludeFileContent/Scripts/DecodeInput.ps1$

        </ScriptBody>

        <Parameters>

          <Parameter>

            <Name>EncodedDataItem</Name>

            <Value>$Data/EncodedItem$</Value>

          </Parameter>

        </Parameters>

        <TimeoutSeconds>30</TimeoutSeconds>

        <StrictErrorHandling>true</StrictErrorHandling>

      </ProbeAction>

      <ConditionDetection ID=CD TypeID=QDL!QND.Library.DataItemEncoder>

        <ReturnMultipleItems>true</ReturnMultipleItems>

      </ConditionDetection>

    </MemberModules>

    <Composition>

      <Node ID=Test>

        <Node ID=CD>

        <Node ID=WF />

      </Node>

      </Node>

    </Composition>

  </Composite>

</ModuleImplementation>

Note the $Data/EncodedItem$ substitution to access the encoded body.

The get back the original DataItem you jut need to add the following lines of POSH to your module:

Powershell

param($EncodedDataItem)

$reader = New-Object System.IO.StringReader $encodedDataItem

$Settings = New-Object System.Xml.XmlReaderSettings

 $settings.ConformanceLevel=[System.Xml.ConformanceLevel]::Fragment

 $xmlReader = [System.Xml.XmlReader]::Create($reader,$Settings)

 $xmlReader.Read()

 $unencoded = [xml]$xmlReader.ReadString()

 

In $unencoded you add the orginal DataItem XML document.

The sample Management Pack

To make it easier to ujnderstand and use the module you can find a companion helper management pack QND.EncodedDataItem.Tester.xml on CodePlex.

The Management Pack implements 3 Tasks targeted at the HealthService class:

          QND DIE Test 1 in 1 out – this task task takes a single DataItem in input from Microsoft.SystemCenter.GetWorkflowsReportProbe and outputs a single DataItem showing the endoded and decoded item.

          QND DIE Test N in N outthis task takes a collection of 10 property bags and returns a collections of 10 encoded dataItems. The task will show just the first DataItem since POSH doesn’t support batching.

          QND DIE Test N in 1 outthis task takes a collection of 10 property bags and returns a single DataItem composed by all 10 DataItems. The task will show the endoded and decodeded composed DataItem

 

References

Implementing Managed Modules – MSDN (http://msdn.microsoft.com/en-us/library/hh769912.aspx)

System Center 2012 – Operations Manager Managed Module Samples – CodePlex (https://opsmgr2012sdk.codeplex.com/)

  • Daniele

This posting is provided “AS IS” with no warranties, and confers no rights.

Advertisements
  1. #1 by bobgreen84 on September 26, 2017 - 6:32 pm

    I unpacked PowerShell Probe Action Module and it allows full DataItem natively if you use $Data$ in parameter value.

    • #2 by Daniele Grandini on September 29, 2017 - 2:33 pm

      did you try it in a SCOM workflow? Which version of the probe are you using? If confirmed this is great news, when I wrote this post and the related code a long ago this was not the case.

  2. #3 by scomstuff on May 4, 2016 - 8:00 am

    Hi Daniel, do you know if this is still required or is there a native way to do it in more recent versions of OM?

  3. #6 by Scott Moss on July 13, 2014 - 4:52 pm

    Reblogged this on System Center Operations Manager 2012 and commented:
    Awesome!

  4. #7 by Tom Riddle on July 7, 2014 - 4:56 pm

    Amazing work. Is there any Managed Module authoring examples and resources that you found useful during the process?

    • #8 by Daniele Grandini on July 8, 2014 - 2:02 pm

      Hi Tom, I will integrate with the few public resources available, still for OpsMgr 2012, a few things have changed with R2.

      • #9 by bobgreen84 on July 23, 2014 - 9:29 am

        Hi, Daniele, where you get information how to write managed modules? On msdn poor articles about managed modules. I want to try my own modules. Do I need to create my own output DataItem everytime or I can use DataItemBase class? Can I override dateCreated value for DataItemBase?

      • #10 by Daniele Grandini on July 25, 2014 - 8:58 am

        Hi Bob,
        I tried myself to use the base DataItem and add a custom property, alas I didn’t find an appropriate constructor nor a method to add custom tags/properties. If you download the code I should have left my tests commented out. If you find a way to do that without defining a derived class please let me know.
        I haven’t tried to override the DateCreated, once again it’s all about the properties and methods of the class (you can check with reflector as I did), by heart I cannot remember a way to directly manipulate the base properties if not (maybe) deriving your own class.

        – Daniele

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: