Advanced DataItem XML extraction in #scom #sysctr


The Management Pack authors know workflow modules pass data one to the other via an XML document. There are many filtering options, most of them are based on condition detections modules where the proper xpath query can be used to select or check for the correct value. The XML query engine in Operations Manager uses the SelectSingleNode method, this can lead to situations where it’s hard to properly process the input XML document.

Before digging into the topic of this post I need to set a couple of aspects related to:

          Batching

          XML processing and substitution

Batching

The XML document passed between modules is composed by one or more DataItems, when more than 1 DataItem is returned the *Batching* property of the modules comes into play. Basically you cannot have a workflow that intermixes Batching=”true” and Batching=”false” modules unless the Batching=”false” module is the last one of the composition. The point is Batching=”false” modules process the DataItem in input and immediately output the results, as soon as the result is posted to the following module the other DataItems are lost.  To summarize:

          n DataItems -> Batching=”false” is OK

          n DataItems -> Batching=”false” -> Batching=”true” doesn’t work, or just the first DataItem is processed

          n DataItems -> Batching=”true” -> Batching=”false” is OK

XML Processing and substitution

We all know the $data variable referrers to the XML document, there are many situations where being able to process the entire XML document in a script would be great, so we could think about using $data to pass the input XML document to the following module as a parameter, such as:

              <ProbeAction ID=PS TypeID=Windows!Microsoft.Windows.PowerShellPropertyBagProbe>

                <ScriptName>ProcessXML.ps1</ScriptName>

                <ScriptBody>

                  <![CDATA[

param([xml] $MyXML)

               

                ]]></ScriptBody>

                <Parameters>

                  <Parameter>

                    <Name>MyXML</Name>

                    <Value>$Data$</Value>

                  </Parameter>

                </Parameters>

                <TimeoutSeconds>300</TimeoutSeconds>

              </ProbeAction>

It would be great, but it does not return the expected result. Every module has the responsibility to process input parameters, alas may modules I know of are doing some sort of escaping on the input parameters so they get flattened out without XML markups keeping just the values of the tags.

Advanced XML extraction

Let’s return to our main topic, and to make thing clear let’s set a sample DataItem document:

XML

<?xml version=1.0?>

<DataItem type=WorkflowsReport time=2012-03-31T17:03:19.0113784+02:00 sourceHealthServiceId=00000000-0000-0000-0000-000000000000>

  <Status>Failed</Status>

  <Count>7</Count>

  <Details>

    <Instance Id={C276AC4F-F29C-73C6-DDB9-86160E79AF3E}>

      <Workflow>System.NetworkManagement.ifMIB.NetworkAdapter.HighInputBroadcastRate</Workflow>

      <Workflow>System.NetworkManagement.ifMIB.NetworkAdapter.InputPacketErrorPercent</Workflow>

      <Workflow>System.NetworkManagement.ifMIB_Cisco.NetworkAdapter.HighOutputQueueDropRate</Workflow>

      <Workflow>System.NetworkManagement.ifMIB.NetworkAdapter.InputPacketBroadcastPercent</Workflow>

      <Workflow>System.NetworkManagement.ifMIB_CiscoRouterEthernet.Interface.HighCollisionRate</Workflow>

      <Workflow>System.NetworkManagement.ifMIB_Cisco.NetworkAdapter.HighInputQueueDropRate</Workflow>

      <Workflow>System.NetworkManagement.ifMIB.NetworkAdapter.HighInputErrorRate</Workflow>

    </Instance>

  </Details>

</DataItem>

Now let’s say my workflow needs to process all the failed <Workflow> tags:

    <ValueExpression>

        <XPathQuery Type=’String’>/DataItem/Details//Workflow</XPathQuery>

    </ValueExpression>

 

The above filter will just return the first workflow tag or System.NetworkManagement.ifMIB.NetworkAdapter.HighInputBroadcastRate

First lesson learned: I didn’t find a way to process the DataItem document the way I wanted to, for example I cannot extract all the Workflow tags.

While I cannot access all the Workflow tags, since under the hood the SelectSingleNode method is used I can leverage the full XPath syntax and I can search for a specific Workflow tag using the “.” (text) token. So if I want to check if “System.NetworkManagement.ifMIB.NetworkAdapter.InputPacketErrorPercent” exists I can write something like:

<SimpleExpression>

  <ValueExpression>

    <XPathQuery Type=String>/DataItem/Details/Instance/Workflow[. = “System.NetworkManagement.ifMIB.NetworkAdapter.InputPacketErrorPercent”]</XPathQuery>

  </ValueExpression>

  <Operator>Equal</Operator>

  <ValueExpression>

    <Value Type=String>System.NetworkManagement.ifMIB.NetworkAdapter.InputPacketErrorPercent</Value>

  </ValueExpression>

</SimpleExpression

Just to give an example of a rule leveraging this option:

XML

 

      <Rule ID=QND.Failed.WFs.FilteredEventCollectionRule Target=SC!Microsoft.SystemCenter.HealthService Enabled=true ConfirmDelivery=false Remotable=true Priority=Normal DiscardLevel=100>
        <Category>EventCollection</Category>
        <DataSources>
          <DataSource ID=DS TypeID=QND.Failed.WFs.Filtered.DS>
            <IntervalSeconds>300</IntervalSeconds>
            <SyncTime />
            <ManagedEntityId />
            <Expression>
              <SimpleExpression>
                <ValueExpression>
                  <XPathQuery Type=String>/DataItem/Details/Instance/Workflow[. = “System.NetworkManagement.ifMIB.NetworkAdapter.InputPacketErrorPercent”]</XPathQuery>
                </ValueExpression>
                <Operator>Equal</Operator>
                <ValueExpression>
                  <Value Type=String>System.NetworkManagement.ifMIB.NetworkAdapter.InputPacketErrorPercent</Value>
                </ValueExpression>
              </SimpleExpression>
            </Expression>
          </DataSource>
        </DataSources>

 

Second lesson learned, since SelectSingleNode is used everything that is available to SelectSingleNode can be used, but while SelectSingleNode returns a node, in Operations Manager what’s returned is the Text property of the node and nothing else.

For more info see the XPath reference on MSDN.

  • Daniele This posting is provided “AS IS” with no warranties, and confers no rights.
Advertisements
  1. #1 by Michiel Wouters on December 5, 2013 - 10:17 am

    Great post. Some time ago I tried to use XPath methods to suppress alerts coming from a workflow checking a log file. Didn’t work. (Ever thried this?). Nice to see that XPath structures does indeed come in handy sometimes.

    • #2 by Daniele Grandini on December 5, 2013 - 5:50 pm

      Hi Michiel, never tried to use XPath to suppress alerts coming from log files. I will check if I can find something.

  2. #3 by Tom on December 3, 2013 - 4:58 pm

    This is such a wonderful post detailing this tag. I would be interested to see if you could build a conditiondetection consolidator module with an input for each workflow line to accomplish the same thing as batching=true (http://blogs.technet.com/b/authormps/archive/2011/02/24/workflow-types.aspx)

    • #4 by Daniele Grandini on December 5, 2013 - 5:53 pm

      Hi Tom,
      in general condition detection modules are batching aware (Batching=true), so you really don’t need to add anything. Or I miss something? I have an old project of mine in the standby list on how to build a condition detection module in C#.

  3. #5 by Ernie on November 30, 2013 - 7:22 pm

    Thank you for posting Daniele, very useful post :)
    Ernie

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: