RRFW XML Configuration Guide

Common rules

The Round Robin Database Framework configuration consists of several XML files. Once the XML configuration is changed, it must be compiled into the database by executing $RRFW_HOME/bin/compilexml. Also, when Renderer notices the database changes, it flushes its cache.

The top-level XML element is always <configuration>, with sub-elements defining various sections, like datasources or views:

  <configuration>
    <!-- Optional inclusion of other XML files -->
    <include filename="myconfig.xml"/>
    <datasources>
      <!-- Data sources tree definition -->
      ...
    </datasources>
    <views>
      <!-- View definitions -->
      ...
    </views>
    <token-sets>
      <!-- Token sets definitions ->
      ...
    </token-sets>
    <monitors>
      <!-- Monitor definitions ->
      ...
    </monitors>
  </configuration>

Multiple XML files are interpreted in additive matter, i.e. <datasources> section from one file is concatenated with the corresponding sections of previous files.

Additional XML files may be added to the compilation list by means of <include> statement. They will be processed recursively before the content of the XML file they are referenced from. The argument filename determines the name of the file in standard XML files directory. It is safe to include the same file several times, RRFW compiler guarantees that files are only processed once.

Some kinds of sections, like <datasources>, allow to define the same elements two or more times. In this case, the previous parameter values are overridden by the new values.

Each component of the configuration is defined by the set of parameters. They are specified in a common manner, differentiating in parameter names only:

  <view name="default-rrd-html">
    <param name="view-type"     value="html" />
    <param name="expires"       value="300" />
    <param name="html-template" value="default-rrd.html" />
  </view>

The parameter value can be specified either by value XML attribute, or by the text contents of the <param> element.

Some parameter values require other parameters to be defined, like in the above example: a view of type html cannot exist without a template.

After all XML files are compiled, the parameters are checked for validity by the compiler.

Character sets

By default, all XML input is treated as UTF-8 (unicode). This is important because all the HTML output generated by RRFW is encoded UTF-8.

However, you may use Latin1 (ISO-8859-1) encoding in your XML files. In order to ensure correct displaying of non-ASCII characters, the encoding must be specified in your XML files:

  <?xml version="1.0" encoding="ISO-8859-1"?>

You need this only in those files containing non-ASCII characters in Latin1 encoding.

In addition, version 1.54_3 or higher of XML::LibXML is required, and RRFW version 0.0.16 or above.

Macros

In the top level of the configuration tree, a number of macros may be defined. Currently they are used in snmp-object parameter only. Macros are specified with <def> elements, being the direct children of <definitions> element:

  <configuration>
    <definitions>
      <!-- IF-MIB:ifTable  -->
      <def name="ifDescr"           value="1.3.6.1.2.1.2.2.1.2" />
      <def name="ifPhysAddress"     value="1.3.6.1.2.1.2.2.1.6" />
      <def name="ifInOctets"        value="1.3.6.1.2.1.2.2.1.10" />
      <def name="ifInUcastPkts"     value="1.3.6.1.2.1.2.2.1.11" />
      <def name="ifInErrors"        value="1.3.6.1.2.1.2.2.1.14" />
      <def name="ifOutOctets"       value="1.3.6.1.2.1.2.2.1.16" />
      <def name="ifOutUcastPkts"    value="1.3.6.1.2.1.2.2.1.17" />
      <def name="ifOutErrors"       value="1.3.6.1.2.1.2.2.1.20" />
      <!-- Default Interface index lookup -->
      <def name="IFIDX"         value="M($ifDescr, %interface-name%)" />
    </definitions>
    ...

These definitions are global across all XML configuration files, and are referenced with dollar sign and the definition name, e.g.:

    <leaf name="ifInErrors">
      <param name="snmp-object" value="$ifInErrors.$IFIDX" />
      ...

Datasource definitions

Datasources are organized into a tree hierarchy. All parameters are inherited by the subtrees and leafs from their parents, and can be overridden on lower levels.

The datasources tree consists of two key types of components: subtree and leaf. A subtree can have child subtrees or leaves. A leaf can never have children. A subtree represents logical aggregation, while the leaf always represends the actual datasource.

In XML configuration, a child subtree or leaf belongs to the parent element, like the following:

  <datasources>
    <!-- This is the first child of the tree root -->
    <subtree name="Netflow">
      <param name="ds-type" value="rrd-file" />
      <param name="comment"
             value="Netfow data collected by FlowScan with CarrierIn.pm" />
      <!-- All Flowscan-generated files reside here -->
      <param name="data-dir" value="/var/local/flowscan/graphs" />
      <subtree name="Exporters">
        <param name="comment" value="Netflow exporting devices" />
        ...
        <!-- all exporters defined here -->
      </subtree>
       ...
      <subtree name="Total">
        <param name="data-file" value="total.rrd" />
        <leaf name="bps">
          <param name="comment" value="Bits per second" />
          <param name="leaf-type" value="rrd-def" />
          <param name="rrd-ds" value="bits" />
          <param name="rrd-cf" value="AVERAGE" />
        </leaf>
        <leaf name="pps">
          <param name="comment" value="Packets per second" />
          <param name="leaf-type" value="rrd-def" />
          <param name="rrd-ds" value="pkts" />
          <param name="rrd-cf" value="AVERAGE" />
        </leaf>
        <leaf name="packlen">
          <param name="comment" value="Average packet length in bytes" />
          <param name="leaf-type" value="rrd-cdef" />
          <param name="rpn-expr" value="{bps},8,/,{pps},/" />
        </leaf>
      </subtree>
    </subtree>
  </datasources>

Each subtree or a leaf is identified by a path, the symbolic notation similar to filesystem paths. In any path notation, subtree names always end with slash (/), and this trailing slash is the part of the name. In this case, any subtree is identified by a path ending with slash, while leaf paths always end with a word symbol. The top-level subtree is identified by a single slash.

The other components of the datasouce definition are templates, aliases, and filepatterns.

Templates

A template is used when it's needed to define multiple different pieces of the configuration in the same way. For instance, the definition for input/output octets and bits can be specified once in a template, and then applied where appropriate.

The piece of XML configuration inside the <template> element is memorized under the template name, and reproduced in every occurrence of <apply-template> with the corresponding template name. The template definition must be the direct child element of <configuration> XML element:

  <datasources>
    <!-- Default views must be defined -->
    <param name="default-subtree-view" value="default-dir-html" />
    <param name="default-leaf-view" value="default-rrd-html" />
    <!-- Many of our RRDs have the field "bits" - let's define it here -->
    <template name="bps">
      <leaf name="bps">
        <param name="comment" value="Bits per second" />
        <param name="leaf-type" value="rrd-def" />
        <param name="rrd-ds" value="bits" />
        <param name="rrd-cf" value="AVERAGE" />
      </leaf>
    </template>
    ...
      <subtree name="Total">
        <param name="data-file" value="total.rrd" />
        <apply-template name="bps" />
        <leaf name="pps">
          <param name="comment" value="Packets per second" />
          <param name="leaf-type" value="rrd-def" />
          <param name="rrd-ds" value="pkts" />
          <param name="rrd-cf" value="AVERAGE" />
        </leaf>
        <leaf name="packlen">
          <param name="comment" value="Average packet length in bytes" />
          <param name="leaf-type" value="rrd-cdef" />
          <param name="rpn-expr" value="bps,8,/,pps,/" />
        </leaf>
      </subtree>

Aliases

Alias is the alternative symbolic name for a subtree or a leaf. It can be even a name from a different subtree hierarchy. If that alternative hierarchy does not exist, the corresponding subtrees are created:

  <subtree name="62.3.44.55">
    <alias>/Netflow/ExportersByName/rtrTelehouse1/</alias>
    ...

Filepatterns

Some subtree hierarchies can be built based on file name pattern. Suppose you have a bunch of RRD files named conforming to a known rule. In this case, filepatterns allow to build the subtrees based on those names.

Parameter data-dir must be defined in the parent node. It determines the directory where the files are searched.

A filepattern is defined by three mandatory attributes:

At the time of compilation, the files are scanned in the given directory, and for those matched, the new subtrees or leaves are created, with one parameter implicitly defined: data-file is set to the file name being matched.

The string given in filere attribute, is a normal Perl regular expression, except one thing: $PARENT is replaced by the parent subtree name. This allows to build nested filepattern constructs:

    <subtree name="Exporters">
      <param name="comment" value="Netflow exporting devices" />
      <param name="data-dir" value="/var/local/flowscan/graphs" />
      <!-- All netflow exporters look the same from here -->
      <filepattern type="subtree" name="$1" filere="exporter_(.*)_total\.rrd">
        <param name="comment" value="" />
        <!-- And here we describe the difference -->
        <detailed match="62.5.55.233">
          <!-- Let's give it a nicer name -->
          <alias>/Netflow/ExportersByName/rtrTelehouse1/</alias>
          <!-- And give all the paramal details if needed -->
          <param name="location" value="Telehouse Street 1, New Yourk" />
          <param name="contact" value="John Smith" />
          <param name="phone" value="+1-800-444555" />
          <subtree name="if11">
              <param name="comment" value="France Telecom 1" />
              <param name="phone" value="+1-800-666555" />
              <!-- We can give it a name from different subtree -->
              <alias>/Netflow/Peering/FranceTelecom-1/</alias>
          </subtree>
        </detailed>
        <!-- Again, on each router, the interfaces look the same -->
        <filepattern type="subtree" name="'if'.$1"
                     filere="exporter_$PARENT_(\d.*)\.rrd">
            <param name="comment" value="Netflow exporting interface" />
            <!-- For each of the interfaces, the bps leaf is taken from the
                 template above -->
            <apply-template name="bps" />
        </filepattern>
      </filepattern>
    </subtree>

In this example, the files are searched in the directory /var/local/flowscan/graphs, and those matching the patterns exporter_[router-ip]_total.rrd and exporter_[router-ip]_[if-index].rrd build the corresponding hierarchy.

The construct <detailed match="[name]"> allows to specify additional parameters for the subtree or leaf whose name coincides the given string. Should no elements match the given name, a warning message is issued by the compiler.

Compile-time variables

Compile-time variables are those defined somewhere in datasource hierarchy, and valid within a given subtree and its children. It is possible to define pieces of XML configuration which are or are not compiled, depending on the value of corresponding variable.

Variables are set by setvar XML element, with mandatory attributes name and value.

Variable values are used in iftrue and iffalse XML elements. Mandatory parameter var specifies the variable name. The child XML elements are compiled if the variable value is true or false, correspondingly. A true value is true or a nonzero number. Undefined variable is identified as false.

Example:

  <template name="cisco-cbqos-classmap-meters">
  ...
    <iftrue var="CiscoIOS_cbQoS::CMNoBufDrop">
      <leaf name="Dropped_No_Buffer">
      ...
      </leaf>
    </iftrue>
  </template>
  <subtree name="QoS_Stats">
    <setvar name="CiscoIOS_cbQoS::CMNoBufDrop" value="true"/>
    ....
  </subtree>

Parameter value substitution

For any given leaf, some parameters may reference the other parameter values, by embracing the parameter name with percent signs:

   <param name="data-file" value="%snmp-host%_hostaverages.rrd" />

The parameter substitution is performed at runtime. The substitution formula may be defined at a higher subtree level, and the substitution itself will occur at leaf level.

Currently parameter substitution is supported for the following parameters: data-file data-dir rrd-ds rpn-expr rrd-create-max rrd-create-min monitor-vars comment graph-title graph-legend descriptive-nickname collector-timeoffset-hashstring collector-scale lower-limit normal-level upper-limit transform-value

Common parameters

RRD-related parameters

Collector-related parameters

RRD-Multigraph leaves

The leaves with ds-type=rrd-multigraph are dedicated for displaying of several datasource values in one graph. Such leaves cannot be referenced for a numerical value, hence cannot be monitored.

Example:

  <subtree name="SampleMulti">
    <leaf name="sample1">
      <param name="ds-type" value="rrd-multigraph" />
      <param name="ds-names" value="in,out" />
      <param name="foobarpath"
             value="/SNMP/Routers/213.230.38.4/FastEthernet0_0" />
      <!-- parameter name tail is formed by the DS name -->
      <param name="ds-expr-in"       value="{%foobarpath%/locIfInBitsSec}" />
      <param name="graph-legend-in"  value="Bits per second in" />
      <param name="line-style-in"   value="AREA" />
      <param name="line-color-in"   value="#00FF00" />
      <param name="line-order-in"    value="1" />
      <param name="ds-expr-out"      value="{%foobarpath%/locIfOutBitsSec}" />
      <param name="graph-legend-out" value="Bits per second out" />
      <param name="line-style-out"   value="LINE2" />
      <param name="line-color-out"   value="#0000FF" />
      <param name="line-order-out"   value="2" />
    </leaf>
  </subtree>

Parameters:

View definitions

In our context, view means any kind of object representation. The same subtree or view can be displayed in different ways and in different formats: HTML, graph image, plain text, etc.

Renderer module handles these view definitions. For any subtree or leaf, it renders the specified view, and keeps the cache of rendered files.

Each subtree or leaf must have a default view. This is controlled by two parameters that may be defined in the root subtree: default-subtree-view and default-leaf-view.

The set of views is flat, though they can inherit the parameters one from another. Each view is referenced by its name, and is defined by the set of parameters. Same way as with datasources, certain parameter values imply the neccessaty to define certain other parameters:

  <views>
    <view name="default-rrgraph">
      <param name="view-type"     value="rrgraph" />
      <param name="expires"       value="300" />
      <param name="width"         value="500" />
      <param name="height"        value="250" />
      <param name="width-hint"    value="580" />
      <param name="line-style"    value="##SingleGraph" />
      <param name="line-color"    value="##SingleGraph" />
      <!-- Daily graph, inherits parameters from the above -->
      <view name="last24h">
        <param name="start"         value="-24h" />
      </view>
      <!-- Weekly graph -->
      <view name="lastweek">
        <param name="start"         value="-7d" />
      </view>
    </view>
  </views>

Currently the view is defined by the configuration only. Probably, in the future additional parameters will be supplied dynamically.

View parameters

For every view, the mandatory parameters are:

The following values of view-type are recognized:

Styling Profiles

Styling profiles allow symbolic names to be used for line type and color.

Two hashes in the beginning and a name refer to the line style from the styling profile, e.g. ##BpsIn, ##green, ##one, ##two.

      <leaf name="InOutBytes">
        <param name="comment" value="Input and Output bits per second graphs"/>
        <param name="ds-type" value="rrd-multigraph"/>
        <param name="ds-names" value="in,out"/>
        <!-- IN -->
        <param name="ds-expr-in" value="{ifInOctets}"/>
        <param name="graph-legend-in" value="Bytes per second in"/>
        <param name="line-style-in" value="##BpsIn"/>
        <param name="line-color-in" value="##BpsIn"/>
        <param name="line-order-in" value="1"/>
        <!-- OUT -->
        <param name="ds-expr-out" value="{ifOutOctets}"/>
        <param name="graph-legend-out" value="Bytes per second out"/>
        <param name="line-style-out" value="##BpsOut"/>
        <param name="line-color-out" value="##BpsOut"/>
        <param name="line-order-out" value="2"/>
      </leaf>

When processed the example above effectivly becomes:

      <leaf name="InOutBytes">
        <param name="comment" value="Input and Output bits per second graphs"/>
        <param name="ds-type" value="rrd-multigraph"/>
        <param name="ds-names" value="in,out"/>
        <!-- IN -->
        <param name="ds-expr-in" value="{ifInOctets}"/>
        <param name="graph-legend-in" value="Bytes per second in"/>
        <param name="line-style-in" value="AREA"/>
        <param name="line-color-in" value="#00FF00"/>
        <param name="line-order-in" value="1"/>
        <!-- OUT -->
        <param name="ds-expr-out" value="{ifOutOctets}"/>
        <param name="graph-legend-out" value="Bytes per second out"/>
        <param name="line-style-out" value="LINE2"/>
        <param name="line-color-out" value="#0000FF"/>
        <param name="line-order-out" value="2"/>
      </leaf>

Schema definitions can be modified in two ways (see the RRFW Styling Profile Guide manual for available styles and override details)

Token sets definitions

Token is a symbolic identifier for each subtree or a leaf. See Round Robin Framework Architecture manual for more details.

A tokenset is a named list of tokens. Its contents can be rendered, and its members can be added or removed at any time.

Each tokenset can have a number of parameters defined. It also inherits the parameter defined in the top <token-sets> XML element:

  <token-sets>
    <param name="default-tset-view" value="default-tset-html" />
    <param name="default-tsetlist-view" value="tset-list-html" />
    <token-set name="jumps">
      <param name="comment" value="Traffic rate jumps" />
    </token-set>
    <token-set name="hw-failures">
      <param name="comment" value="Holt-Winters prediction failures" />
    </token-set>
  </token-sets>

Parameter default-tsetlist-view is mandatory for tokenset list. It defines the default view when displaying the list of tokensets.

The following parameters are mandatory for tokensets:

Monitor definitions

Monitor is a named set of parameters that defines the behaviour of monitor module. Each leaf can be given a number of monitors via monitor parameter.

Upon monitor module run, an action is launched if the alarm conditions of a given monitor are satisfied.

  <monitors>
    <!-- First define the actions -->
    <!-- This action will put the graphs of alarmed datasources in
         a single alarm report page -->
    <action name="graph-hw-failures">
      <param name="action-type" value="tset" />
      <param name="set-name" value="hw-failures" />
    </action>
    <action name="graph-jumps">
      <param name="action-type" value="tset" />
      <param name="set-name" value="jumps" />
    </action>
    <monitor name="hw-failures">
      <param name="monitor-type" value="failures" />
      <param name="action" value="graph-hw-failures" />
      <param name="expires" value="3600" />
    </monitor>
    <!-- alarm if 5 minutes away it was 10 times lower -->
    <monitor name="high-jumps">
      <param name="monitor-type" value="expression" />
      <param name="rpn-expr" value="{(-300)},10,*,GT,{},{(-300)},10,/,LT,OR" />
      <param name="action" value="graph-jumps" />
      <param name="expires" value="3600" />
      <param name="comment"
             value="Value jumped more than 10-fold in 5 minutes" />
    </monitor>
    <monitor name="hundred-megs">
      <param name="monitor-type" value="expression" />
      <param name="rpn-expr" value="100,1024,1024,*,*,GT" />
      <param name="action" value="graph-jumps" />
      <param name="expires" value="3600" />
      <param name="comment"
             value="Traffic is higher than 10 mbps" />
    </monitor>
  </monitors>

Event types

Should the alarm condition occur, a series of events is happening in sequentional order:

Monitor parameters

Action parameters

Author

Copyright (c) 2002-2003 Stanislav Sinyagin <ssinyagin@yahoo.com>