package xapi-rrd

  1. Overview
  2. Docs

This module provides a util that records data in a way that's compatible with rrdtool.

module Fring = Rrd_fring
module Utils = Rrd_utils
exception No_RRA_Available
exception Invalid_data_source of string
type ds_owner =
  1. | VM of string
  2. | Host
  3. | SR of string
type ds_type =
  1. | Absolute
  2. | Gauge
  3. | Derive

Data source types - see ds datatype

val rpc_of_ds_type : ds_type -> Rpc.t
val ds_type_of_rpc : Rpc.t -> ds_type
type cf_type =
  1. | CF_Average
  2. | CF_Min
  3. | CF_Max
  4. | CF_Last

Consolidation function - see RRA datatype

type ds_value_type =
  1. | VT_Float of float
  2. | VT_Int64 of int64
  3. | VT_Unknown

Container so that we can handle different typed inputs

val rpc_of_ds_value_type : ds_value_type -> Rpc.t
val ds_value_type_of_rpc : Rpc.t -> ds_value_type
type sampling_frequency =
  1. | Five_Seconds
val rpc_of_sampling_frequency : sampling_frequency -> Rpc.t
val sampling_frequency_of_rpc : Rpc.t -> sampling_frequency
val (+++) : int64 -> int64 -> int64
val (---) : int64 -> int64 -> int64
val (***) : int64 -> int64 -> int64
val (///) : int64 -> int64 -> int64
val ds_type_to_string : ds_type -> string
val cf_type_of_string : string -> cf_type
val cf_type_to_string : cf_type -> string
val ds_value_to_string : ds_value_type -> string

The CDP preparation scratch area. The 'value' field should be accumulated in such a way that it always contains the value that will eventually be the CDP. This means that for averages, we accumulate 1/n * the PDP, and renormalise when we have unknown PDPs. For the other types it's much easier

type cdp_prep = {
  1. mutable cdp_value : float;
  2. mutable cdp_unknown_pdps : int;
    (*

    How may PDPs have been unknown so far

    *)
}

DS - a data source This defines how we deal with incoming data. Type is one of:

  • Absolute: meaning that the incoming data is an absolute rate
  • Derive: meaning that the rate must come from the difference between the incoming data and the previous value
  • Gauge: meaning that the value isn't a rate at all (e.g. temperature, load avg)

Optionally, there is a maximum time greater than which we mark the PDPs as unknown.

type ds = {
  1. ds_name : string;
    (*

    Name

    *)
  2. ds_ty : ds_type;
    (*

    Type of rate the input must be processed as, see above

    *)
  3. ds_min : float;
  4. ds_max : float;
  5. ds_mrhb : float;
    (*

    Maximum time between updates

    *)
  6. mutable ds_last : ds_value_type;
    (*

    Last raw value that was processed

    *)
  7. mutable ds_value : float;
    (*

    Current calculated rate of the PDP

    *)
  8. mutable ds_unknown_sec : float;
    (*

    Number of seconds that are unknown in the current PDP

    *)
}
val rpc_of_ds : ds -> Rpc.t
val ds_of_rpc : Rpc.t -> ds

RRA - RRD archive This is an archive that holds consolidated data points (CDPs) belonging to a single consolidation function. They are stored in rings buffers, each one related to a single different data-source. It defines the type of consolidation that happens (average, max, min or last), the number of primary data points (PDPs) that go to make a CDP, and the number of CDPs to store.

To better visualize how the datapoints are stored:

│ Datasources ┃ ┃ ┃ └─────────────────┨ Memory ┃ cputime ┃ Consolidators ┃ ┃ ┃ ━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━┫ Average ┃ Fring of CDPs ┃ Fring of CDPs ┃ ← RRA ━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━┫ Max ┃ Fring of CDPs ┃ Fring of CDPs ┃ ← RRA ━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━┛

type rra = {
  1. rra_cf : cf_type;
    (*

    consolidation function

    *)
  2. rra_row_cnt : int;
    (*

    number of entries to store

    *)
  3. rra_pdp_cnt : int;
    (*

    number of pdps per cdp

    *)
  4. rra_xff : float;
    (*

    proportion of missing pdps at which we mark the cdp as unknown

    *)
  5. rra_data : Fring.t array;
    (*

    Stored data, one ring per datasource

    *)
  6. rra_cdps : cdp_prep array;
    (*

    scratch area for consolidated datapoint preparation

    *)
  7. mutable rra_updatehook : (rrd -> int -> unit) option;
    (*

    Hook that gets called when an update happens

    *)
}

The container for the DSs and RRAs. Also specifies the period between pdps

and rrd = {
  1. mutable last_updated : float;
    (*

    Last updated time in seconds

    *)
  2. timestep : int64;
    (*

    Period between PDPs

    *)
  3. rrd_dss : ds array;
  4. rrd_rras : rra array;
}
val copy_cdp_prep : cdp_prep -> cdp_prep
val copy_rra : rra -> rra
val copy_ds : ds -> ds
val copy_rrd : rrd -> rrd
val cf_init_value : cf_type -> ds -> float
val get_times : float -> int64 -> int64 * float

Helper function to get the start time and age of the current/last PDP

val do_cfs : rra -> int -> float array -> unit

Update the CDP value with a number (start_pdp_offset) of PDPs.

val rra_update : rrd -> int64 -> int -> float array -> unit

Update the RRAs with a number of PDPs.

val process_ds_value : ds -> ds_value_type -> float -> bool -> float
val ds_update : rrd -> float -> ds_value_type array -> (float -> float) array -> bool -> unit
val ds_update_named : rrd -> float -> new_domid:bool -> (string * (ds_value_type * (float -> float))) list -> unit

Update the rrd with named values rather than just an ordered array

val ds_names : rrd -> string list

Get registered DS names

val rra_create : cf_type -> int -> int -> float -> rra

create an rra structure

val ds_create : string -> ds_type -> ?min:float -> ?max:float -> ?mrhb:float -> ds_value_type -> ds
val rrd_create : ds array -> rra array -> int64 -> float -> rrd

Add in a new DS into a pre-existing RRD. Preserves data of all the other archives and fills the new one full of NaNs. Note that this doesn't fill in the CDP values correctly at the moment!

  • parameter now

    = Unix.gettimeofday ()

val rrd_add_ds : rrd -> float -> ds -> rrd
val rrd_remove_ds : rrd -> string -> rrd

Remove the named DS from an RRD. Removes all of the data associated with it, too

val find_best_rras : rrd -> int -> cf_type option -> int64 -> rra list

Find the RRA with a particular CF that contains a particular start time, and also has a minimum pdp_cnt. If it can't find an appropriate one, either return the RRA with the correct CF that has the most ancient data, or raise No_RRA_Available if there's not archive with the correct CF. Assumes the RRAs are stored in increasing time-length

val query_named_ds : rrd -> float -> string -> cf_type -> float
val from_xml : Xmlm.input -> rrd
val xml_to_output : rrd -> Xmlm.output -> unit
module Json : sig ... end
val json_to_string : rrd -> string
module Statefile_latency : sig ... end