creating many graph instances in cacti

cacti_db.py is a script which will clone an existing data-template and graph-template for multiple (host,rrd_file,template) instances.

In the scenario it was created for, data is (externally) collected into a (directory) tree of RRD files. While there are only a handful of unique RRD file and graph types, there are a large number of instances of those templates. Good examples of this are:

  • cpu load
  • 1,5,15 minute averages
  • 1 per machine
  • disk util
  • avail, used, free
  • ~5 per machine
  • cache stats
  • (hit, miss, size) stats
  • 2 .. 20 caches/”node”, depending type.

This collected data might exist in the following tree of RRD files:

/servers/hostA/cpu.rrd /servers/hostA/disk/root.rrd /servers/hostA/disk/usr.rrd /servers/hostA/disk/data/logs.rrd /node/nodeA/cache/foo.rrd /node/nodeA/cache/bar.rrd […15 more caches…]

Cacti has the following templates manually configured:

/servers//cpu /servers//disk /node/*/_cache

The script, then, knows how to relate RRD files of the regex pattern r'''/servers/([^/]+)/cpu''' to the template(s) “/servers/*/cpu”, using the regexp group 1 as the name of the host; edit the ‘rrd_types‘ global to suit your scenario.

Instances are named as their RRD file name (yes, skipping Cacti’s own |template_formatting|).

The script will delete instances with the same name, allowing you to re-run it pretty liberally to pick up new rrd files, changes in templates, &c.

The process of cloning is as follows; see ‘execute_plan_alpha(...)‘ in the script for the gory details.

  • find the host in table ‘host‘.
  • data template instantiation
  • find the data template id (by name) in ‘data_template‘.
  • if an instance with the name already exists: delete it.
  • insert into data_local, get new (autoinc) id for the instance.
  • get the list of rrd DSes from the template (from ‘data_template_rrd‘)
  • copy each, maintaining the instance -> template id map.
  • instantiate the data_template itself (‘data_template_data‘)
  • copy the RRAs as well (‘data_template_data_rra‘)
  • graph instantiation
  • get the graph_template_id (by name) from ‘graph_templates
  • insert into ‘graph_local‘, get autoinc id.
  • instantiate graph (‘graph_template‘).
  • copy ‘graph_instance_item‘s, as this is where the graph-ds/rrd-ds mapping is stored, run the relevant ids through the map retained earlier.

For details on Cacti 0.8’s rather atrocious database schema, see Cacti Forums: Database schema… WTF? and Cacti Forums: Cacti data relationship diagram.