Two distributed vaping instances on the same graph

This example shows how two run 2 separate vaping instances connected to a vodka instance and plot on the same graph.

Running a distributed setup

For a complete understanding of running a distributed vaping setup, please read the Distributed Latency example in the quickstart section.

Requires vodka 2.2.6

This requires that you install vodka 2.2.6 or higher:

pip install vodka>=2.2.6

Vaping instance 1 - this one will ping 1.1.1.1

examples/distributed_shared_graph/vaping-1/config.yml:

probes:
  - name: latency
    type: std_fping
    output:
      - zmq_vodka

    groups:
      - name: public_dns
        hosts:
          - host: 1.1.1.1

plugins:
  - name: std_fping
    type: fping
    count: 10
    interval: 3s

  - name: zmq_vodka
    type: zeromq
    bind: tcp://127.0.0.1:6021


logging:
  version: 1
  formatters:
    simple:
      format: '%(asctime)s - %(name)s - %(levelname)s: %(message)s'
  handlers:
    console:
      class: logging.StreamHandler
      level: DEBUG
      formatter: simple
      stream: ext://sys.stdout

    #file:
    #  class: logging.FileHandler
    #  level: DEBUG
    #  formatter: simple
    #  filename: /home/dev/sandbox/vaping/vaping.log

  loggers:
    vaping:
      level: DEBUG
      handlers:
        - console
        #- file
vaping start --home=examples/distributed_shared_graph/vaping-1 --debug

Vaping instance 2 - this one will ping 8.8.8.8

examples/distributed_shared_graph/vaping-2/config.yml:

probes:
  - name: latency
    type: std_fping
    output:
      - zmq_vodka

    groups:
      - name: public_dns
        hosts:
          - host: 8.8.8.8

plugins:
  - name: std_fping
    type: fping
    count: 10
    interval: 3s

  - name: zmq_vodka
    type: zeromq
    bind: tcp://127.0.0.1:6022


logging:
  version: 1
  formatters:
    simple:
      format: '%(asctime)s - %(name)s - %(levelname)s: %(message)s'
  handlers:
    console:
      class: logging.StreamHandler
      level: DEBUG
      formatter: simple
      stream: ext://sys.stdout

    #file:
    #  class: logging.FileHandler
    #  level: DEBUG
    #  formatter: simple
    #  filename: /home/dev/sandbox/vaping/vaping.log

  loggers:
    vaping:
      level: DEBUG
      handlers:
        - console
        #- file
vaping start --home=examples/distributed_shared_graph/vaping-2 --debug

Vodka

examples/distributed_shared_graph/vodka/config.yml:

data:
  - type: fping
    handlers:
      - type: index
        index: host
      - type: store
        container: list
        limit: 500

apps:
  graphsrv:
    enabled: true

    # here is where we define host config
    groups:
      latency:
        public_dns:
          8.8.8.8:
            name: Google
            color: red
          1.1.1.1:
            name: Cloudflare
            color: blue

    graphs:
      multitarget:
        id_field: host
        type: multitarget
        plot_y: avg
        format_y: ms

      smokestack:
        id_field: host
        type: smokestack
        plot_y: avg

plugins:

  # zero mq probe plugin to vaping-1 instance
  # (latency name is important, so it can be routed properly to
  # the similarly named group)
  - name: latency
    type: zeromq_probe
    data: fping
    interval: 1.0
    bind: tcp://127.0.0.1:6021
    async: thread

  # zero mq probe plugin to vaping-2 instance
  # We cannot have duplicate names so we name it `latency2`
  # However we still want it to get collected to `latency`
  # We will do that using the `data_id` config
  - name: latency_2
    type: zeromq_probe
    data_id: latency
    data: fping
    interval: 1.0
    bind: tcp://127.0.0.1:6022
    async: thread


  - name: http
    type: flask
    bind: 0.0.0.0:7021
    debug: true
    # set this to gunicorn or uwsgi depending on what you want to run with
    server: gunicorn
    async: gevent
    routes:
      /targets : graphsrv->targets
      /graph_data :
        methods:
          - POST
          - GET
        target: graphsrv->graph_data
      /graph : graphsrv->graph_view
      /overview_read_file : graphsrv->overview_read_file
      /: graphsrv->overview_view


logging:
  version: 1
  formatters:
    simple:
      format: '%(asctime)s - %(name)s - %(levelname)s: %(message)s'
  handlers:
    console:
      class: logging.StreamHandler
      level: DEBUG
      formatter: simple
      stream: ext://sys.stdout
  loggers:
    vodka:
      level: DEBUG
      handlers:
        - console
export VODKA_HOME=examples/distributed_shared_graph/vodka
gunicorn -b 0.0.0.0:7021 vodka.runners.wsgi:application

MTR

This example will show you how to setup an MTR graph:

Requires graphsrv 1.3.0

You need to run graphsrv 1.3.0 or later in order to be able to render MTR graphs.

pip install graphsrv>=1.3.0

Requires traceroute

We use the traceroute command to determine the hops to send to fping. Make sure it is installed.

MTR Graph is currently experimental

The MTR graph is introduced to vaping in version 0.6.0 and should be considered an early iteration of MTR data visualization to vaping.

We have ideas on how to make it better, but would also love to hear your thoughts on it.

Pay close attention to the commented lines in the example below, as you need to do the following:

  • setup the mtr probe
  • setup the fping_mtr plugin
  • setup the fping_mtr data type
  • setup the mtr graph

examples/mtr/config.yml:


probes:

  # mtr probe
  - name: mtr
    type: fping_mtr

    # we need to let graphsrv know what the default
    # graph to render data from this probe should be,
    # in this case we want it to be `mtr`
    default_graph: mtr

    # we need to let graphsrv know what data group
    # to use for data from this probe. Since the probe
    # itself is not setting up any groups unlike the `fping`
    # probe.
    group: mtr.cloudflare_dns

    # target host for the mtr probe
    host: 1.1.1.1

    output:
      - vodka

plugins:

  # mtr plugin
  - name: fping_mtr
    type: fping_mtr
    interval: 3s


  - name: vodka
    type: vodka

    data:

      # We have to add a new data type to handle mtr data
      - type: fping_mtr
        handlers:
          - type: index
            index: host
          - type: store
            container: list
            limit: 100


    apps:
      graphsrv:
        enabled: true
        graphs:
          multitarget:
            id_field: host
            type: multitarget
            plot_y: avg
            format_y: ms

          smokestack:
            id_field: host
            type: smokestack
            plot_y: avg

          # let graphsrv know to initialize the `mtr`
          # graph for use
          mtr:
            id_field: host
            type: mtr

    plugins:

      - name: http
        type: flask
        bind: 0.0.0.0:7021
        debug: true
        static_url_path: /static
        server: self
        async: thread
        routes:
          /targets : graphsrv->targets
          /graph_data :
            methods:
              - POST
              - GET
            target: graphsrv->graph_data
          /graph : graphsrv->graph_view
          /overview_read_file : graphsrv->overview_read_file
          /: graphsrv->overview_view



logging:
  version: 1
  formatters:
    simple:
      format: '%(asctime)s - %(name)s - %(levelname)s: %(message)s'
  handlers:
    console:
      class: logging.StreamHandler
      level: DEBUG
      formatter: simple
      stream: ext://sys.stdout

    #file:
    #  class: logging.FileHandler
    #  level: DEBUG
    #  formatter: simple
    #  filename: /home/dev/sandbox/vaping/vaping.log

  loggers:
    vaping:
      level: DEBUG
      handlers:
        - console
        #- file
    vodka:
      level: DEBUG
      handlers:
        - console
        #- file
vaping start --home=examples/mtr --debug

MTR - Distributed

When setting up a distributed MTR probe, the group on the vodka end should be setup like this:

apps:
  graphsrv:
    enabled: true

    groups:
      mtr:
        config:
          default_graph: mtr
        cloudflare_dns:
          1.1.1.1:
            name: Cloudflare
            color: mediumpurple