vaping.plugins.logparse

Classes


LogParse

LogParse(vaping.plugins.FileProbe)

Log parse plugin base

Will parse a log line by line and probe to emit data over a specified interval.

Config

  • path (str): log file path
  • fields (dict): field definition

    field name as key

    parser regex pattern to parse field value, needs to one group in it

    type value type (int, float etc.)

    aggregate how to aggregate the field if aggregation is turned on (sum, avg, eval)

    eval evaluate to create the value, other fields values will be available in the string formatting

  • time_parser (dict) if specified will be passed to strptime to generate a timestamp from the logline

time_parser:
    find: \d\d:\d\d:\d\d
    format: %H:%M:%S
  • exclude (list): list of regex patterns that will cause lines to be excluded on match
  • include (list): list of regex patterns that will cause
  • aggregate (dict): aggregation config -count aggregate n lines

Instance Attributes

  • stack (list)
  • fields (dict): field config
  • aggregate_count (int)
  • exclude (list)
  • include (list)
  • time_parser (dict)

Methods

aggregate

def aggregate(self, messages)

Takes a list of messages and aggregates them according to aggration config

Arguments

  • messagges (list<dict>)

Returns

list of aggregated messages (list<dict>)


aggregate_avg

def aggregate_avg(self, field_name, rows)

Aggregate average value

Arguments

  • field_name (str): field to aggregate
  • rows (list): list of vaping message data rows

Returns

avg (float)


aggregate_eval

def aggregate_eval(self, field_name, rows)

Aggregate using an eval() result

Needs to have eval set in the field config. Value will be passed straight to the eval() function so caution is advised.

Arguments

  • field_name (str): field to aggregate
  • rows (list): list of vaping message data rows

Returns

eval result


aggregate_field

def aggregate_field(self, field_name, rows)

takes a field name and a set of rows and will return an aggregated value

this requires for the field to have it's aggregate config specified in the probe config

Arguments

  • field_name (str)
  • rows (list)

Returns

aggregated value


aggregate_message

def aggregate_message(self, message)

Takesa vaping message with multiple items in it's data property and aggregates that data

Arguments

  • message (dict): vaping message dict

aggregate_sum

def aggregate_sum(self, field_name, rows)

Aggregate sum

Arguments

  • field_name (str): field to aggregate
  • rows (list): list of vaping message data rows

Returns

sum


init

def init(self)

called after the plugin is initialized, plugin may define this for any other initialization code


parse_field_value

def parse_field_value(self, field, line)

takes a field definition and a log line and attempts to parse out the field's value


parse_line

def parse_line(self, line)

Here is where we parse values out of a single line read from the log and return a dict containg keys and values

Arguments

  • line (str)

process_line

def process_line(self, line, data)

The data dict represents the current emit object, depending on your interval multiple lines may be included in the same emit object.

Should return the data object

Arguments

  • line (str): log line
  • data (dict): current emit dict

process_messages

def process_messages(self, messages)

Process vaping messages before the are emitted

Aggregation is handled here

Arguments

  • messages (list): list of vaping messages

Returns

Result of self.aggregate


validate_interval

def validate_interval(self, value)

validates a string describing elapsed time or time duration

Arguments

  • value (str): elapsed time (example: 1d2h)

Returns

seconds (float)