vaping.plugins.logparse

Classes

AggregateSchema

AggregateSchema(confu.schema.core.Schema)

Describes a confu schema.

Instantiate confu attributes as properties of the schema.

As the schema itself is a confu attribute, you may nest schemas within schemas

Example

class MySchema(Schema):
    str_attr = Str()

Class Attributes

  • count (Int Instance): Aggregate n lines

FieldSchema

FieldSchema(confu.schema.core.Schema)

Describes a confu schema.

Instantiate confu attributes as properties of the schema.

As the schema itself is a confu attribute, you may nest schemas within schemas

Example

class MySchema(Schema):
    str_attr = Str()

Class Attributes

  • aggregate (Str Instance): How to aggregate the field if aggregation is turned on (sum, avg, eval)
  • eval (Str Instance): Evaluate to create the value, other fields' values will be available in the string formatting
  • parser (Str Instance): Regex pattern to parse field value, needs to one group in it
  • type (Str Instance): Value type (int, float etc.)

LogParse

LogParse(vaping.plugins.FileProbe)

Log parse plugin base

Will parse a log line by line and probe to emit data over a specified interval.

Config

  • path (str): log file path

  • fields (dict): field definition

    field name as key

    parser regex pattern to parse field value, needs to one group in it

    type value type (int, float etc.)

    aggregate how to aggregate the field if aggregation is turned on (sum, avg, eval)

    eval evaluate to create the value, other fields values will be available in the string formatting

  • time_parser (dict) if specified will be passed to strptime to generate a timestamp from the logline

time_parser:
    find: \d\d:\d\d:\d\d
    format: %H:%M:%S
  • exclude (list): list of regex patterns that will cause lines to be excluded on match
  • include (list): list of regex patterns that will cause lines to be included on match
  • aggregate (dict): aggregation config -count aggregate n lines

Instance Attributes

  • stack (list)
  • fields (dict): field config
  • aggregate_count (int)
  • exclude (list)
  • include (list)
  • time_parser (dict)

Class Attributes

  • ConfigSchema (LogParseSchema Class): Base plugin config schema

Methods

aggregate

def aggregate(self, messages)

Takes a list of messages and aggregates them according to aggration config

Arguments

  • messagges (list<dict>)

Returns

list of aggregated messages (list<dict>)

aggregate_avg

def aggregate_avg(self, field_name, rows)

Aggregate average value

Arguments

  • field_name (str): field to aggregate
  • rows (list): list of vaping message data rows

Returns

avg (float)

aggregate_eval

def aggregate_eval(self, field_name, rows)

Aggregate using an eval() result

Needs to have eval set in the field config. Value will be passed straight to the eval() function so caution is advised.

Arguments

  • field_name (str): field to aggregate
  • rows (list): list of vaping message data rows

Returns

eval result

aggregate_field

def aggregate_field(self, field_name, rows)

takes a field name and a set of rows and will return an aggregated value

this requires for the field to have it's aggregate config specified in the probe config

Arguments

  • field_name (str)
  • rows (list)

Returns

aggregated value

aggregate_message

def aggregate_message(self, message)

Takesa vaping message with multiple items in it's data property and aggregates that data

Arguments

  • message (dict): vaping message dict

aggregate_sum

def aggregate_sum(self, field_name, rows)

Aggregate sum

Arguments

  • field_name (str): field to aggregate
  • rows (list): list of vaping message data rows

Returns

sum

init

def init(self)

called after the plugin is initialized, plugin may define this for any other initialization code

parse_field_value

def parse_field_value(self, field, line)

takes a field definition and a log line and attempts to parse out the field's value

parse_line

def parse_line(self, line)

Here is where we parse values out of a single line read from the log and return a dict containg keys and values

Arguments

  • line (str)

process_line

def process_line(self, line, data)

The data dict represents the current emit object, depending on your interval multiple lines may be included in the same emit object.

Should return the data object

Arguments

  • line (str): log line
  • data (dict): current emit dict

process_messages

def process_messages(self, messages)

Process vaping messages before the are emitted

Aggregation is handled here

Arguments

  • messages (list): list of vaping messages

Returns

Result of self.aggregate

validate_interval

def validate_interval(self, value)

validates a string describing elapsed time or time duration

Arguments

  • value (str): elapsed time (example: 1d2h)

Returns

seconds (float)

LogParseSchema

LogParseSchema(vaping.plugins.PluginConfigSchema)

Define a schema for FPing and also define defaults.

Class Attributes

  • aggregate (AggregateSchema Instance): aggregation config
  • exclude (List Instance): list of regex patterns that will cause lines to be excluded on match
  • fields (Dict Instance): Field definition
  • include (List Instance): list of regex patterns that will cause lines to be included on match
  • time_parser (TimeParserSchema Instance): If specified will be passed to strptime to generate a timestamp from the logline

TimeParserSchema

TimeParserSchema(confu.schema.core.Schema)

Describes a confu schema.

Instantiate confu attributes as properties of the schema.

As the schema itself is a confu attribute, you may nest schemas within schemas

Example

class MySchema(Schema):
    str_attr = Str()

Class Attributes

  • find (Str Instance): Regex string to find timestamps.
  • format (Str Instance): Datetime format to output timestamps.