Validator Reference

osivalidator.linked_proto_field module

This class describes a wrapper on protobuf fields that add a link with parent message and bind message to some additional information.

class LinkedProtoField(value, name=None, parent=None)

Bases: object

This class describes a wrapper on protobuf fields that add a link with parent message and bind message to some additional information.

The Protobuf’s RepeatedCompositeContainer that describes repeated field are replaced with Python lists. The field information (parent message and field name) for the repeated field are here bounded to each element of the list.

property all_field_descriptors

List all the fields descriptors, i.e even the one that are not set

property fields

Overloading of protobuf ListFields function that return a list of LinkedProtoFields.

Only works if the field is composite, raise an AttributeError otherwise.

get_field(field_name)

If the LinkedProtoField wraps a message, return the field of this message. Otherwise, raise an AttributeError.

has_field(field_name)

Check if a protobuf message has an attribute/field even if this is a repeated field.

If it is a repeated field, this function returns false if there is no element into it.

property is_message

Return true if the field contain a message

property message_type

Return a path to the message type in OSI3 as a ProtoMessagePath

query(path, parent=False)

Return a LinkedProtoField from a path.

Example of path: ./global_ground_truth/moving_object

to_dict()

Return the dict version of the protobuf message.

Compute the dict only once, then store it and retrieve it.

osivalidator.osi_doxygen_xml module

This module carry the XML version of Doxygen documentation to parse the rules into it.

class OSIDoxygenXML

Bases: object

This class creates XML from *.proto Files Documentation of OSI and can parse the rules from it.

generate_osi_doxygen_xml()

Generate the Doxygen XML documentation in the OSI path

get_files()

Return the path of the fields in OSI

parse_rules()

Parse the Doxygen XML documentation to get the rules

osivalidator.osi_general_validator module

Main class and entry point of the OSI Validator.

close_pool(pool)

Cleanly close a pool to free the memory

command_line_arguments()

Define and handle command line interface

main()

Main method

process_timestep(timestep)

Process one timestep

osivalidator.osi_id_manager module

This module contains the OSIIDManager which manage IDs and check for references and unicity according to the OSI KPIs.

class OSIIDManager(logger)

Bases: object

Manage the ID of OSI Messages for verification of unicity and references

get_all_messages_by_id(message_id, message_t=None)

Retrieve all the message by giving an id

get_message_by_id(message_id, message_t=None)

Retrieve only one message by giving an id

refer(referer, identifier, expected_type, condition=None)

Add a reference Condition is a function that will be applied on the found object if the reference is resolved

register_message(message_id, message)

Register one message in the ID manager

reset()

Erase all data in the ID manager

resolve_references(timestamp)

Check if references are compliant

resolve_unicity(timestamp)

Check for double ID

message_t_filter(message, message_t)

Check if a message is of type message_t

osivalidator.osi_rules module

This module contains all the useful classes to describe the tree of the validation rules tree.

class FieldRules(name, rules=None, path=None, root=None)

Bases: osivalidator.osi_rules.OSIRuleNode

This class manages the rules of a Field in a Message Type

add_rule(rule)

Add a new rule to a field with the parameters. For example:

self.add_rule(Rule(verb="is_less_than_or_equal_to", params=2))

The rule can also be a dictionary containing one key (the rule verb) with one value (the parameter). For example:

self.add_rule({"is_less_than_or_equal_to": 2})
get_rule(verb)

Return the rule object for the verb rule_verb in this field.

has_rule(rule)

Check if a field has the rule rule

list_rules()

List the rules of a field

class MessageTypeRules(name, fields=None, root=None)

Bases: osivalidator.osi_rules.TypeRulesContainer

This class manages the fields of a Message Type

add_field(field)

Add a field with or without rules to a Message Type

get_field(field_name)
class OSIRuleNode(path=None)

Bases: object

Represents any node in the tree of OSI rules

property path

Return the path of the node

class OSIRules

Bases: object

This class collects validation rules

from_dict(rules_dict=None, rules_container=None)

Translate dict rules into objects rules

from_xml_doxygen()

Parse the Doxygen XML documentation to get the rules

from_yaml(yaml_content)

Import from a string

from_yaml_directory(path=None)

Collect validation rules found in the directory.

from_yaml_file(path)

Import from a file

get_rules()

Return the rules

class ProtoMessagePath(path=None)

Bases: object

Represents a path to a message object

child_path(child)

Return a new path for the child

pretty_html()

Return a pretty html version of the message path

class Rule(**kwargs)

Bases: osivalidator.osi_rules.OSIRuleNode

This class manages one rule

from_dict(rule_dict: dict)

Instantiate Rule object from a dictionary

property path

Return the path of the node

property targeted_field
class Severity

Bases: enum.Enum

Description of the severity of the raised error if a rule does not comply.

ERROR = 40
INFO = 20
WARN = 30
class TypeRulesContainer(nested_types=None, root=None)

Bases: osivalidator.osi_rules.OSIRuleNode

This class defines either a MessageType or a list of MessageTypes

add_type(message_type)

Add a message type in the TypeContainer

add_type_from_path(path, fields=None)

Add a message type in the TypeContainer by giving a path

The path must be a list and the last element of the list is the name of the message type.

If the message type already exists, it is not added.

get_type(message_path)

Get a MessageType by name or path

osivalidator.osi_rules_checker module

This module contains all the rules which a message or an attribute of a message from an OSI scenario can comply.

class OSIRulesChecker(logger=None)

Bases: object

This class contains all the available rules to write OSI requirements and the necessary methods to check their compliance.

The rule methods are marked with *Rule*.

check_rule(parent_field, rule)

Check if a field comply with a rule given the *parent* field

log(severity, message)

Wrapper for the logger of the Validation Software

set_timestamp(timestamp, ts_id)

Set the timestamp for the analysis

osivalidator.osi_rules_implementations module

This module contains the implementation of each rule to be used in the requirements or in the Doxygen documentation.

All these rules are bounded into “OSIRulesChecker”, so they have access to all its attributes and methods.

add_default_rules_to_subfields(message, type_rules)

Add default rules to fields of message fields (subfields)

check_if(self, field, rule)

Evaluate rules if some statements are verified:

Parameters

  • params – statements

  • extra_params – do_check: rules to validate if statements are true

Structure:

a_field: - check_if: {params: statements} do_check: {extra_params: rules to validate if statements are true}

Example:

a_field: - check_if: - is_set: # Statements target: parent.environment.temperature - another_statement: statement parameter do_check: # Check that will be performed only if the statements are True - is_less_than_or_equal_to: 0.5 - is_greater_than_or_equal_to: 0

first_element(self, field, rule)

Check rule for first message of a repeated field.

Parameters

params – dictionary of rules to be checked for the first message (mapping)

is_different_to(self, field, rule)

Check if a number is different from the parameter.

Parameters

params – the inequality to check (float or bool)

Example: ` - is_different_to: 1 `

is_equal_to(self, field, rule)

Check if a number equals the parameter.

Parameters

params – the equality to check (float or bool)

Example: ` - is_equal_to: 1 `

is_globally_unique(self, field, rule)

Register an ID in the OSI ID manager to later perform a ID consistency validation.

Must be set to an Identifier.

Parameters

params – none

is_greater_than(self, field, rule)

Check if a number is over a minimum.

Parameters

params – the minimum (float)

is_greater_than_or_equal_to(self, field, rule)

Check if a number is over or equal a minimum.

Parameters

params – the minimum (float)

is_iso_country_code(self, field, rule)

Check if a string is a ISO country code.

Parameters

params – none

is_less_than(self, field, rule)

Check if a number is under a maximum.

Parameters

params – the maximum (float)

is_less_than_or_equal_to(self, field, rule)

Check if a number is under or equal a maximum.

Parameters

params – the maximum (float)

is_optional(self, field, rule)

This rule set the is_set one on a “Warning” severity.

Parameters

params – none

is_set(self, field, rule)

Check if a field is set or if a repeated field has at least one element.

Parameters

params – none

is_valid(self, field, rule)

Check if a field message is valid, that is all the inner rules of the message in the field are complying.

Parameters

params – none

last_element(self, field, rule)

Check rule for last message of a repeated field.

Parameters

  • field – Field to which the rule needs to comply

  • rule – dictionary of rules to be checked for the last message (mapping)

pre_check(func)

Decorator for rules that need to be checked before knowing that the field exists or not

refers_to(self, field, rule)

Add a reference to another message by ID.

Parameters

params – Type name of the referred object (string)

repeated_selector(func)

Decorator for selector-rules that take

rule_implementation(func)

Decorator to label rules method implementations

osivalidator.osi_scenario module

Module that contains OSIDataContainer class to handle and manage OSI scenarios.

class OSIScenario(show_progress=True, path=None, type_name='SensorView')

Bases: object

This class wrap OSI data. It can import and decode OSI scenarios.

cache_messages_in_index_range(begin, end)

Put all messages from index begin to index end in the cache. Then the method get_message_by_index can access to it in a faster way.

Using this method again clear the last cache and replace it with a new one.

from_file(path, type_name='SensorView', max_index=-1)

Import a scenario from a file

get_message_by_index(index)

Get a message by its index. Try first to get it from the cache made by the method cache_messages_in_index_range.

get_messages_in_index_range(begin, end)

Yield an iterator over messages of indexes between begin and end included.

retrieve_message_offsets(max_index)

Retrieve the offsets of all the messages of the scenario and store them in the message_offsets attribute of the object

It returns the number of discovered timesteps

update_bar(progress_bar, new_index)
get_size_from_file_stream(file_object)

Return a file size from a file stream given in parameters

osivalidator.osi_validator_logger module

Module which contains OSIValidatorLogger and some logging filters who wrap the Python logging module.

class ErrorFilter(name='')

Bases: logging.Filter

Filter for the logger which take INFO and ERROR messages

filter(record)

Determine if the specified record is to be logged.

Is the specified record to be logged? Returns 0 for no, nonzero for yes. If deemed appropriate, the record may be modified in-place.

class InfoFilter(name='')

Bases: logging.Filter

Filter which only take INFO messages

filter(record)

Determine if the specified record is to be logged.

Is the specified record to be logged? Returns 0 for no, nonzero for yes. If deemed appropriate, the record may be modified in-place.

class OSIValidatorLogger(debug=False)

Bases: object

Wrapper for the Python logger

create_database(timestamp, output_path)

Create an SQLite database and set the table for logs

debug(timestamp, msg, *args, **kwargs)

Wrapper for python debug logger

error(timestamp, msg, *args, **kwargs)

Wrapper for python error logger

flush(log_queue=None, timestamp=None, from_id=None)

Flush the output to the database

Parameters

  • log_queue – list of tuple of messages that have to be flushed

  • timestamp – if not None, only the timestamp given will be flushed

  • from_id – if None, the timestamp will be in millisecond, otherwise it will be the id of the timestamp according to the lookup table given in this parameter

info(timestamp, msg, *args, **kwargs)

Wrapper for python info logger

init(debug, verbose, output_path, files=False)

Initialize the OSI Validator Logger. Useful to reinitialize the object.

init_cli_output(verbose)

Initialize the CLI output

init_logging_storage(files, output_path)

Initialize (create or set handler) for the specified logging storage

synthetize_results_from_sqlite()

Aggregate the sqlite log and output a synthetized version of the result

warning(timestamp, msg, *args, **kwargs)

Wrapper for python warning logger

class WarningFilter(name='')

Bases: logging.Filter

Filter for the logger which take INFO and WARNING messages

filter(record)

Determine if the specified record is to be logged.

Is the specified record to be logged? Returns 0 for no, nonzero for yes. If deemed appropriate, the record may be modified in-place.

log(func)

Wrapper for logging function

print_synthesis(title, color, ranges_messages_table)

Print the (range, messages) table in a nice way, precessed with title and the number of messages

Module contents

OSI Validator