processor module

File : processor.py Author : Valentin Kuznetsov <vkuznet AT gmail dot com> Description: Processor module

Define a generic Processor object.

class OptionParser

Bases: object

User based option parser.

class PipelineItem

Bases: object

An object that can be supplied as one of the items in Pipeline.items.

execute(data, schema=None, **kwargs)

Run the appropriate method of the object and return the result.

Parameters:

• schema (str) – The name of a schema associated with the data that will be returned.

• kwargs (dict) – A dictionary of any positional and keyword arguments to supply to the read, process, or write method.

Returns:

The wrapped result of running read, process, or write.

Return type:

list[PipelineData]

get_config(data=None, config=None, schema='', remove=True, **kwargs)

Look through data for an item whose value for the first ‘schema’ key matches schema. Convert the value for that item’s ‘data’ key into the configuration’s Pydantic model identified by schema and return it. If no item is found and config is specified, validate it against the configuration’s Pydantic model identified by schema and return it.

Parameters:

• data (list[PipelineData], optional) – Input data from a previous PipelineItem.

• schema (str) – Name of the BaseModel class to match in data & return.

• config (dict, optional) – Initialization parameters for an instance of the Pydantic model identified by schema, required if data is unspecified, invalid or does not contain an item that matches the schema.

• remove (bool, optional) – If there is a matching entry in data, remove it from the list, defaults to True.

Raises:

ValueError – If there’s no match for schema in data.

Returns:

The first matching configuration model.

Return type:

BaseModel

get_data(data, name=None, schema=None, remove=True)

Look through data for an item whose ‘data’ value is a nexusformat.nexus.NXobject object or matches a given name or schema. Pick the item for which the ‘name’ key matches name if set or the ‘schema’ key matches schema if set, pick the last match for a nexusformat.nexus.NXobject object otherwise. Return the data object.

Parameters:

• data (list[PipelineData].) – Input data from a previous PipelineItem.

• name (str) – Name of the data item to match in data & return.

• schema (str) – Name of the BaseModel class to match in data & return.

• remove (bool, optional) – If there is a matching entry in data, remove it from the list, defaults to True.

Raises:

ValueError – If there’s no match for name or ‘schema` in data, or if there is no object of type nexusformat.nexus.NXobject.

Returns:

The last matching data item.

Return type:

obj

static get_default_nxentry(nxobject)

Given a nexusformat.nexus.NXroot or nexusformat.nexus.NXentry object, return the default or first nexusformat.nexus.NXentry match.

Parameters:

nxobject (nexusformat.nexus.NXroot, nexusformat.nexus.NXentry) – Input data.

Raises:

ValueError – If unable to retrieve a nexusformat.nexus.NXentry object.

Returns:

The input data if a nexusformat.nexus.NXentry object or the default or first nexusformat.nexus.NXentry object if a nexusformat.nexus.NXroot object.

Return type:

nexusformat.nexus.NXentry

static unwrap_pipelinedata(data)

Given a list of PipelineData objects, return a list of their data values.

Parameters:

data (list[PipelineData]) – Input data to read, write, or process that needs to be unwrapped from PipelineData before use.

Returns:

The ‘data’ values of the items in the input data.

Return type:

list[object]

class Processor

Bases: PipelineItem

Generic data processor.

The job of any Processor in a Pipeline is to receive data returned by the previous PipelineItem, process it in some way, and return the result for the next PipelineItem to use as input.

process(data)

Extract the contents of the input data, add a string to it, and return the amended value.

Parameters:

data – Input data.

Returns:

Processed data.

main(opt_parser=<class 'processor.OptionParser'>)

Main function.