Processing Container (ProCon)

Computations in PineXQ are done by Workers running a ProcessingContainer, or short ProCon, which is also the name of the framework. ProCon provides an unobtrusive wrapper around function definitions without introducing new semantics, allowing for a clean definition of the computational task, while handling all cloud-related communication and data-management transparently in the background. Workers can execute a predefined set of these functions called ProcessingSteps. To the outside each ProcessingStep exposes a Manifest, containing a description of its parameters, input- and output-data, and the documentation, allowing to compose processing chains from these single steps.

Parameters and results can be persisted as WorkData using DataSlots, which can be reused in other computations. DataSlots handle accessing data by passing it as a simple parameter to the hosted ProcessingStep, while abstracting away how to access and (de-)serialize the data. In its most basic form WorkData is then represented as files, but, depending on the platform the container runs on, it can be any form of storage. This removes the code and configuration required from the function implementation.