Parameters

The parameters of a ProcessingStep are specified by the type annotations given. These are required so the functions signature is well-defined and can be passed to the functions manifest. For larger inputs, or inputs that should depend on other computations, use a DataSlot instead. Built-in Python types, such as bool, str, int, float, and list, can be passed directly, and any desired constraints can be added through Pydantic.

Caution

While Pydantic can handle many ‘imported’ parameters, such as dates and StrEnums, you must wrap these in a dataclass or Pydantic model, as in the examples below, or annotate them to be parsed in non-strict mode.

primitives.py

from typing import Any
from pydantic import conlist, NonNegativeInt

def many_parameters(
            self,
            a: dict,
            b: int,
            c: conlist(int, max_length=3) = (1, 2, 3),
            d: bool = True
) -> int:

my_dataclass.py

import dataclasses
from dataclasses import dataclass

@dataclass
class SubDataclass:
    x: int
    y: float

@dataclass
class MyDataclass:
    i: int
    f: float
    sub: SubDataclass

def use_dataclass(self, my_dc: MyDataclass) -> int:
    ...

my_pydantic_model.py

from pydantic import BaseModel, Field
from typing import Annotated
from enum import StrEnum
import datetime

class Lang(StrEnum):
    ENGLISH = 'EN'
    GERMAN = 'DE'

class SubBaseModel(BaseModel):
    # annotate type union with types, which are displayed on the portal.
    text: Annotated[str, Field(title="str")] | Annotated [None, Field(title="none")]
    date: datetime.date

class MyBaseModel(BaseModel):
    lang: Lang
    sub: SubBaseModel

def use_basemodel(self, my_bm: MyBaseModel) -> int:
    ...

Enriching parameters with metadata

By default each parameter is described in the functions manifest by its name as title and the docstring is used as description. To allow more control when configuring a ProcessingSteps parameters it is possible to manually add Pydantic annotations. These annotations will be carried over to the ProcessingSteps manifest (as JsonSchema) and will also be used in the Portal to render input forms.

Annotating in the function signature

When generating the JsonSchema for parameters we use pydantic to extract it. Using Python’s typing.Annotated it is possible to attach metadata to types. Following the signature: Annotated[<type>, <metadata>] the first item is the type and the second the metadata. Since Pydantic needs to see and understand the metadata the Field object is used to carry title and description, or control whether Pydantic’s validation is strict:

from typing import Annotated
from pydantic import Field
from pinexq.procon.step import Step, version
import datetime

class MyStepCollection(Step):
    @version("0.1.2")
    def showcase_annotated(
            self,
            dataset_name: Annotated[str, Field(title="Dataset Name", description="The name used to find the dataset inside the given root data.")],
            dataset_date: Annotated[datetime.date, Field(title="Dataset Date", strict=False)], # must be non-strict to parse string into date
            index: Annotated[int, Field(title="Index", description="The n-th item used from the selected dataset")],
        ) -> str:
            """..."""
            return "my result"

Annotating dataclass

Enhancing a dataclass follows the same concept as used in function signatures by adding typing.Annotated with pydantic Field:

from dataclasses import dataclass
from typing import Annotated
from pydantic import Field

@dataclass
class SimplePoint:
    # Use Annotated to attach Pydantic metadata to standard dataclasses
    x: Annotated[int, Field(title="X Coordinate", description="Horizontal axis")]
    y: Annotated[int, Field(title="Y Coordinate", description="Vertical axis")]

Annotating Pydantic models

In Pydantic models the field object can be used directly:

from pydantic import BaseModel, Field
from typing import Annotated

class Tree(BaseModel):
    """
    This docstring can also serve as the model description
    if no title/description is provided in Config.
    """
    name: Annotated[str, Field(title="str")] | Annotated[None, Field(title="None")] = Field(title="Tree family name", description="The family name of the tree"))
    age: int = Field(title="User Age", description="Age in years")

Take care not to confuse annotations of types with annotations of fields!

When to annotate

Annotating parameters allows more information to be shown on the portal, and thus to your users. While this is always nice to have, there are some situations where it is especially important to annotate:

• When using a type union, the component types should always be given titles in annotation, as these cannot be inferred by the portal. This is especially important if each option has a particular meaning.
• If the internal name of a variable is not clear, the variable should be given a human-readable title, and optionally a description.
• If a parameter is a non-primitive object (and not a model or dataclass), such as a datetime.date or a StrEnum, the annotation must include strict=False in order to allow parsing.
• While Pydantic models are usually annotated via Fields, it is sometimes also necessary to annotate the types, e.g. in type unions, or to use the same validator for every field of a given type.