Creating Sub-Jobs

A sub-job is typically spawned from a job programmatically as part of an algorithm where the execution is parallelized. It only differs from regular jobs as it keeps a reference to the job it was spawned from (ParentJob) and a flag that marks it as a sub-job. As there are no restrictions regarding the relation between job and sub-job, any job can be used as parent for a sub-job (even sub-jobs).

    from hypermedia_client.job_management.tool import Job

    ...

    job = Job(client)
    job.create(name="MyJob")

    sub_job = job.create_sub_job(name="MySubJob")

    ...

    job.delete()
    sub_job.delete()

However, the intended use is to spawn a job from inside the currently running job. The ProCon framework provides convenience functions to acquire the current job-object already initialized with the client connecting to the corresponding JMA.

from pinexq.procon.step import Step
from pinexq.procon.jobmanagement import job_from_step_context


# noinspection PyMethodMayBeStatic
class SubJobStep(Step):
    """
    Job-master example to manage sub-jobs from Step functions.
    """

    @version("0.1.0-dev1")
    def dev_primes_in_range(self, n: int) -> list[int]:
        """Returns all prime numbers between 0..n.

        Args:
            n: the upper limit of the range.
        Returns:
            A list of all numbers that are primes.
        """
        current_job = job_from_step_context(self)
        sub_jobs = []
        for i in range(n):
            sub_jobs.append((
                current_job
                .create_sub_job(f'test_for_prime_{n}')
                .select_processing('dev_is_prime')
                .configure_parameters(p=i)
                .start()
            ))
        current_job.wait_for_sub_jobs_complete(timeout_ms=60_000)
        sub_job_results = [job.get_result() for job in sub_jobs]

        return [p for p, is_prime in sub_job_results if is_prime]

    @version("0.1.0-dev1")
    def dev_is_prime(self, p: int) -> tuple[int, bool]:
        """Test if a given number is a prime.

        Args:
            p: A integer number to be tested
        Returns:
            A tuple with `p` and True if `p` is a prime, otherwise False.
        """
        for i in range(2, p):
            if (p % i) == 0:
                return p, False
        return p, True


if __name__ == '__main__':
    SubJobStep()

The job_from_step_context(self) function requests the Job from the Step container of the current worker. After all sub-jobs are instantiated and started, the wait_for_sub_jobs_complete() method of the parent job allows it to wait for all its sub-jobs to finish execution. This will synchronously block the current thread until completion. For more fine-grained control of the sub-jobs’ status, you may query the state of each sub-job explicitly.