Add timely logging for output to stdout and stderr

1. Problem

Output of jobs to stdout/stderr is written to the scheduler log at the end of a task. For long running tasks this is a major drawback as you would not know what a job is doing unless it is terminated.

2. Solution

A task should be accompanied by a process or thread that checks the size of the temporary files for stdout and stderr. The check interval is not less than 10s. When a change of the file size is detected then content of the temporary file is added to the scheduler log by a spooler_log.info() call. For optimisation the delta to a previous file check is considered by maintaining the position of a previous check and reporting the difference.

Detail a): Should a line in a temporay file for stdout/stderr not be terminated by \n then this line is cached and will be added to the scheduler log after a later (or final) file check.

Detail b): This behaviour applies in the same way to locally and remotely executed jobs, i.e. the task communicates output to stdout/stderr to the Job Scheduler. No process or functionality of the Job Scheduler main task is involved when a tasks updates the scheduler log. This requirement is essential to support output of remotely executed jobs that report directly to a local Job Scheduler instance.

3. Scope

• This solution applies to any jobs writing output to stdout/stderr including job types as <job language="shell"> and <process>.
• This solution is available for remotely executed jobs.