Provide task logs and order logs from files

Current Situation

• The JS7 streams output created by jobs to the stdout and stderr channels, it does not create files. This applies to the Agent that streams log output to the Controller and it applies to the Controller that streams log output to JOC Cockpit.
• There are many good reasons for this behavior such as speed, disk space consumption and cleanup of log files.
• However, users would like to have log output available from files, for example for analysis with tools such as Splunk, Elastic Search etc.

Desired Behavior

• The JOC Cockpit History Service currently creates temporary files from the log output of orders and tasks. The files are compressed and are added to the JS7 database.
• In addition, the History Service should copy temporary log files to a location in the file system.
  • File names in the target location include
    • task.<workflow-name>.<order-id>.<job-label>.<task-id>.log
    • order.<workflow-name>.<order-id>.log
    • order.<workflow-name>.<order-id>.json
  • Users can pick up files from this location using 3rd-party tools or from a workflow triggered by File Watching. Users are responsible for cleanup of log files in the target location.
  • Users can configure log file creation to be applicable to successful/failed tasks and/or orders. In addition the target directory can be configured.
  • The configuration items are made available from the joc section the JOC Cockpit Settings page:
    • log_ext_directory
    • log_ext_order_history
    • log_ext_order
    • log_ext_task
• Users should consider the timing of log file creation:
  • task logs are created immediately on arrival with JOC Cockpit, i.e. in near real-time.
  • order logs are created on completion of an order.