YADE should support the -return-values parameter to forward the file transfer history to a Master/Controller

Current Situation

• YADE can be operated as a JITL job for JobScheduler 1.x and can be executed by its CLI.
• When executed as a JITL job then
  • parallelism of tasks slows down performance,
  • the file transfer history is not supported if YADE is operated with an Agent that has no access to the reporting database.

Desired Behavior

• YADE should be executable as a shell job to provide better performance with parallel tasks.
• When running as a JITL job then YADE should benefit from better performance by asynchronously feeding the file transfer history. This behavior can be activated by use of the parameter async_history = true.
• The YADE shell job should be added the -return-values parameter that is assigned the temporary file used by JobScheduler to forward order variables to subsequent jobs.
  • for JobScheduler 1.13 the path of the temporary file is available from the SCHEDULER_RETURN_VALUES environment variable.
  • for JobScheduler 2.0 the path of the temporary file is available from the JS7_RETURN_VALUES environment variable.
• If the -return-values parameter is used then YADE should write the transfer history entries (indicating which files have been transferred) in a compressed format to an order variable.

Maintainer Notes

• The functionality is primarily targeted towards use of YADE as a shell job with JS7.
• For use of YADE as a JITL job with JobScheduler 1.x users can expect performance benefits.
• It remains to be checked if the functionality later on can be made available for use of YADE with a shell job for JobScheduler 1.x
  • The file transfer history for the YADE shell job is not implemented with release 1.13.9 as this requires to pass an order variable to the temporary file indicated by SCHEDULER_RETURN_VALUES. The JobScheduler Master 1.x currently does not support to pick up order variables used for the file transfer history.