Allow Agents to be paused and continued

Current Situation

• All Agents running in an Agent Cluster are considered for job execution by a Master.
• Should users want an Agent not to be considered for job execution then the Agent has to be stopped by use of the Agent start script or its Web Service API.

Desired Behavior

• Users would want to pause job execution for a specific Agent instance or for an Agent Cluster, e.g. due to maintenance windows.
• The Agent should not be stopped (terminated), but put into a paused state.
  • When a pause is added then no new tasks are started for an Agent. Any running tasks can complete.
• Paused Agents can be continued by a respective REST Web Service request.
• The operation to pause and to continue an Agent
  • should be available from the Agent Web Service.
  • should be available from the Master Web Service (the JOC Cockpit does not connect to Agents directly) and
    • should include to specify a process class, i.e. all Agent instances of the Agent Cluster are paused/continued,
    • should include to specify individual Agent instances by their respective URL.
• If an Agent Cluster is paused then
  • this includes that any tasks that are started for the Agent Cluster remain in state "waiting_for_resource".
  • the behavior is the same as the configuration <process_class max_processes="0"/>
• If an individual Agent it paused in an Agent Cluster
  • then the Agent is handled by the Master in same way as if it were unreachable.
  • then In a passive or active Agent Cluster the remaining Agents will be assigned tasks by the Master.
• The status information of an Agent that is paused is returned by the respective Agent Web Service response to a status request.

Further Implementation

• The Agent start script should support use of the "pause" and "continue" commands (see later separate issue).
• The JOC Cockpit REST Web Services and GUI should implement respective operations (see later separate issue)

Maintainer Notes

• This feature is not intended for immediate implementation and therefore is not assigned a release.
• Please vote for fhis feature if you want this to become available in future releases.