Allow targeted activation of JobScheduler Master cluster member

Current Situation

• The JobScheduler Master can be operated as a passive cluster with one active (primary) instance and any number of inactive (backup) instances.
• The Master startup parameter -exclusive allows to specify the initially active (primary) instance. The startup parameters -exclusive -backup -precedence allow to specify the inactive (backup) instances. The precedence determines which backup Master will become active in case of fail-over or switch-over.
• The cluster configuration is static from startup parameters. 

Desired Behavior

• The cluster configuration is made available in a dynamic way that allows users to choose which backup instance is to become active. This behavior is desired for environments without automatic fail-over when users operate their own (hardware/software) cluster and do not want the JobScheduler Master to decide to which instance fail-over is performed.
• To this purpose the Master will consider the paused mode of a backup instance during fail-over and switch-over:
  • Users can start a Master in paused mode. This operation is available from the command line:

    ./jobscheduler.sh|.cmd start -pause
    

• • Users can revert the paused mode of a Master. This operation is available from the command line and from the GUI:

    ./jobscheduler.sh|.cmd continue
    

• The Master Cluster will not consider any paused inactive (backup) instance for fail-over and switch-over.
• Example
  • Assume 3 Master instances P1 (active, primary), B1 (inactive backup 1), B2 (inactive backup 2)
  • Users can put B1 and B2 in paused mode. This prevents automatic fail-over in case that P1 becomes unavailable.
  • Users can decide to revert the paused mode of B1 or B2. The first backup Master that is not in paused mode will become active for the JobScheduler Cluster.

Maintainer Note

• A similar result can be achieved more easily if only one of the Master instances P1, B1, B2 is started at a given time.
• However, users might prefer to use running Master instances and not to start the Master on demand.
• If you always want to start the JobScheduler with the command line option -pause then you can put this into the start parameter.
  ./bin/jobscheduler_environment_variables.sh

   SCHEDULER_START_PARAMS="$SCHEDULER_PARAMS \"-pid-file=$SCHEDULER_PID\" -env=LD_LIBRARY_PATH=\"$ORIG_LD_LIBRARY_PATH\" -pause"
  

  .\bin\jobscheduler_environment_variables.cmd

   set SCHEDULER_START_PARAMS=%SCHEDULER_PARAMS% -pid-file=%SCHEDULER_PID% -pause
  

  then you can omit the option in the command line.