Use metrics for selection of a Subagent

Current Situation

• Subagent Clusters can be configured for fixed-priority (active-passive) execution and for round-robin (active-active) execution of jobs.
• Users would like to add to this scheme selection of Subagents based on metrics.

Desired Behavior

• Basically, all available Subagents are candidates for execution of the next job.
• Subagent Plugin
  • A plugin mechanism allows implementation of a Java class that is executed by the Subgent
    • to collect metrics at an ongoing basis that remain with the plugin:
      • CPU consumption,
      • memory consumption.
    • to provide indicators from existing metrics, for example 80 to indicate 80% average CPU consumption during the last 2 minutes. The plugin returns indicators when requested by the Subagent.
  • Later versions of JS7 will offer an internal interface for Plugins.
• Director Agent
  • The Director Agent receives indicators from Subagents at an ongoing basis.
  • Additional indicators are provided by the Director Agent:
    • number of running jobs,
    • number of jobs running per Subagent in a Subagent Cluster
  • The Director Agent will check indicators provided by Subagents to decide which Subagent should execute the next job. If a Subagent does not respond within 0.5s then it is not considered for execution of the next job.
• Subagent Cluster
  • The Subagent Cluster configuration is improved to allow specification of an expression for the priority attribute of each Subagent:
    • An expression language is added that offers algebraic functions to be used for the priority.
  • Expressions
    • Indicators are made available from variables that can be used in a priority expression:
      • $js7SubagentProcessCount{: number of processes running with the Subagent
      • $js7ClusterSubagentProcessCount: number of processes running with the Subagent for the given Subagent Cluster
      • $js7CpuLoad
      • $js7CommittedVirtualMemorySize
      • $js7FreeMemorySize
      • $js7TotalMemorySize
    • Expressions must evaluate to numeric values. Expressions can return a missing value indicating that the related Subagent should not be used.
  • Examples for expressions
    • Example to consider the Subagent with the least number of tasks running for the given Subagent Cluster

      -$js7ClusterSubagentProcessCount

    • Example for conditional use of a Subagent that does not run any task

      if $js7SubagentProcessCount == 0 then 0 else missing

    • Example for conditional use of a Subagent that does not run a task in the given Subagent Cluster

      if $js7ClusterSubagentProcessCount == 0 then 0 else missing

    • Example for least consumption of CPU (weighted by 2) & Memory and least overall number of tasks running in the Subagent (weighted by 3):

      -$js7CpuLoad * 2 + $js7FreeMemorySize / 1000000000 - $js7SubagentProcessCount * 3

  • The Subagent with the highest priority value will be used for the next task. If the expression evaluates to a missing value then the Subagent is not considered for execution of the next job.