Job Streams with In Conditions and Out Conditions

Current Situation

There is an event handling mechanism for job streams that works as an add-on to JobScheduler. The event handler determines on which condition what jobs should be started.

Desired Behavior

See the full Job Streams feature description.

A new event handling mechanism for job streams should be provided by a JobScheduler Plugin for better backward dependencies.

The plugin will be enabled with the config/scheduler.xml file like this:

<plugin java_class="com.sos.jobstreams.plugins.JobStreamsPlugin"/>"/>

Each job can have In Conditions and Out Conditions. Jobs are assigned to a job stream. The jobs of a job stream can be located in a number of folders.

In Conditions

• Each job can have one ore more In Conditions.
• Each In Condition has an expression and a list of commands that will be executed when the expression is evaluated to true.
• When the expression has been evaluated to true, the expression will be marked as "consumed".
  • Only non-consumed expressions will be evaluated
• An expression includes a number of boolean values such as
  • does an event exist?
    • The event is specified by event:name_of_event
      • event: is optional
      • name_of_event can be the name of the job stream that created the event.
      • name_of_event can have the date when the event was created
        • [*] : any date
        • [today] : this is the default
        • [yesterday] : event was created yesterday
        • [today - 2] : event was created the day before yesterday
        • [yesterday - 2] : event was created the day before yesterday
        • [yesterday + 1] : event was created today
        • 2019.169 : event was created in the year 2019 the 169th day
        • [prev]: Will be substituted with the date of the last run
        • [prevSuccessful]: Will be substituted with the date of the last successful run
        • [prevError]: Will be substituted with the date of the last run with error
  • does a file exist?
    • Example event:xxx and event:yyy[yesterday] and not (fileexist:test.txt)
    • Example xxx and myworkflow.yyy[today-2] and not (fileexist:test.txt)
  •  Did a job run?
    • lastCompletedRunEndedSuccessful
    • lastCompletedRunEndedWithError
    • lastCompletedRunEndedTodaySuccessful
    • lastCompletedRunEndedTodayWithError
    • lastCompletedRunEndedWithError
    • lastCompletedIsEndedBefore
    • lastCompletedSuccessulIsEndedBefore
    • lastCompletedWithErrorIsEndedBefore
    • lastCompletedIsStartedBefore
    • lastCompletedSuccessfulIsStartedBefore
    • lastCompletedWithErrorIsStartedBefore
    • isStartedToday
    • isStartedTodayCompletedSuccessful
    • isStartedTodayCompletedWithError
    • isStartedTodayCompleted
    • isCompletedToday
    • isCompletedTodaySuccessfully
    • isCompletedTodayWithError
    • isCompletedAfter
    • isCompletedWithErrorAfter
    • isCompletedSuccessfulAfter
    • isStartedAfter
    • isStartedWithErrorAfter
    • isStartedSuccessfulAfter

• A command can have parameters.
  • Commands are
    • start_job: starts the job the In Condition is assigned to.
      • Parameter: now, now+20 .. to specify when the job should start.
    • show_log: to create a info log line
      • Parameter: The log line
    • create_event: to create an event
      • Parameter: the name of the event
    • delete_event: to delete an event
      • Parameter: the name of the event.

Example for an expression

event:j_ev1[yesterday-1] and j_ev2[prev] and myWorkflow.j_ev3 and job:isCompletedToday or job:myJob.isCompletedTodaySuccessully

Out Conditions

• Each job can have one or more Out Conditions.
• Each Out Condition has an expression and a list of events that will be created or deleted when the expression is evaluated to true.
  • Each event can have the command create or delete
  • The default command is create
  • The command is specified with command:event
  • When the command is delete then in addition the job stream and the session can be specified.
    • example: create:event1
    • example: deletete:event1
    • example: delete:workflow.event1[yesterday]
• An expression includes a number of boolean values such as

• • Like In Conditions
  • Additionally return code of the last job run
    • The return code can be specified
      • single value: returncode:0
      • less than: returncode:-2
      • greater than: returncode: 2-
      • interval: returncode: 1-7
      • mixed: returncode:0,1-7,99-
    • Example returncode:0 and event:xxx and event:yyy and not (fileexist:test.txt)
    • Example returncode 1 and xxx and myworkflow.yyy and not (fileexist:test.txt)
    • Example returncode 0

Each time a job completes, all Out Conditions of the job will be evaluated. After that, all In Conditions of each job will be evaluated.

JOC Cockpit

JOC Cockpit provides

• an editor for In Conditions and Out Conditions for each job.
• a graphical view for jobs per job stream.
• adding an event manually
• removing an event manually
• removing the "consumed" flag for expressions
• resetting a job stream
  • removing all events in the job stream
  • removing all "consumed" flags in the job stream
• a list of job streams per folder
• a list of events
  • filtered by job stream
  • all events