Provide a function for checking the job history in job monitors by use of the XML interface

Starting Point

• Users would like to check in individual job scripts or monitor scripts
  • if a job has been executed in the current period, e.g. for the current day
  • what execution result (success/failure) was created on completion of the job
• The intention for checking the job execution history is to automate decision making based on the point in time or execution result of a previous job start.

Feature

• This feature is a subset of JITL-212
• A Java object is provided for history checks that can be used
  • in individual jobs that are implemented with Java or JavaScript.
  • in individual job monitors that are implemented with Java or JavaScript.
• The job history returns a set of objects and properties:
  • Objects
    • Object running: the current job run
    • Object lastCompleted: the last job run being completed
    • Object lastCompletedSuccessful: the last, successfully completed execution of the job
    • Object lastCompletedWithError: the last, unsuccessfully completed execution of the job
  • Properties per object:
    • Property name: Job Name
    • Property id: Task ID
    • Property position: position of the object in the job execution history. A value 0 signals the most recent entry, larger values suggest older entries.
    • Property found: true if the associated object is not null
    • Property start; the start date of the job run
    • Property end; the end date of the job run
    • Property executionResult: the exit code or -1
    • Property error: 1 if the exit code is > 0
    • Property errorMessage: the error message
    • Property errorCode; can include an error code
• When querying the job history the following arguments are specified for an instance of the JobHistory class.
  • The job name (a relative name or absolute path)
  • The limit for the number of entries from the job history
  • The limit for specific point in time up to that the job history is considered
• The job history provides the following methods:
  • JobHistoryInfoEntry getLastExecution(): get the history object for the most recently started job run including running jobs and jobs that completed successfully or with error.
  • JobHistoryInfoEntry getLastCompletedExecution(): get the history object for the most recently started job run for jobs that completed successfully or with error.
  • boolean isStartedToday(): indicates that the job run started within the current period and applies to running jobs and completed job runs.
  • boolean isStartedTodayCompleted(): indicates that the job run started and completed within the current period and applies to job runs that completed successfully or with errror.
  • boolean isStartedTodayCompletedSuccessful(): indicates that the job run started and completed successfully within the current period.
  • boolean isStartedTodayCompletedWithError(): indicates that the job run started and completed with error within the current period.
  • boolean isCompletedToday(): indicates that the job run completed in the current period and applies to job runs that completed successfully or with error.
  • boolean isCompletedTodaySuccessful(): inicates that the job run completed successfully within the current period.
  • boolean isCompletedTodayWithError(): indicates that the job run completed with error within the current period.
  • boolean endedAfter("03:00:00"): indicates that the job run completed after the given date.
  • boolean endedWithSuccessfulAfter("03:00:00"): indicates that the job run completed successfully after the given date.
  • boolean endedWithErrorAfter("03:00:00"): indicates that the job run completed with error after the given date.
  • boolean startedAfter("03:00:00"): indicates that the job run started after the given date. The job might be completed or running.
  • boolean startedWithSuccessfulAfter("03:00:00"): indictes that the job run started after the given date and does not report any errors. The job might be completed or running.
  • boolean startedWithErrorAfter("03:00:00"): indicates that the job run started after the given date and does report errors. The job might be completed or running.
  • int getCount());
  • since 1.9.8 1.20.2
    • int getNumberOfCompleted()
    • int getNumberOfCompletedSuccessful()
    • int getNumberOfCompletedWithError()
    • int getNumberOfStarts());
    • boolean isCompletedBefore
    • boolean isCompletedWithErrorBefore
    • boolean isCompletedSuccessfulBefore

Usage

• Example for use of the methods with a job script, see attached check_run_history.job.xml:

  function spooler_process()  {
  
      // using the current job name and default limit
      var jobHistoryInfo = getJobHistoryInfoObject("job1");
  
      // using the current job name and the given limit        
      // var jobHistoryInfo = getJobHistoryInfoObject( "", 30 );
  
      // using the given job name and default limit
      // var jobHistoryInfo = getJobHistoryInfoObject( "job1" );
  
      // using the given job name and the given limit
      // var jobHistoryInfo = getJobHistoryInfoObject( "job1", 20 );
  
      // Since 1.9.8 1.10.2
      // jobHistoryInfo = jobHistory.getJobInfo("job1","-8:10:00:00..-4:14:00:00");
      // reporting some info on job runs
  
      report( jobHistoryInfo.getLastCompleted() );
      report( jobHistoryInfo.getRunning() );
      report( jobHistoryInfo.getLastCompletedSuccessful() );
      report( jobHistoryInfo.getLastCompletedWithError() );
  
      // report on currently running job
      report( jobHistoryInfo.getLastExecution() );
   
      spooler_log.info("isStartedToday: " + jobHistoryInfo.isStartedToday());
      spooler_log.info("isStartedTodayCompletedSuccessful: " + jobHistoryInfo.isStartedTodayCompletedSuccessful());
      spooler_log.info("isStartedTodayCompletedWithError: " + jobHistoryInfo.isStartedTodayCompletedWithError());
      spooler_log.info("isStartedTodayCompleted: " + jobHistoryInfo.isStartedTodayCompleted());
      spooler_log.info("isCompletedToday: " + jobHistoryInfo.isCompletedToday());
      spooler_log.info("isCompletedTodaySuccessful: " + jobHistoryInfo.isCompletedTodaySuccessful());
      spooler_log.info("isCompletedTodayWithError: " + jobHistoryInfo.isCompletedTodayWithError());
  
      spooler_log.info("endedAfter: " + jobHistoryInfo.endedAfter("03:00:00"));
      spooler_log.info("endedWithErrorAfter: " + jobHistoryInfo.endedWithErrorAfter("03:00:00"));
      spooler_log.info("endedSuccessfulAfter: " + jobHistoryInfo.endedSuccessfulAfter("03:00:00"));
  
      spooler_log.info("startedAfter: " + jobHistoryInfo.startedAfter("03:00:00"));
      spooler_log.info("startedWithErrorAfter: " + jobHistoryInfo.startedWithErrorAfter("03:00:00"));
      spooler_log.info("startedSuccessfulAfter: " + jobHistoryInfo.startedSuccessfulAfter("03:00:00"));
  
   
  
  	spooler_log.info("isStartedToday:" + jobHistoryInfo.queryHistory("isStartedToday"));
      spooler_log.info("isStartedTodayCompletedSuccessful:" + jobHistoryInfo.queryHistory("isStartedTodayCompletedSuccessful"));
      spooler_log.info("isStartedTodayCompletedWithError:" + jobHistoryInfo.queryHistory("isStartedTodayCompletedWithError"));
      spooler_log.info("isStartedTodayCompleted:" + jobHistoryInfo.queryHistory("isStartedTodayCompleted"));
      spooler_log.info("isCompletedToday:" + jobHistoryInfo.queryHistory("isCompletedToday"));
      spooler_log.info("isCompletedTodaySuccessfully:" + jobHistoryInfo.queryHistory("isCompletedTodaySuccessful"));
      spooler_log.info("isCompletedTodayWithError:" + jobHistoryInfo.queryHistory("isCompletedTodayWithError "));
  		
      //since 1.10.2 1.9.8 also possible
      spooler_log.info("isCompletedAfter:" + jobHistoryInfo.queryHistory("isCompletedAfter(-1:10:48:33)"));
      spooler_log.info("isCompletedWithErrorAfter:" + jobHistoryInfo.queryHistory("isCompletedWithErrorAfter(03:00:00)"));
      spooler_log.info("isCompletedSuccessfulAfter:" + jobHistoryInfo.queryHistory("isCompletedSuccessfulAfter(03:00:00)"));
  
      spooler_log.info("isStartedAfter:" + jobHistoryInfo.queryHistory("isStartedAfter(-1:10:48:33)"));
      spooler_log.info("isStartedWithErrorAfter:" + jobHistoryInfo.queryHistory("isStartedWithErrorAfter(03:00:00)"));
      spooler_log.info("isStartedSuccessfulAfter:" + jobHistoryInfo.queryHistory("isStartedSuccessfulAfter(03:00:00)"));
  
      // example how to convert Date fields (start date)
      if ( jobHistoryInfo.getLastCompletedSuccessful().found ) {
          var d = jobHistoryInfo.getLastCompletedSuccessful().start;
          if (d != null) {
   
              spooler_log.info ("------->" + d );
              spooler_log.info("-------------> d.toString(): " + lastRunCompleted);
              spooler_log.info("-------------> with format: " + d.format(java.time.format.DateTimeFormatter.ofPattern("dd.MM.yyyy HH:mm:ss")));
  
          }
      }
  
      spooler_log.info ( "check if the job started on the previous day before a specific point in time" );
      var jobHistoryInfo = getJobHistoryInfoObject( "job1", "", "-1:10:43:56" );
      spooler_log.info ( "completedBefore -1:10:43:56: " + jobHistoryInfo.isCompletedBefore());
      spooler_log.info ( "completedWithErrorBefore -1:10:43:56: " + jobHistoryInfo.isCompletedWithErrorBefore() );
      spooler_log.info ( "completedSuccessfulBefore -1:10:43:56: " + jobHistoryInfo.isCompletedSuccessfulBefore() );
  
      //since 1.10.2 1.9.8 also possible
      spooler_log.info ( "completedBefore -1:10:43:56: " + jobHistoryInfo.queryHistory("isCompletedBefore"));
      spooler_log.info ( "completedWithErrorBefore -1:10:43:56: " + jobHistoryInfo.queryHistory("isCompletedWithErrorBefore"));
      spooler_log.info ( "completedSuccessfulBefore -1:10:43:56: " + jobHistoryInfo.queryHistory("isCompletedSuccessfulBefore"));
       
  // The time intervall can be given as:
  // from..to
  // ..to           from will be 00:00:00
  // from..       to will be 00:00:00
  // to             from will be 00:00:00
   
      return false;
  }
  
  
  
  function getJobHistoryInfoObject(jobname, limit, timelimit) {
      var jobHistory = new Packages.com.sos.jitl.checkrunhistory.JobHistory( spooler.delegate );
  
      if ((jobname === undefined) || (jobname === "")) {
        jobname = spooler_task.job.name;
      }
      if (limit === undefined || limit === "") {
         limit = "100";
      }
      if (timelimit === undefined) {
         timelimit = "";
      }
  
  
      var jobHistoryInfo = jobHistory.getJobInfo( jobname, limit, timelimit );
  
      //since 1.10.2 1.9.8 also possible
  	spooler_log.info("Records found:" + jobHistory.getCount());
  
  	spooler_log.info ("Completed Records found:" + jobHistory.getNumberOfCompleted());
  	spooler_log.info ("CompletedSuccessfulRecords found:" + jobHistory.getNumberOfCompletedSuccessful());
  	spooler_log.info ("CompletedWithError Records found:" + jobHistory.getNumberOfCompletedWithError());
  	spooler_log.info ("Starts Records found:" + jobHistory. getNumberOfStarts());
  
      return jobHistoryInfo;
  }
  
  function report( reportItem ) {
      spooler_log.info( "_____________________________" );
      if (reportItem.found) {
          spooler_log.info( "Name:" + reportItem.name );
          spooler_log.info( "Task ID: " + reportItem.id );
          spooler_log.info( "Job Name: " + reportItem.jobName );
          spooler_log.info( "Position: " + reportItem.position );
          spooler_log.info( "Start Time: " + reportItem.start );
          spooler_log.info( "End Time: " + reportItem.end );
          spooler_log.info( "Exit Code: " + reportItem.executionResult );
          spooler_log.info( "Error Message: " + reportItem.errorMessage );
          spooler_log.info( "Error: " + reportItem.error );
          spooler_log.info( "ErrorCode: " + reportItem.errorCode );
      } else {
          spooler_log.info( "Name: " + reportItem.name + " not found" );
      }
  }
  

• To identify if a running job is a rerun of a previously failed job execution you can use the following script for a pre-processing monitor. The script will expose the information if a rerun has been identified with the environment variable SCHEDULER_PARAM_IS_RERUN, see attached check_rerun.job.xml for an integrated job sample and check_rerun_with_monitor_use.job.xml and check_rerun.monitor.xml] for a re-usable monitor sample:

  function spooler_task_before() {
      var jobHistory = new Packages.com.sos.jitl.checkrunhistory.JobHistory( spooler.delegate );
      var jobHistoryInfo = jobHistory.getJobInfo( spooler_task.job.name, 100, "" );
      var isRerun = (jobHistoryInfo.getLastCompletedWithError().found && jobHistoryInfo.getLastCompletedWithError().position == 0);
  
      var params = spooler_task.params;
      params.set_var("IS_RERUN", isRerun );
      spooler_log.info("Job rerun identified: " + isRerun);
  
      return true;
  }
  

  The script retrieves the last job execution that completed with error and checks if the position of the object provided is the first in the recent history list. The first position of a job that recently completed with error is

  • 0 if the script is executed as a pre-processing monitor script before a task start.
  • 1 if the script is executed as a job script for a running task. That task would be added to the begin of the recent history list.
• In order to check if a job did run before a given point in time you can query a job history object that takes a time limit into consideration:

  	jobHistoryInfo = jobHistory.getJobInfo( "job1", 100, "-1:10:43:56" );
  	spooler_log.info( "endedBefore -1:10:43:16:" + jobHistoryInfo.getLastCompleted().found );
  	spooler_log.info( "endedBeforeWithError -1:10:43:16:" + jobHistoryInfo.getLastComletedWithError().found );
  	spooler_log.info( "endedBeforeSuccessful -1:10:43:16:" + jobHistoryInfo.getLastCompletedSuccessful().found );
  

  or

    	jobHistory.setTimeLimit( "-1:10:43:56" );
  	jobHistoryInfo = jobHistory.getJobInfo( "job1", 100, "" );
  	spooler_log.info( "endedBefore -1:10:43:16:" + jobHistoryInfo.getLastCompleted().found );
  	spooler_log.info( "endedBeforeWithError -1:10:43:16:" + jobHistoryInfo.getLastComletedWithError().found );
  	spooler_log.info( "endedBeforeSuccessful -1:10:43:16:" + jobHistoryInfo.getLastCompletedSuccessful().found );
  

Implementation

• The implementation makes use of API methods, i.e. XML commands, to retrieve the job history instead of a database connection.
• The solution can be used with Agents.
• No separate HTTP connection is created, the solution makes use of the HTTP(S) connection that is established between Master and Agent.