Uploaded image for project: 'JOC - JobScheduler Operations Center'
  1. JOC - JobScheduler Operations Center
  2. JOC-1655

Provide scripts to clean up the Daily Plan

    XMLWordPrintable

Details

    Description

      Current Situation

      Dependencies between workflows include that JS7 - Notice Boards are used by a JS7 - PostNotices Instruction to create Notices that other workflows are waiting for using the JS7 - ExpectNotices Instruction and JS7 - ConsumeNotices Instruction.

      • By default such dependencies remain in place for an arbitrary duration until they can be resolved.
      • Example:
        • Workflow A includes a PostNotices Instruction to create a Notice.
        • Workflow B includes a ConsumeNotices Instruction to expect and to remove the Notice.
        • Assume that Workflow A is delayed - possibly for days - and that no related Notice is created. This makes the order in Workflow B wait until the Notice arrives. Should the order in Workflow A be cancelled then the order in Workflow B will wait endlessly unless it is cancelled too. The reason being that Notices are based on Daily Plan dates and act on orders that share the same Daily Plan date as the Notice.

      For some users this is the desired behavior, for others it is not: users handle dependencies in the JS7 - Daily Plan individually.

      • Some users want dependencies to remain in place until they can be resolved.
        • Example:
          • Assume Workflow A to load reporting data to a Data Warehouse on a daily basis. Assume Workflow B to run reporting jobs for the Data Warehouse.
          • If reporting data are sequential and build upon each other then the order in Workflow B has to wait until reporting data provided by Workflow A are present before creating a report.
          • In this scenario users want Workflow B to be delayed for hours or days until data from Workflow A are ready as the sequential nature of data delivery forces this behavior.
      • Some users want dependencies to be dropped after a certain period.
        • Example:
          • Assume a scenario of classic batch processing when Workflow A loads data to a database that are processed by Workflow B with both workflows running on a daily basis.
          • If Workflow A is late within a day then the order for Workflow B should wait. However, if Workflow A exceeds the 24 hours' period of the Daily Plan it should be cancelled as the same Workflow A is executed by a new order on the next day. In this scenario Workflow B should not wait longer for Workflow A than the Daily Plan's period.

      Desired Behavior

      • The effective solution will become available with JS-2100 that implements the Daily Plan and dependencies with the Controller and allows to drop Orders and Notices on completion of the Daily Plan's 24 hours' period.
      • An intermediate solution is provided from shell scripts as explained with JS7 - How to clean up the Daily Plan from past Dependencies

      Attachments

        Issue Links

          Activity

            People

              ap Andreas Püschel
              ap Andreas Püschel
              Andreas Püschel Andreas Püschel
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: