Offer flexible options how to run the Cleanup Service

Current Situation

• JOC Cockpit's Cleanup Service works in a discreet way: it deletes data in batches (default: 1000 records) and checks if the History Service adds entries to the job history in parallel. Should this be true then the Cleanup Service terminates having processed the current batch and will restart later on.
• The reason being that some DBMS products are particularly sensitive to concurrent transactions: MySQL (InnoDB) does not offer row level locking for indices and will more frequently raise deadlocks. PostgreSQL and Oracle are less susceptible to concurrent transactions.

Desired Behaviour

• The Cleanup Service is modified to work without transactions.
  • This includes to pick up deletion of records from any situation when the Cleanup Service might have been aborted inadvertedly.
  • This guarantees that no deadlocks will occur when running the Cleanup Service and History Service in parallel.
• The Cleanup Service is added two settings:
  • to force start of the Cleanup Service at the given point in time.
    • If the option is used then no check is performed for concurrent transactions of the History Service.
    • If the option is not used then the behavior as specified in section "Current Situation" is applied.
  • to pause the History Service when the Cleanup Service is running.
    • For DBMS such as PostgreSQL and Oracle it is safe to use this option.
    • For DBMS such as MySQL it is not recommended to use this option.
    • Pausing the History Service includes to set a timeout that is in proportion to the batch size. Experience tells that a batch of 1000 records will be deleted in <2s.  The Cleanup Service will tell the History Service to consider the timeout in case that the Cleanup Service will not complete the current transaction and otherwise will release the History Service's timeout before expiration.
• The Cleanup Service is added the operation "Start now" from a button in the Settings page:
  • The Cleanup Service will start immediately and will consider existing settings to pause or not to pause the History Service.