New Log Management

Current Situation

• JobScheduler logs using a proprietary log mechanism
  • into several log files
    • scheduler.log
    • scheduler-[timestamp].log
    • task logs
    • order logs
  • into the database
    • SCHEDULER_ORDER_HISTORY
    • SCHEDULER_TASK_HISTORY
• Jobs write their logs
  • using JobScheduler log api directly
  • using log4j or SLF4J and an adapter for JobScheduler log api
    • log levels are reduced to log levels available from log4j
    • some "good" log4j features are not used because the log must be piped back to JobScheduler log api
    • worst of both worlds
• JOC displays logs
  • by requesting them from JobScheduler by http
  • with auto-update: http connection is held open and produces unfinished html

Feature

• Logs are collected by a log server (i.e. logstash)
• JobScheduler and jobs write logs in a way that provides metadata which can be collected by the log server
• Filtering over metadata
• JOC displays logs
  • by requesting them from the logserver
  • with auto-update