Uploaded image for project: 'JOC - JobScheduler Operations Center'
  1. JOC - JobScheduler Operations Center
  2. JOC-440

Improve Performance of Log Downloads

    XMLWordPrintable

Details

    Description

      Current Situation

      • The JobScheduler main log is offered for download with ./jobscheduler/log API.
      • This API results in an OutOfMemoryError if the main log is too big as the log is stored in memory of the web service before being delivered to the client.
      • It depends on the Java option -Xmx, which specifies the maximum size of the memory allocation pool.
      • JOC's Jetty is started with -Xmx512m per default which causes an OutOfMemoryError being raised for main logs larger than about 100MB.

      Desired Behavior

      • An OutOfMemoryError should not be raised for at least main logs <= 1GB.
      • The processing of logs for download is changed to make use of two separate API requests:
        • Info Request: the main log is requested from the Master and is stored to a temporary file. File size and meta data are returned when calling this API
        • Download Request: the temporary file is directly downloaded without any additional meta data.
      • The log file download should behave as any download and allow the client browser to display the progress of transfer.
      • Order and Task logs will be offered for download too with JOC-435.

      Maintainer Notes

      • The ./jobscheduler/log API requested first the JobScheduler main log at the JobScheduler.
      • At some point you reach other limits e.g. the web server of the JobScheduler doesn't respond to the JOC Cockpit web service because of a SocketTimeoutException if the log is larger than 1GB.

      Attachments

        Issue Links

          Activity

            People

              oh Oliver Haufe
              oh Oliver Haufe
              Mahendra Patidar Mahendra Patidar
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: