Uploaded image for project: 'YADE - Yet Another Data Exchange Tool'
  1. YADE - Yet Another Data Exchange Tool
  2. YADE-374

YADE JITL does not give priority to the JobScheduler Order parameter

    XMLWordPrintable

Details

    • Fix
    • Status: Released (View Workflow)
    • Minor
    • Resolution: Fixed
    • 1.10
    • 1.10.1
    • None
    • None

    Description

      As of now situation

      • Due to XML editor's configuration checking rules ( by design ) each profile should have atleast file_spec/file_path/file_list parameter set.
      • In some cases its not possible to know the file_spec or file_path e.g. Directory monitoring by the JobScheduler.
      • The YADE is being called from JobScheduler via JITL Job.
      • The YADE uses a basic profile with source, target, jump host, where file_path parameter is dynamically set from JobScheduler.
      • From JobScheduler JITL job its possible to set YADE parameters from JobScheduler's Order parameter e.g. JADE's file_path =SCHEDULER_FILE_PATH.
      • Profile
        [sg_outbound]
        operation                           = copytointernet
        
        source_protocol                     = local
        file_path                           = FROM_FILE_ORDER_SOURCE
        check_steady_state_of_files         = true
        check_steady_state_interval         = 1
        steady_state_count                  = 30
        
        jump_include                        = protocol_fragment_jump@jump-gateway
        
        target_include                      = protocol_fragment_sftp@sftp-sg
        target_dir                          = FROM_JOBSCHEDULER
        
        mandator                            = SG
        
      • As defined in the above mentioned profile, when JobScheduler JITL job set the parameter file_path with value of Order parameter SCHEDULER_FILE_PATH, one can see from the log file that parameter replacement indeed work as expected but in the end the parameter value from jade_settings.ini get the preference.
      • JITL Job
        <job order="yes" stop_on_error="no" title="JITL Job for JobScheduler Advanced Data Exchange">
          <settings/>
            <description>
             <include file="jobs/jade4DMZJob.xml"/>
           </description>
        <params>
          <param name="settings" value="/mnt/r2d2/backup/sos/JADE-Demo-Examples/sos/jade_settings.ini"/>
          <param name="profile" value="sg_outbound"/>
          <param name="target_dir" value="/home/tokyo/data/sos/sg/in"/>
          <param name="file_path" value="\${scheduler_file_path}"/>
        </params>
        <script java_class="sos.scheduler.jade.Jade4DMZJob" language="java"/>
        <monitor name="configuration_monitor" ordering="0">
        <script java_class="sos.scheduler.managed.configuration.ConfigurationOrderMonitor" java_class_path="" language="java"/>
        </monitor>
        <run_time/>
        </job>
        
      • Error
        2015-10-05 16:31:35.293+0200 [info]   (Task SOS/OUTBOUND/SG/jade_file_transfer:6107166)  INFO [main] (SOSDataExchangeEngine.java:392) - 
        2015-10-05 16:31:35.293+0200 [info]   (Task SOS/OUTBOUND/SG/jade_file_transfer:6107166) ************************************************************************* 
        2015-10-05 16:31:35.293+0200 [info]   (Task SOS/OUTBOUND/SG/jade_file_transfer:6107166)  execution status     = failure. Errors reported. 
        2015-10-05 16:31:35.293+0200 [info]   (Task SOS/OUTBOUND/SG/jade_file_transfer:6107166)  successful transfers = 0 
        2015-10-05 16:31:35.293+0200 [info]   (Task SOS/OUTBOUND/SG/jade_file_transfer:6107166)  skipped transfers    = 0 
        2015-10-05 16:31:35.293+0200 [info]   (Task SOS/OUTBOUND/SG/jade_file_transfer:6107166)  failed transfers     = 0 
        2015-10-05 16:31:35.293+0200 [info]   (Task SOS/OUTBOUND/SG/jade_file_transfer:6107166)  last error           = SOSJADE_E_0099: No files found for file_path 'FROM_FILE_ORDER_SOURCE'
        2015-10-05 16:31:35.293+0200 [info]   (Task SOS/OUTBOUND/SG/jade_file_transfer:6107166)  
        2015-10-05 16:31:35.293+0200 [info]   (Task SOS/OUTBOUND/SG/jade_file_transfer:6107166) *************************************************************************
        
      • If YADE is called from the Command line and parameter value is being overwritten with a command line parameter , it did work as expected e.g. parameter value from command line has precedence over parameter value from jade_settings.ini.

      How to reproduce

      • JobScheduler
        • create a empty file at /home/jenkins/data/sos/vol1/SFTP/data/out/SG
        • Check the Gollum.sos:4410 jobscheduler for error message
      • YADE CLI
         
        /home/jenkins/sos-berlin.com/jade/jade_client.1.10.0/bin/jade4dmz.sh -settings="/mnt/r2d2/backup/sos/JADE-Demo-Examples/sos/jade_settings.ini" -profile="sg_outbound" -file_path="/home/jenkins/data/sos/vol1/SFTP/data/out/abc123.txt"  -target_dir="/home/tokyo/data/sos/sg/in"
        

      Desired behavior

      • The Order parameter should have precedence over parameters defined in the jade_settings.ini

      Attachments

        Activity

          People

            oh Oliver Haufe
            mp Mahendra Patidar
            Mahendra Patidar Mahendra Patidar
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 1 day
                1d
                Remaining:
                Remaining Estimate - 1 day
                1d
                Logged:
                Time Spent - Not Specified
                Not Specified