Uploaded image for project: 'YADE - Yet Another Data Exchange Tool'
  1. YADE - Yet Another Data Exchange Tool
  2. YADE-402

YADE should perform parallel file transfers

    XMLWordPrintable

Details

    • Feature
    • Status: Deferred (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • None
    • YADE Client
    • None

    Description

      Current Situation

      • When YADE transfer profile is configured to do parallel file transfers using ConcurrentTransfer=true and MaxConcurrentTransfers=4
      • From FTP server's log one can see YADE create 5 sessions, but transfer only 1 file at time
      • This behavior is independent of source/target protocol being used.

      Test Setup

      • YADE Settings
        [globals]
        history                             = /home/test/sos-berlin.com/jade_client/jade_releases/jade_client.1.10/logs/jade_history.csv
        verbose                             = 0
        
        
        [protocol_fragment_sftp@sftp_homer]
        protocol                            = sftp
        
        host                                = homer.sos
        user                                = test
        ssh_auth_method                     = publickey
        ssh_auth_file                       = /home/test/.ssh/id_rsa
        proxy_protocol                      = http
        proxy_host                          = homer.sos
        proxy_port                          = 3128
        proxy_user                          = proxy_user
        proxy_password                      = 12345
        strict_host_key_checking            = false
        
        [01_05_Local2SFTPFileSizeMFiles010ConcurrentTransfer02]
        operation                           = copy
        
        source_protocol                     = local
        file_spec                           = ^TEST-M-[0-9]{3}.TXT$
        source_dir                          = /home/test/data/from_galadriel
        
        target_include                      = protocol_fragment_sftp@sftp_homer
        target_dir                          = /home/test/data/to_homer/
        
        Concurrent_Transfer                 = true
        Max_Concurrent_Transfers            = 4
        

      Desired Behavior

      • YADE should transfer multiple files in parallel when ConcurrentTransfer=true and MaxConcurrentTransfers=4 ( MaxConcurrentTransfers parameter can be any integer)

      Attachments

        1. YADE-402.ini
          1 kB
        2. YADE-402.xml
          3 kB

        Issue Links

          Activity

            People

              re Robert Ehrlich
              mp Mahendra Patidar
              Mahendra Patidar Mahendra Patidar
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: