YADE SFTP - auto detect shell

Current Situation

• The YADE SFTP provider automatically executes a remote command (uname) after connecting to detect the shell type and environment.
  • This is useful to have more detailed information about the server and when YADE Pre- or Post-Processing is used.
• If only SFTP is allowed, the server normally responds with a channel failure message, which is handled gracefully.
• However, some SFTP servers immediately close the entire connection when an exec command is attempted, which causes the entire YADE transfer to fail.

Desired Behavior

• Automatically reconnect if the connection was closed due to an attempted remote command execution at the beginning of the transfer.
• Introduce a new configuration parameter ProtocolFragments/SFTPFragment/DisableAutoDetectShell that,
  • when set to true, skips automatic shell detection while still relying on server identity for OS detection.
    • This prevents unnecessary exec calls and possible reconnects on SFTP-only servers, ensuring stable transfers.

Patch

• A cumulative patch is available for JS7 versions 2.8.0 and 2.8.1.
• The patch includes the following fixes:
  • YADE-631 - Renaming source files during transfer
  • YADE-632 - YADE SFTP - auto detect shell
    • Note:
      • The patch does not yet support setting the new configuration item DisableAutoDetectShell via JOC Cockpit - CONFIGURATION - File Transfer GUI.
      • However, it can be added manually as the last element in an SFTPFragment, for example:

      • <ProtocolFragments>
            <SFTPFragment name="sftp">
                <BasicConnection> ...
                <SSHAuthentication> ....
                ...
                <DisableAutoDetectShell>true</DisableAutoDetectShell>
            </SFTPFragment>
        

      • Manual changes can be safely applied for transfers that rely on an XML file stored on disk, without using the JobResource mechanism.
        • e.g.: -settings=/<some path>/<some yade settings>.xml
      • Note: Configuring this directly in a JobResource is possible, but changes will be lost if the JobResource is redeployed through the File Transfer GUI.
  • YADE-633 - YADE Pre/Post Processing - custom ProcessingCommandDelimiter is ignored
  • JITL-761 - SOSCommandResult - getExitCode handling
• Download
  • Download the patch here: js7_agent.2.8-PATCH.YADE-631_YADE-632_YADE-633_JITL-761.jar [ sha256 ] [ sig ] [ tsr ]
• Applying the patch
  • Since YADE-626, YADE can be executed not only as a standalone Shell Job, but also as an integrated Java JITL Job.
  • This means the patch can/should be deployed in two locations:
    • <agent install>/lib/patches - for YADE configured as a JITL Job (com.sos.jitl.jobs.yade.YADEJob)
      • See JS7 - Patches for Agent
      • Note: The Agent instance has to be restarted to apply patch.
    • <agent install>/yade/lib/patches - for YADE configured as a Shell Job
      • Note: The Agent instance does not need to be restarted to apply the patch

Test Instructions

• Automatic reconnect:
  • Configure an SFTP server to allow SFTP only.
  • Note: This behavior may be difficult to observe in practice, as most common SFTP servers (e.g., OpenSSH) do not close the connection when only the SFTP subsystem is supported.
    • The CompleteFTP Server can be used to reproduce the scenario.
      • After installation, make sure that "Site - SFTP/SCP - SSH terminal access enabled" is not checked.
    • If the connection is closed, the automatic reconnect can be observed at the DEBUG log level.
  • Expected Behavior: regardless of whether the connection was closed or not, the YADE transfer should not fail due to the execution of the uname command.
• DisableAutoDetectShell configuration:
  • Note: Patch testing
    • Export an existing XML configuration from the File Transfer GUI.
    • Configure YADE job to use this settings file instead of a JobResource.
  • Configure DisableAutoDetectShell and run YADE with the DEBUG log level.
  • The DEBUG logs should clearly indicate whether uname remote exec command was executed or skipped.