Idle timeout is set for HTTP connections between Master and Agent

Current Situation

• Communication to and from the Agent goes over an HTTP connection which is established by the Master.
• Every HTTP connection starts a TCP connection on the Agent side.
• For the sake of reuse, the HTTP tunnel is kept and the TCP connection is kept open.
• Depending on firewall settings, it can occur that idle TCP connections are dropped after some time. The following error is thrown on the Master side:

  SCHEDULER-488  This remote JobScheduler 'Agent(0,http://RemoteServer:4445,None)' is not accessible: spray.can.Http$ConnectionException: ErrorClosed(An existing connection was forcibly closed by the remote host) 

Desired Behavior

• A fixed timeout for idle HTTP connections between Master and Agent will be set to 10 minutes. The corresponding TCP connection will be removed on the Agent side after 10 minutes without communication between Master and Agent.
• HTTP connections between Master and Agent will still be reused if they take place within the 10 minutes.

Delimitations

• This is a fixed timeout, there is no way to configure the timeout dynamically.
• Consequence: Should a user want to change the default setting http_heartbeat_timeout="60" for a process class, there is a maximum for the setting, that is 300 (5 minutes).