question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

SCP data transfer speed comparison with OpenSSH

See original GitHub issue

I was testing the performance of the SCP method from the sshj library for uploading a file from my machine to a remote one and I find it significantly slower compared to the OpenSSH’s scp command which I executed from my centos machine.

The tests which I performed were all done in the same environment (local network with 2 centos machines) and I was transferring 1 GB file to a remote machine.

  • The scp command gives me about 180 MB/s upload speed: scp testfile-1gb username@1.2.3.4:/tmp

  • When I tried to use the sshj’s scp to upload the same file it only gives me about 40 MB/s. Here is a simple java code I’m using:

import net.schmizz.sshj.SSHClient;
import net.schmizz.sshj.transport.verification.PromiscuousVerifier;
import java.io.IOException;

public class TestSshj {
    public static void main(String[] args) throws IOException {
        SSHClient ssh = new SSHClient();
        try {
            ssh.addHostKeyVerifier(new PromiscuousVerifier());
            ssh.connect("1.2.3.4");
            ssh.authPassword("username", "password");
            ssh.newSCPFileTransfer().upload("test-1gb", "/tmp");
        }
        finally {
            ssh.close();
        }
    }
}

I also tried using SFTP method for uploading the data to see if there is any difference but it gives pretty much the same slow-ish speed. Another thing I tried changing was the encryption algorithms (I tried using the same as OpenSSH was using) but that didn’t help either.

Is this kind of performance difference expected or can it be improved somehow?

Issue Analytics

  • State:open
  • Created 2 years ago
  • Reactions:4
  • Comments:22 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
simhrocommented, Dec 16, 2021

I’ve tried with the buffered stream as you’ve shown but I’m not getting any speed improvement. I was testing it by uploading a 5GB file.

1reaction
hierynomuscommented, Jun 9, 2021

If you use FileSystemFile, that uses direct FileInputStream and FileOutputStream. It might be worthwhile to wrap those in a Buffered(Input|Output)Stream before passing them

Read more comments on GitHub >

github_iconTop Results From Across the Web

Transferring large amount of data over the network: scp, tar
Conclusions. Scp is by far the slowest transfer method, 623% slower than the fastest case scenario. Contrary to the common conception that it's...
Read more >
Say No to scp - Fasterdata
DO NOT USE THESE TOOLS if you need to transfer large data sets across a network path with a RTT of more than...
Read more >
FTP/FTPS/SFTP/SCP - Speed comparison [closed] - Server Fault
My understanding is that PSC submitted the patches to the openbsd folks (who write openssh). They were not interested. I heard vague statements...
Read more >
File transfer speed : SCP vs Copy-Paste over SSH - Super User
Transfer using scp file.txt user@hostip:/tmp/file.txt -> Done almost instantly · Transfer by using copy and paste in nano over ssh -> Take almost ......
Read more >
scp speed limit? - Unix & Linux Stack Exchange
On that same everything, using secure copy scp which is using the SSH protocol, I typically see 80-90 MB/sec (according to the text...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found