Resume a large scp transfer

December 18, 2006

Today I was tyring to transfer a huge file over a ssh connection. Everything was working fine, but at 94% the connection stalled and the transfer has been interrupted. $#!”&%, I thought, but luckily joen.dk helped me a lot!

If you need to resume an scp transfer, try with rsync:

rsync --partial --progress --rsh=ssh user@host:remote_file local_file

Advertisements

31 Responses to “Resume a large scp transfer”

  1. svrmarty Says:

    please use
    –partial –progress

  2. svrmarty Says:

    —partial —progress

  3. svrmarty Says:


    --partial --progress

  4. svrmarty Says:

    “–partial –progress”

  5. andrewmcdonough Says:

    Excellent advice. For some reason, scp stalls on me occasionally. Using rsync instead solved the problem.

    —-
    http://www.andrewmcdonough.com


  6. […] via Resume a large scp transfer « Yes, yet another blog…. […]


  7. Thank you! Didn’t want to throw away 1.3GB of my 5.2GB file transfer just because it stalled and timed out. Worked great! 🙂


  8. […] Fonte: https://yyab.wordpress.com/2006/12/18/resume-a-large-scp-transfer/ […]


  9. Isn’t working for me do I need the FULL path of both host location AND local path?

    Will it be an issue if the SCP session is currently downloading? What if the host file is in the root directory how do I write that out?

    Sorry very basic questions. Not able to get this to work.


  10. @zero day: you’ll need the relative path of the local file, and the path of the remote file relative to the home directory. Absolute paths will work too.

    Yes, you’ll need to stop scp from downloading before you start rsync, as they’ll both want to write to the same file. (If scp is working for you, though, you may as well leave it running and only use rsync if scp fails.)

    If the file is on the root directory of the host, just put a slash at the beginning (ie specify its absolute path), for example “/myfile.tgz”.


  11. Thank you! You save my day!


  12. Thanks for this great alternative to scp!

  13. Josh Enders Says:

    Perfect. Thanks!


  14. doesn’t it require rsync to be installed on the other side?

  15. Ben Arons Says:

    brilliant suggestion. thanks!


  16. This is the second time I’ve come across this page after I ran into a dropped connection on a large SCP file. Easier to google than remember!

    Yes, it does need rsync on both sides of the connection – often it will be faster to install rsync than to copy the whole file again!


  17. This saved me 6 hours! Thank you 🙂


  18. wondering if you can enter options for the file transfer (for example, if I want to decend down a directory for copying, rather than single file)?


  19. […] and scp leaves you with a broken file. Having encountered this problem, I was lucky enough to find this post which explains a remedy: Use rsync in order to complete the interrupted […]

  20. balrog Says:

    Thanks, great tip. I notice though that if rsync gets killed during an upload, or your system dies in the middle of a transfer, it will not write what it uploaded and on resume will start from the point it started last time.

    Notice the argument order is the same as with SCP and since I was uploading rather than downloading, I got it wrong and truncated my local file, fortunately was easy to regenerate 🙂


  21. Thank you! Very helpful!


  22. Good to know, thank you !


  23. This is a slam dunk buddy — thanks for this! 🙂


  24. Thank you a lot 🙂

  25. fierceunknown Says:

    woot!

  26. squaregoldfish Says:

    Nearly 10 years on, and it’s still useful 🙂


  27. This is perfect – if you are transferring the remote file to your local host. If you do it the other way around without switching the arguments you destroy the file…


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: