Lesson 7.4: Transferring files: wget, curl, scp and rsync

Downloading from the web with wget and curl and checking the download, logging in to another machine with SSH (host keys, known_hosts, passwords), copying files to and from it with scp, and keeping two directories in sync with rsync, locally and over SSH.

22 min

You will learn to

Download a file with wget and curl and verify it with sha256sum
Log in to a remote machine with ssh and explain the host key question
Run a single command on a remote machine
Copy files in both directions with scp
Synchronise directories with rsync and explain the trailing slash

Key terms

URL: Uniform Resource Locator: scheme, host and path of a resource, e.g. http://downloads.lhm.example/lhm-scan/LATEST.
wget: A non-interactive downloader from the GNU project. Saves to a file named after the URL; -O NAME chooses the name, -q is quiet, -c continues an interrupted download.
curl: A tool for transferring data with URLs, in many protocols. Prints to standard output by default; -o NAME or -O saves, -I shows only headers, -s is silent, -L follows redirects.
SSH: Secure Shell: an encrypted protocol (port 22) for logging in to another machine and running commands there. ssh is the client, sshd the server.
Host key and fingerprint: Every SSH server has a key pair that proves its identity. The fingerprint is a short hash of its public key; on first contact you are asked to confirm it.
~/.ssh/known_hosts: The list of host keys you have accepted. ssh checks every connection against it.
Key-based login: Instead of a password you can log in with your own key pair: ssh-keygen creates it, ssh-copy-id installs the public half in the server's ~/.ssh/authorized_keys.
scp: Secure copy over SSH: scp SOURCE... DEST, where a remote side is user@host:path. -r copies directories, -P sets the port.
rsync: Synchronises files locally or over SSH, sending only the differences. -a archive mode, -v verbose, -n dry run (show, don't copy), --delete removes files that no longer exist in the source.

← Networking from the command line · All lessons · Converting text files: line endings and encodings →