Learn › Linux Foundations › Networking and Remote Access

Chapter 22, Part 2. Networking and Remote Access: ssh, ssh-keygen, ssh-keyscan and scp

Final test: Chapter 22, Networking and Remote Access. Twenty-five objectives on a real Debian 12 server covering both parts: ip address, ip link, ip route, getent, /etc/resolv.conf, ss and curl, then ssh to the account deploy over 127.0.0.1, host key fingerprints, known_hosts, ssh-keygen -F and -R, ssh-keyscan and scp. Fifty minutes, graded from what you leave behind and from the commands you run.

Chapter 22, Part 2. Networking and Remote Access

This chapter covers the following topics:

This is Part 2 of 2. Part 1 covered Sections 22.1 to 22.6, the tools that read how a server is connected. This part covers Sections 22.7 to 22.13: SSH, the way an administrator works on another machine, the keys that make it safe, and copying files with scp. It ends in the chapter lab, which covers all thirteen sections.

What you already know: Section 22.5 found sshd, the SSH server, listening on port 22 of this machine, and Section 7.5 switched to another account with su on the same machine. SSH reaches an account through a network connection instead, on this machine or on any other.

Objectives

After this part you can:

Start the shared practice machine from the panel before using these terminals. They run your commands on a real Debian server as admin@server1, and each step receives practice credit only after its requested command finishes with the expected result. The SSH sessions in this part are real: from admin on this machine to the account deploy on the same machine, over the loopback address 127.0.0.1. Fingerprints, keys, dates and transfer rates on your machine differ from the ones shown, and each step says which. The graded lab at the end runs on a real Debian 12 server. Follow its startup status in the workspace and check the current session time there. It covers both parts of this chapter: 25 objectives, graded from the files and the keys you leave behind and from the commands you run.

Pre-check 1 of 5: the two questions

Five questions before the material. A wrong answer costs nothing: the explanation states the fact and names the section that teaches it.

Pre-check 2 of 5: where accepted keys are kept

Pre-check 3 of 5: nested shells

Pre-check 4 of 5: a changed key

Pre-check 5 of 5: copying

22.7 SSH: host keys and user keys

SSH, the Secure Shell, gives you a shell on another machine over the network. Everything that crosses the network is encrypted: scrambled so that only the two ends can read it. The client is ssh, the program you type, and the server is sshd, the program that listens on port 22 (Exercise 22-12).

Every connection answers two questions, each with its own key. The client asks the first: is this the right server? The server proves who it is with its host key, a key pair that belongs to the machine. The server asks the second: may you log in as this account? The client proves who it is with a user key, a key pair that belongs to a person or an account. Neither answer implies the other: a server that accepts your key may still be the wrong server.

A key pair is two linked keys. The private key stays with its owner and signs; the public key may be given to anyone, and checks those signatures. A key pair proves who holds the private key without ever sending it.

This exercise uses a disposable practice key pair already prepared for your account on this VM. It is for loopback practice here, not a production administrator credential. Inspect its filenames, modes and public half; do not print, copy or reuse the private key on another machine. SSH keeps user keys in ~/.ssh. You are in your home directory. List it in long form:

prompt: admin@server1:~$ answer: ls -l ~/.ssh ||| ls -l ~/.ssh/ ||| ls -l /home/admin/.ssh output: total 8 -rw------- 1 admin admin 399 Sep 30 15:18 id_ed25519 -rw-r--r-- 1 admin admin 93 Sep 30 15:18 id_ed25519.pub hint: Section 22.7. The long listing from Section 2.4, then the directory ~/.ssh.

id_ed25519 is your private key: readable and writable by you alone, -rw------- (Section 2.4). ssh refuses to use a private key that other accounts can read. id_ed25519.pub is the public key, readable by everyone. ED25519 is the kind of key, a modern signature method. The dates depend on when this VM's key files were created, so yours may differ; the sizes are the same for every key of this kind and comment.

History. Tatu Ylönen wrote the first SSH in 1995 at the Helsinki University of Technology, after passwords were captured on the university's network. OpenSSH began in 1999 as a project of the OpenBSD operating system, built from the last freely licensed release of that program, and it is the SSH Debian ships. The version of the protocol in use today, SSH-2, was published in January 2006 as RFC 4250 to RFC 4256, the documents that define internet protocols.

22.7 The server's key pair

The server keeps its host keys in /etc/ssh, one pair for each kind of key. You are in your home directory. List the ED25519 pair in long form:

prompt: admin@server1:~$ answer: ls -l /etc/ssh/ssh_host_ed25519_key /etc/ssh/ssh_host_ed25519_key.pub output: -rw------- 1 root root 411 Sep 30 15:17 /etc/ssh/ssh_host_ed25519_key -rw-r--r-- 1 root root 96 Sep 30 15:17 /etc/ssh/ssh_host_ed25519_key.pub hint: Section 22.7. The long listing, then the two files /etc/ssh/ssh_host_ed25519_key and /etc/ssh/ssh_host_ed25519_key.pub.

The same pattern, owned by root: the private host key is readable by root alone, and the public one by everyone. A client never needs the private key; it checks the server against the public one. Your dates and sizes differ: the pair was made when this machine was created.

22.7 A fingerprint: ssh-keygen -lf

A public key is a long string. A fingerprint is a short value calculated from it with a hash function, a calculation that gives a different result for a different input, so comparing fingerprints compares keys. ssh-keygen makes and reads keys: -l prints a fingerprint, and -f names the key file.

The right fingerprint has to come from somewhere other than the connection being checked. Reading it on the server itself, in a terminal already connected to it, is such a trusted channel. On a real server that is its console or its administrator's record; here it is this browser terminal. You are in your home directory. Print the fingerprint of the server's ED25519 host key:

prompt: admin@server1:~$ answer: ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub ||| ssh-keygen -l -f /etc/ssh/ssh_host_ed25519_key.pub output: 256 SHA256:pEVmCIYQ6pV5lqYAdWao9Ua1Xa0Lcb/7YEceMPcVjvw root@localhost (ED25519) hint: Section 22.7. ssh-keygen with -l and -f, then the public key file /etc/ssh/ssh_host_ed25519_key.pub.

Four fields: the key size in bits, 256; the fingerprint, SHA256: and 43 characters; the key's comment, which names where the key was made; and the kind of key in brackets. Your fingerprint differs: every machine has its own host key. Keep it on the screen, because Section 22.8 compares it with what ssh shows.

22.8 First contact: ssh

ssh USER@HOST opens a session as the account USER on the machine HOST. This server has a second account for this chapter, deploy, which accepts SSH logins from this machine only, over the loopback address; Section 22.13 shows the rule.

The first time ssh meets a host, ~/.ssh/known_hosts, the file of known hosts, has no entry for it. ssh cannot tell by itself whether the host is the right one, so it shows the fingerprint of the key the host sent and asks. Compare it with the one from Exercise 22-19. If the two are equal, type yes and press Enter. You are in your home directory. Open a session as deploy on 127.0.0.1:

prompt: admin@server1:~$ answer: ssh [email protected] output: The authenticity of host '127.0.0.1 (127.0.0.1)' can't be established. ED25519 key fingerprint is SHA256:pEVmCIYQ6pV5lqYAdWao9Ua1Xa0Lcb/7YEceMPcVjvw. This key is not known by any other names. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '127.0.0.1' (ED25519) to the list of known hosts. ... hint: Section 22.8. ssh, the account deploy, an at sign and the address 127.0.0.1; answer yes after comparing the fingerprints.

Line 1: ssh has no record of 127.0.0.1, so it cannot tell whether this is the host you mean. Line 2: the fingerprint of the key the host sent; on your machine it equals the one from Exercise 22-19, character for character. Line 3: no other host name you connected to uses this key. Line 4: the question. yes trusts the key, no ends the connection, and pasting the fingerprint trusts it only if it matches.

Line 5: ssh stored the key in ~/.ssh/known_hosts and checks every later connection to 127.0.0.1 against it. The lines trimmed here as ... are the server's message of the day, then deploy's prompt, deploy@server1:~$.

Typing yes without comparing turns the check into a formality. Over a real network, a different fingerprint at this question means another machine answered, and the right answer is no.

22.9 Inside the session: whoami

The prompt now reads deploy@server1:~$: you type into a shell of the account deploy, on server1, in deploy's home directory. Your own shell, admin's, still runs underneath: it started ssh and waits for it to end. Shells started inside other shells are nested, and the prompt is the one sign of which shell you type into. Inside the session, print the account this shell runs as:

prompt: deploy@server1:~$ answer: whoami output: deploy hint: Section 22.9. Inside the session, the identity command of Section 1.4, with no options.

deploy. Section 1.4 printed admin with the same command. In this shell the files, the history and the permissions are deploy's. After you return to admin's shell, one command given to ssh can print the same word (Section 22.10); the chapter lab accepts either form. Here, at deploy's prompt, type whoami.

22.9 The machine: hostname

You are inside the session as deploy. Print the name of the machine:

prompt: deploy@server1:~$ answer: hostname output: server1 hint: Section 22.9. Inside the session, the command of Section 1.5 that prints the machine's name.

server1, the same machine: the connection left through lo and came back in to sshd on this server, so this is a different account on the same machine. Over a real network this command prints the other machine's name, which is how you confirm where a session landed.

22.9 Leaving the session: exit

exit ends the shell you type into (Section 1.13). In the session that ends the session and ssh with it, and the shell that started ssh prompts again. Type it once: a second exit would end your own shell, the browser terminal, as well. You are inside the session as deploy. Leave it:

prompt: deploy@server1:~$ answer: exit ||| logout output: hint: Section 22.9. The command of Section 1.13 that ends the shell you type into, typed once.

Two lines appear. deploy's shell prints logout, because an SSH session starts a login shell, and ssh reports that the connection to 127.0.0.1 closed. The prompt reads admin@server1:~$ again.

22.10 One command over SSH

A command after the host runs on the other side in place of a shell, and ssh ends when the command does. The output comes back to your terminal; no prompt of the other account ever appears. You are in your home directory. Print deploy's identity record:

prompt: admin@server1:~$ answer: ssh [email protected] id output: uid=950(deploy) gid=950(deploy) groups=950(deploy) hint: Section 22.10. ssh, then [email protected], then the identity command id on the same line.

deploy is user ID 950, with a group of its own, also 950; Section 7.2 read the same kind of record for admin. The record came from the other side of the connection, where id ran as deploy. The prompt never changed, because no session was opened.

22.10 Where the command runs

You are in your home directory. Print the working directory of a command run as deploy:

prompt: admin@server1:~$ answer: ssh [email protected] pwd output: /home/deploy hint: Section 22.10. The same form as Exercise 22-24, with pwd as the command.

/home/deploy: a command given to ssh starts in the home directory of the account it runs as, the directory the session prompt showed as ~. A file name given to such a command is read from there, which Section 22.12 uses.

22.11 Known hosts: ssh-keygen -F

~/.ssh/known_hosts holds one line for each host key you accepted. Debian's ssh settings hash the host name in each line: they store a one-way value calculated from the name instead of the name, so the file does not list the machines you connect to. ssh-keygen -F HOST finds the line for a host anyway. You are in your home directory. Look up 127.0.0.1:

prompt: admin@server1:~$ answer: ssh-keygen -F 127.0.0.1 ||| ssh-keygen -F 127.0.0.1 -f ~/.ssh/known_hosts output:

Host 127.0.0.1 found: line 1

1HfWq991uRSQHLKcLKCUoDQiw4So=9KJ0gjzXgvE6zhchE77rCy2PT8E= ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFwo8/y0PGzC25Q3Cm2xgF83d1upTzLdR+XNJfwVftA8

hint: Section 22.11. ssh-keygen with -F, then the host 127.0.0.1.

The comment line gives the line number that matched. The line itself holds the hashed name, starting with |1|, the kind of key, and the public key: the same key the host sent at first contact. Your line differs: the hash is salted, mixed with random bytes each time it is written, so no two machines store the same text.

22.11 The scripted check: ssh-keyscan

A script cannot answer yes. ssh-keyscan connects to a host, asks for its host key, prints it as a known hosts line and leaves; -t ed25519 asks for one kind of key. It proves nothing by itself: it prints whatever answered. You are in your home directory. Print the ED25519 key that 127.0.0.1 offers:

prompt: admin@server1:~$ answer: ssh-keyscan -t ed25519 127.0.0.1 output:

127.0.0.1:22 SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u9

127.0.0.1 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFwo8/y0PGzC25Q3Cm2xgF83d1upTzLdR+XNJfwVftA8 hint: Section 22.11. ssh-keyscan, the option -t with the kind ed25519, then the host 127.0.0.1.

The comment line names the host, the port and the version of the SSH server that answered; your version string may end differently. The second line is a known hosts line, ready to add. A script that must trust a host compares the fingerprint of that key with one obtained through a trusted channel, and only then appends the line with >> (Section 11.4):

ssh-keyscan -t ed25519 127.0.0.1 >> ~/.ssh/known_hosts

Your known_hosts already holds this key from Section 22.8. The line above would add a second, unhashed entry for the same key, which ssh reads the same way.

22.11 A stored key that no longer matches

~/remote/known_hosts.stale is a known hosts file prepared for this chapter. Its entry for 127.0.0.1 holds a different key, as a known hosts file does after a server's host key was replaced. ssh-keygen -lf prints a fingerprint for every line of a known hosts file. You are in your home directory. Print the fingerprints in the prepared file:

prompt: admin@server1:~$ answer: ssh-keygen -lf ~/remote/known_hosts.stale ||| ssh-keygen -l -f ~/remote/known_hosts.stale output: 256 SHA256:QgyHbstoRlf+1TjfDq3Qd42DvgHy3PgEQO6ugxEzvPU 127.0.0.1 (ED25519) 256 SHA256:Bn33syZyomDt1Xqc6KHd1GlHPly45v1hhAT1ozjFB0w backup1,10.0.0.5 (ED25519) 256 SHA256:P1wfrjM0eAlZkS1sVSlLL17YdlHD+lIOarPtAHPeRb4 db1,10.0.0.10 (ED25519) hint: Section 22.11. ssh-keygen with -l and -f, then the file ~/remote/known_hosts.stale.

Three entries, each with its fingerprint, the names it is stored under and its kind: 127.0.0.1, then backup1 and db1, each written with its name and its address. The fingerprint stored for 127.0.0.1 is not the one Exercise 22-19 read on the server, so the stored key is not the key this server has. These three lines are the same on every machine.

22.11 The changed-key warning

-o sets one ssh option for one command. UserKnownHostsFile names the known hosts file ssh reads and writes, so this connection checks the host against the prepared file and leaves your own ~/.ssh/known_hosts alone. Yo

Practice Chapter 22, Part 2. Networking and Remote Access: ssh, ssh-keygen, ssh-keyscan and scp in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.

More lessons in Networking and Remote Access