Learn › The Network Stack › The Application Layer
curl -v - a hands-on Linux lab on a real virtual machine.
Request line, headers, status code, body. The protocol you use a thousand times a day, seen raw.
Every web page you have ever loaded started with an HTTP request. Your browser sent a message that said GET / HTTP/1.1 and the server sent back a status code, some headers, and the page content. This happens thousands of times a day, completely hidden behind the browser interface.
It is also plain text. Not a binary format, not a protocol you need a special tool to decode. Lines of readable text, sent over a TCP connection, that you could type by hand if you had to.
This lesson strips away the browser. You will send HTTP requests yourself and read every line of the conversation, then capture the same exchange on the wire and watch it ride the stack you have spent this whole track building.
The black boxes below are a practice terminal: a safe sandbox that checks the one command each step teaches. The lab user is student and the machine is named http-lab. Values that change on every run, such as tool versions, timestamps and port numbers, are printed as labels rather than numbers, because your run will produce its own.
HTTP stands for HyperText Transfer Protocol. It is the protocol that powers the web. When your browser loads a page, it sends an HTTP request to a server, and the server sends back an HTTP response.
An HTTP request has three parts:
1. Request line - The method (GET, POST, PUT, DELETE), the path (/index.html), and the HTTP version. 2. Headers - Key-value pairs that carry metadata. Host, User-Agent, Accept, Content-Type. 3. Body - Optional data sent with the request (used in POST and PUT).
An HTTP response also has three parts:
1. Status line - The HTTP version, the status code (200, 404, 500), and a reason phrase. 2. Headers - Metadata about the response. Content-Type, Content-Length, Server. 3. Body - The actual content. HTML, JSON, an image, a file.
Three parts each way, and the shape never changes. Once you can see that shape on a screen, every web API, every browser developer tools panel and every load balancer log reads the same way.
The first version of HTTP, written by Tim Berners-Lee around 1991 and known afterwards as HTTP/0.9, was one line long. The client sent GET /page and the server sent back the document. No headers. No status codes. No version number, because there was only one version. When the document ended, the server closed the connection, and that was how you knew it had finished.
Status codes and headers arrived with HTTP/1.0 in 1996, and persistent connections with HTTP/1.1 the year after. Everything you are about to read on screen, the Host: line, the 200, the Content-Length, is a patch on that original one-line idea.
The lab server you are about to talk to speaks HTTP/1.0, not 1.1. That is not a mistake in the lab. It is a small, very common Python web server. Watching a modern client and an old server agree on a version is one of the more useful things on the screen you are about to read.
This lab builds two machines inside your one VM using network namespaces. A network namespace is a private copy of the network stack: its own interfaces, its own addresses, its own routing table. They are joined by a virtual cable:
[client] 10.0.1.10 <--------> 10.0.1.20 [server]The 10.0.1.0/24 network exists only inside those namespaces, so a command typed plainly in your shell cannot reach the server. To run something inside the client namespace, put sudo ip netns exec client in front of it. The send-curl and cap helpers already do that.
The server runs an HTTP service on port 80. It serves two files:
/index.html - Contains "Hello from the server"/data.json - Contains {"status":"ok","message":"The Linux Camp"}These command lines get long, so lean on tab completion. Type send- and press Tab and the shell finishes send-curl. Tab completes command names and file paths alike, and it is the cheapest way to stop a typo halfway through a long line.
Your progress is tracked automatically. Just type the commands naturally and you will see a checkmark appear when each task completes.
curl is a command-line tool that sends HTTP requests. The -v flag (verbose) shows you everything: what curl sent and what the server sent back. curl will announce itself as HTTP/1.1, because that is what curl speaks.
>>> Whatever the server speaks. The version in a request line is a statement of what the client understands, not an instruction. The server answers in a version it can actually speak, and if that is older than the client asked for, the client copes. This server answers in HTTP/1.0 even though curl asked in 1.1, because this simple Python server only speaks 1.0. If you picked HTTP/1.1, that is the reasonable assumption, and it is worth unlearning here. A version mismatch you did not expect explains a whole class of odd behaviour, starting with connections that close after every single request. If you picked no version, both directions carry one, and the reply puts it first: the status line begins with the version, then the code, then the reason phrase.
The helper runs sudo ip netns exec client curl -v http://10.0.1.20/ for you.
The output is laid out as shown below. The exact curl and Python version strings depend on what is installed, and the Date header is the clock on the server at the moment you asked.
send-curl
prompt: student@http-lab:~$ answer: send-curl ||| sudo ip netns exec client curl -v http://10.0.1.20/ output:
> GET / HTTP/1.1 > Host: 10.0.1.20 > User-Agent: curl/<version> > Accept: */* > < HTTP/1.0 200 OK < Server: SimpleHTTP/0.6 Python/<version> < Date: <day>, <DD Mon YYYY HH:MM:SS> GMT < Content-type: text/html < Content-Length: 22 < Hello from the server hint: The helper that sends a verbose request from the client namespace is send-curl, or type the long form: sudo ip netns exec client curl -v http://10.0.1.20/
There is the whole conversation, and the version question is settled on the first < line: HTTP/1.0 200 OK.
Look at the left edge before you read anything else. Every line starts with one of three characters, and those characters are curl telling you who said what. That is the single most useful habit in this entire lesson, because it is the difference between reading a protocol and reading a wall of text.
Making this request and reading what came back is the first of the five things your lab grades.
The body of that response reads Hello from the server. Count the characters and you get 21. The server said Content-Length: 22.
>>> Neither is wrong. Content-Length: 22 is the size of the body in bytes: the 21 characters of "Hello from the server" plus the newline at the end of the file. Text files on Unix conventionally end with a newline, and a newline is one byte like any other. If you picked the header, be careful with that instinct in this specific field: Content-Length has to be exact, because the client uses it to know when the body has finished. A wrong value there does not produce a warning, it produces a hang or a truncated page. If you counted 22 characters, count again with the newline in mind. The lesson is that a byte count and a character count are different questions, and HTTP always asks the first one.
Now walk the whole screen in order:
* are curl talking to you about the connection. They are not part of the HTTP conversation.> are what curl sent (the request).< are what the server sent back (the response).GET / HTTP/1.1 is the request line. GET is the method, / is the path, HTTP/1.1 is the version.Host: 10.0.1.20 is a request header. It tells the server which site the client wants, which matters when one server hosts many.HTTP/1.0 200 OK is the status line. 200 means success. The server answers in HTTP/1.0 even though curl asked in 1.1, because this simple Python server only speaks 1.0.Content-type: text/html is a response header. It tells the client what kind of data is in the body.Content-Length: 22 is the size of the body in bytes: the 21 characters of "Hello from the server" plus the newline at the end of the file.Hello from the server is the response body. This is the actual content.The bare > and the bare < are worth a second look. Each one is a blank line, and a blank line is how HTTP says the headers are finished and the body starts. It is not decoration and it is not optional.
Now capture that request from the network side. Order matters: the whole exchange is over in milliseconds, so the capture has to be listening before the request goes out. This waits three seconds in the background while tshark gets going, then sends the request:
(sleep 3; send-curl) &Now start the capture in that window. -a duration:8 stops the capture after eight seconds and hands your prompt back. -Y http is a display filter: capture everything, show only HTTP.
Before you run it, commit to a number: how many packets did that exchange take, and how many of them will this filter show you? Packet numbers and timestamps will be whatever your run produces.
cap -a duration:8 -Y http
prompt: student@http-lab:~$ answer: cap -a duration:8 -Y http ||| cap -Y http -a duration:8 output: 1 0.000123 10.0.1.10 -> 10.0.1.20 HTTP GET / HTTP/1.1 2 0.001234 10.0.1.20 -> 10.0.1.10 HTTP HTTP/1.0 200 OK (text/html) hint: The cap helper runs tshark inside the client namespace: cap -a duration:8 -Y http
tshark shows you the HTTP layer decoded. The request method (GET), the path (/), the response status (200 OK), and even the content type. This is HTTP riding on TCP, riding on IP, riding on Ethernet. Every layer you have seen in this track, working together.
Compare the two screens. curl showed you eleven lines of conversation. tshark shows the same conversation as two packets, because everything curl printed as separate lines travelled together inside one packet each way. Capturing it is the second of the five things your lab grades.
Only two lines, for a request that actually took around ten packets. The display filter hid the TCP handshake, the acknowledgements and the teardown, because none of those carry HTTP. Drop -Y http and you see all of it.
Summary lines are for reading. When you want values to feed to another program, -T fields prints field values instead, and -e names each field you want. Ask for two fields, the request method and the response code, and two lines come back: one for the request packet and one for the response packet.
>>> It is the empty first field. One line comes out per packet, with the requested fields separated by a tab, in the order you named them. The first line has the request method (GET) and an empty response code, because a request has no status. The second line is the reverse: no method, and the status code 200. That leading gap on line two is the empty first field. If you picked indentation, that reading is worth correcting early, because it will cost you when you feed this into a script: the columns are positional, and an empty column still takes its place. If you picked a bug, this is exactly how any tabular output has to behave. A missing value cannot shift the columns left, or nothing downstream could tell which column it was reading.
Start the traffic first, exactly as before:
(sleep 3; send-curl) &Then ask for the two fields by name. This is the form you reach for in scripts, because the output is columns rather than prose.
cap -a duration:8 -Y http -T fields -e http.request.method -e http.response.code
prompt: student@http-lab:~$ answer: cap -a duration:8 -Y http -T fields -e http.request.method -e http.response.code output: GET 200 hint: The same capture with field output switched on and two fields named: cap -a duration:8 -Y http -T fields -e http.request.method -e http.response.code
Two lines, two columns, no prose. GET with an empty second column, then an empty first column with 200.
Here are some useful HTTP fields:
| Field | What it shows |
|---|---|
http.request.method | GET, POST, PUT, DELETE |
http.request.uri | The path requested (/index.html, /data.json) |
http.response.code | Status code (200, 404, 500) |
http.host | The Host header value |
http.content_type | The Content-Type of the response |
http.content_length | The size of the response body in bytes |
http.user_agent | The User-Agent header (identifies the client) |
Every one of those field names is the same name tshark shows in a verbose dissection, which is the next step. When you do not know what a field is called, dissect one packet with -V and read the name off the screen. Extracting fields this way is the third of the five things your lab grades.
-V prints every layer of a packet instead of a summary line, and the http.request filter narrows the capture to the request itself, so you get one packet rather than a screenful.
Start the traffic first again:
(sleep 3; send-curl) &You will see the full stack, top to bottom. The frame size and the source port on the client are different on every run.
cap -a duration:8 -V -Y "http.request"
prompt: student@http-lab:~$ answer: cap -a duration:8 -V -Y "http.request" ||| cap -a duration:8 -V -Y http.request ||| cap -a duration:8 -Y "http.request" -V output: Frame <n>: <size> bytes on wire
Ethernet II, Src: 02:00:0a:00:01:0a, Dst: 02:00:0a:00:01:14 Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.0.1.10, Dst: 10.0.1.20 Protocol: TCP (6)
Transmission Control Protocol, Src Port: <ephemeral port>, Dst Port: 80 Flags: 0x018 (PSH, ACK)
Hypertext Transfer Protocol GET / HTTP/1.1\r\n Host: 10.0.1.20\r\n User-Agent: curl/<version>\r\n Accept: */*\r\n \r\n hint: The same capture with verbose mode and a filter for the request packet: cap -a duration:8 -V -Y "http.request"
Four layers, visible in one packet:
1. Ethernet - MAC addresses, EtherType 0x0800 (IPv4). 2. IP - Source and destination IP, protocol number 6 (TCP). 3. TCP - Source and destination ports. Port 80 is HTTP. The PSH flag means "push this data to the application." 4. HTTP - The actual request. GET method, path /, HTTP/1.1 version, and the request headers.
Each header ends with \r\n (carriage return + newline). A blank \r\n separates the headers from the body. This is the format of every HTTP message.
That last \r\n on its own line is the blank line you already met as a bare > in the curl output. Same byte sequence, two different tools, two different ways of showing it to you. Reading this dissection is the fourth of the five things your lab grades.
Stop and look at what that last screen actually was. MAC addresses, an EtherType, an IP header, a protocol number, TCP ports, a PSH flag, and then a line of readable English asking for a file. Every lesson in this track, stacked inside a single frame, in the order the machine unpacks them.
You did not learn those layers separately and then imagine how they fit. You captured one real packet and read them off it, outermost first.
What is left is small: asking the same server for something else, and knowing what it says when the answer is no.
The server has more than just the index page. The other file is the JSON one, and the only thing that changes is the path at the end of the URL. Leave -v off this time so that the body is all you get.
sudo ip netns exec client curl http://10.0.1.20/data.json
prompt: student@http-lab:~$ answer: sudo ip netns exec client curl http://10.0.1.20/data.json output: {"status":"ok","message":"The Linux Camp"} hint: The same curl, with /data.json on the end of the URL instead of /: sudo ip netns exec client curl http://10.0.1.20/data.json
The path in the request line changed from / to /data.json, and the server returned JSON instead of HTML. The Content-Type header in the response will say application/json instead of text/html.
This is how web APIs work. Different paths return different data. The same HTTP protocol, the same request/response cycle, but the path and content type change. Fetching a specific resource is the fifth and last of the things your lab grades.
Try adding -v to see the full request and response headers: sudo ip netns exec client curl -v http://10.0.1.20/data.json
Every response carries a status code, and the code is the first thing to read when something looks wrong:
| Code | Meaning | When you see it |
|---|---|---|
200 | OK | The request succeeded and the server returned data |
301 | Moved Permanently | The resource moved to a new URL (follow the Location header) |
304 | Not Modified | The cached version is still valid (no body returned) |
400 | Bad Request | The server could not understand your request |
403 | Forbidden | You do not have permission to access this resource |
404 | Not Found | The path does not exist on the server |
500 | Internal Server Error | Something broke on the server side |
If you request a path that does not exist (like /missing.html), the server answers with a 404. That is not an error in your command. It is the server telling you the resource is not there, and curl reports it as a normal, successful exchange. This particular server writes the status line as HTTP/1.0 404 File not found rather than the more common "Not Found", because the reason phrase is chosen by whoever wrote the server.
Scaffolding off. No command is printed from here on.
Learn this screen in a lab rather than in an outage. Ask that server for a file it does not have, /missing.html, with the headers visible so you can read the status line it sends back.
The reply is a full verbose ex
Practice HTTP in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.