Learn › Local AI on Linux › What a model is
Chapter 1 lab: What a model is. Five objectives on a real Ubuntu 24.04 machine with no internet: download Qwen3 0.6B from the machine's mirror, check it with sha256sum, read its metadata with gguf-dump, write its exact parameter count, and write the token IDs of a sentence from llama-tokenize. Free, one lab a day.
This chapter covers the following topics:
What you already know: Linux Foundations. This chapter uses its files and paths, curl, pipes and redirection, grep and awk. Every new term is defined where it first appears.
Objectives
After this chapter you can:
Each exercise in this chapter is a practice terminal. It checks the one command it asks for, and nothing you type in it can break anything. After two wrong tries a See answer button appears. When you start the practice machine from the panel, the real server takes the place of these simulated terminals. The machine is an Ubuntu 24.04 server with 4 processor cores, 8 GB of memory and no graphics card. It has no route to the internet: every file this chapter downloads comes from a mirror on the machine itself. It takes about three minutes to boot and fetch its files, up to four on a slow start. The graded lab at the end of the chapter runs on the same kind of machine. It is free, one lab a day, and it grades you from the files you create and the commands you run.
Five questions before the material. A wrong answer costs nothing: the explanation states the fact and names the section that teaches it.
>>> A model file holds weights: numbers set during training. A program such as llama.cpp reads them and does the arithmetic that turns your text into the model's reply. Section 1.1.
>>> Open-weight means you can download the weights. The licence still sets the terms, and the training data is usually not published, so open-weight is not the same as open source. Section 1.2.
>>> A SHA-256 checksum is computed from every byte of the file. One changed or missing byte gives a completely different checksum, so a match proves the file is the published one. Section 1.4.
>>> The B counts parameters, the numbers in the model. Names round, and the exact count in a file can be higher than the name suggests: Section 1.6 counts this one and finds 751,632,384.
>>> Ten. Each word becomes one token here, and the full stop is a tenth. Longer or rarer words split into several tokens, so tokens and words only roughly line up. Section 1.8.
A program written by a person follows rules that person typed. A model works differently. It is a large set of numbers, called weights, that a training process adjusted, step by step, until the model's output became useful. Each number is one parameter. A name such as Qwen3 0.6B means a model with about 0.6 billion parameters; 8B means about 8 billion.
A large language model, or LLM, is a model trained on text. It writes text one token at a time, where a token is a piece of text: often a whole word, sometimes part of one. For each new token the model predicts which token comes next, one is chosen, it is added to the text, and the step repeats. Section 1.8 splits a sentence into tokens.
The weights do nothing on their own. A program reads them from the file and does the arithmetic. This chapter uses the tools of llama.cpp, an engine written in C and C++ that runs models on ordinary processors and on graphics cards. Chapter 2 uses Ollama, a program built on the same kind of engine.
Training builds the weights. Inference is using them to produce output. Training the largest language models takes thousands of graphics cards for weeks; small models can be trained on one machine. Everything in Part 1 is inference: the weights in the file you download do not change when you use them, and Chapter 2 proves it with the file's checksum.
The model in this chapter is Qwen3 0.6B, published by the Qwen team at Alibaba Cloud under the Apache 2.0 licence. It is small enough to download, read and run in seconds on this machine, and its file has the same kinds of parts as the larger models in its family.
Model files are published on sites such as Hugging Face, the main site for sharing models and datasets, and in the library of models that Ollama downloads from.
An open-weight model publishes its weights for download, under a licence. A closed model does not: you send it text over the internet and receive a reply, but you never hold the weights. OpenAI's main GPT models, Claude and Gemini are closed. gpt-oss from OpenAI and Gemma from Google are open-weight models from the same companies. Open-weight is not the same as open source: the training data, and the code that trained the model, are usually not published.
This machine has no route to the internet. Ask it for Hugging Face with curl https://huggingface.co/ and the name does not even resolve:
curl: (6) Could not resolve host: huggingface.coEvery file this chapter needs comes from a mirror on the machine itself, a local web server at http://mirror.internal/. It holds pinned copies of the files, each checked against its published checksum when the machine was built. Ask it for its list of models:
prompt: admin@server1:~$ answer: curl http://mirror.internal/models/ output: <html> <head><title>Index of /models/</title></head> <body> <h1>Index of /models/</h1><hr><pre><a href="../">../</a> <a href="LICENSE">LICENSE</a> 15-Jan-2026 09:00 11338 <a href="SHA256SUMS">SHA256SUMS</a> 15-Jan-2026 09:00 338 <a href="SOURCES.txt">SOURCES.txt</a> 15-Jan-2026 09:00 879 <a href="qwen3-0.6b-f16.gguf">qwen3-0.6b-f16.gguf</a> 15-Jan-2026 09:00 1509347040 <a href="qwen3-0.6b-q4_k_m.gguf">qwen3-0.6b-q4_k_m.gguf</a> 15-Jan-2026 09:00 522640096 <a href="qwen3-1.7b-q4_k_m.gguf">qwen3-1.7b-q4_k_m.gguf</a> 15-Jan-2026 09:00 1359279776 </pre><hr></body> </html> hint: Section 1.2. curl, then the address of the mirror's models directory, http://mirror.internal/models/ with its closing slash.
The mirror answers with an index page, the way a web server lists a directory. Each file has a date and a size in bytes. Three are models: Qwen3 0.6B in the Q4_K_M format, 522,640,096 bytes; the same model at 16 bits per number, 1,509,347,040 bytes; and Qwen3 1.7B in Q4_K_M, 1,359,279,776 bytes. A bit is a single 0 or 1. Q4_K_M is called 4-bit because most of its numbers take 4 bits, but it is a mix, and llama.cpp counts this file at about 5.5 bits per number. That is why the 16-bit file is 2.9 times larger, not 4 times. Chapter 4 explains the bit counts. The other three files describe the models: their checksums, their sources and their licence.
A file you download should come with a record of where it came from. The mirror keeps one, SOURCES.txt:
prompt: admin@server1:~$ answer: curl http://mirror.internal/models/SOURCES.txt output: Where each file on this mirror came from. Every file was checked against the SHA-256 below when this machine was built.
qwen3-0.6b-q4_k_m.gguf Qwen3 0.6B by the Qwen team (Alibaba Cloud), 4-bit (Q4_K_M). Ollama library model qwen3:0.6b, model layer sha256:7f4030143c1c477224c5434f8272c662a8b042079a0a584f0a27a1684fe2e1fa qwen3-1.7b-q4_k_m.gguf Qwen3 1.7B by the Qwen team (Alibaba Cloud), 4-bit (Q4_K_M). Ollama library model qwen3:1.7b, model layer sha256:3d0b790534fe4b79525fc3692950408dca41171676ed7e21db57af5c65ef6ab6 qwen3-0.6b-f16.gguf Qwen3 0.6B by the Qwen team (Alibaba Cloud), 16-bit (F16). Ollama library model qwen3:0.6b-fp16, model layer sha256:bc2421370aa09f86eedd8e57e5b5ff3d1200f4638623c749c2ec820c3b5bff97 LICENSE The licence published with these models: Apache License 2.0.
The same files are also served as an Ollama registry under /v2/. hint: Section 1.2. curl, then the file SOURCES.txt in the mirror's models directory, http://mirror.internal/models/.
Each model file is a copy of a model in Ollama's library: qwen3:0.6b is the 0.6B model in the 4-bit Q4_K_M format. The long sha256: value after each one is the file's SHA-256 checksum, which Section 1.4 recomputes on your own copy. Q4_K_M names the 4-bit storage mix that Section 1.6 looks inside. The /v2/ registry at the end is what Chapter 2's Ollama downloads from.
An open-weight model comes with a licence: the terms under which you may use, change and share it. Read the first lines of the one on the mirror. The -s option silences curl's progress meter and its error messages, and head -n 4 keeps the first four lines:
prompt: admin@server1:~$ answer: curl -s http://mirror.internal/models/LICENSE | head -n 4 output: Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ hint: Section 1.2. curl, silenced, on the file LICENSE in http://mirror.internal/models/, piped into head for the first four lines.
Apache License, Version 2.0. It allows use, change and redistribution, including commercial use, on condition that you give recipients a copy of the licence, mark the files you changed, and keep the copyright and attribution notices. Other models carry other licences, and some limit how a model may be used, so read the licence before you build on a model. The model's own file records its licence too: Section 1.5 finds it.
Downloads, models, tools and your answers each get a directory of their own in your home directory. mkdir makes all four at once:
prompt: admin@server1:~$ answer: mkdir ~/models ~/downloads ~/tools ~/answers output: hint: Section 1.3. mkdir with four directories in your home: ~/models, ~/downloads, ~/tools and ~/answers.
mkdir prints nothing when it succeeds. List the directories in your home with ls -d ~/*/, where the trailing slash matches directories only:
/home/admin/answers/ /home/admin/models/
/home/admin/downloads/ /home/admin/tools/The download in the next step saves the file into the directory you are standing in. Move into ~/models first:
prompt: admin@server1:~$ answer: cd ~/models output: hint: Section 1.3. cd into the new directory ~/models.
The prompt now shows ~/models. The commands in Sections 1.3 to 1.6 name the model file without a path, so they work from here. Each also accepts the full name, ~/models/qwen3-0.6b-q4_k_m.gguf, from anywhere.
curl fetches the address it is given. With -O it saves what it receives to a file named after the last part of the address, here qwen3-0.6b-q4_k_m.gguf. With -R it gives the file the date the server reports, instead of the time of the download. The name itself carries three facts: the model (qwen3, 0.6b), how its numbers are stored (q4_k_m, Section 1.6) and the file format (.gguf, Section 1.5).
prompt: admin@server1:~/models$ answer: curl -R -O http://mirror.internal/models/qwen3-0.6b-q4_k_m.gguf output: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 498M 100 498M 0 0 652M 0 --:--:-- --:--:-- --:--:-- 651M hint: Section 1.3. In ~/models, curl with the two options that keep the name and the date, then http://mirror.internal/models/qwen3-0.6b-q4_k_m.gguf.
curl's progress meter: 100 percent of 498M received. The speeds and times are yours and will differ from the ones shown. The file came from a server on the same machine, which is why the download takes about a second.
A long listing shows the size of the file in bytes:
prompt: admin@server1:~/models$ answer: ls -l ||| ls -l ~/models output: total 510392 -rw-rw-r-- 1 admin admin 522640096 Jan 15 2026 qwen3-0.6b-q4_k_m.gguf hint: Section 1.3. The long listing of ~/models, the directory you are standing in.
522640096 bytes, the same number as the mirror's index: the whole file arrived. The date is the mirror's, 15 January 2026, because of -R. The first line, total, is the disk space the directory's files use, counted in blocks of 1,024 bytes.
Add -h and the size is printed in larger units:
prompt: admin@server1:~/models$ answer: ls -lh ||| ls -lh ~/models output: total 499M -rw-rw-r-- 1 admin admin 499M Jan 15 2026 qwen3-0.6b-q4_k_m.gguf hint: Section 1.3. The same long listing of ~/models, with the option for sizes people read.
499M. Here M means 1,048,576 bytes, 1,024 times 1,024. The file is 498.4 of those units, and ls -h rounds up, so it prints 499M; curl's meter rounded down to 498M. Counted in millions of bytes, the same file is 522.6 MB. When two tools disagree about a size, compare the exact byte counts.
A download can be cut short, damaged on the way, or replaced by a different file. Size alone cannot rule that out. A checksum can. SHA-256 reads every byte of a file and computes a 256-bit number from them, printed as 64 hexadecimal digits (base 16: the digits 0 to 9 and a to f). Change one byte anywhere and the checksum is completely different, and no practical way is known to make a different file with the same checksum. The publisher prints the checksum; you compute your own and compare.
prompt: admin@server1:~/models$ answer: sha256sum qwen3-0.6b-q4_k_m.gguf ||| sha256sum ~/models/qwen3-0.6b-q4_k_m.gguf output: 7f4030143c1c477224c5434f8272c662a8b042079a0a584f0a27a1684fe2e1fa qwen3-0.6b-q4_k_m.gguf hint: Section 1.4. sha256sum on the model file, qwen3-0.6b-q4_k_m.gguf in ~/models.
The checksum, two spaces, then the file name. It begins 7f4030143c1c and matches the sha256: line for this file in SOURCES.txt, Section 1.2. Ollama names each model layer by this same checksum, which is how Chapter 2 finds this file again inside Ollama.
Comparing 64 digits by eye is slow and easy to get wrong. Publishers put the checksums in a file, often named SHA256SUMS, with one line per file in the format sha256sum prints. Fetch the mirror's list into ~/models:
prompt: admin@server1:~/models$ answer: curl -O http://mirror.internal/models/SHA256SUMS output: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 338 100 338 0 0 239k 0 --:--:-- --:--:-- --:--:-- 330k hint: Section 1.4. In ~/models, curl with the option that keeps the name, then http://mirror.internal/models/SHA256SUMS.
338 bytes: four lines, one per file on the mirror, each a checksum and a file name. Your speed column will differ.
With -c, sha256sum reads a list instead of computing one: for each line it computes the checksum of the named file and compares. The list names four files and you downloaded one, so add --ignore-missing to skip the files that are not there:
prompt: admin@server1:~/models$ answer: sha256sum -c --ignore-missing SHA256SUMS output: qwen3-0.6b-q4_k_m.gguf: OK hint: Section 1.4. In ~/models, sha256sum in check mode on the list SHA256SUMS, skipping the files you did not download.
OK: the file you hold is byte for byte the file the list describes, so the download was not damaged. It proves nothing about who wrote the list: take the list from the publisher, over a connection you trust. Here, SOURCES.txt records that the mirror checked each file against the publisher's checksum when the machine was built. Run this check after every download of a model, a tool or an image.
Leave out --ignore-missing and every file on the list that you did not download is reported. sha256sum -c SHA256SUMS prints:
sha256sum: LICENSE: No such file or directory
LICENSE: FAILED open or read
sha256sum: qwen3-0.6b-f16.gguf: No such file or directory
qwen3-0.6b-f16.gguf: FAILED open or read
qwen3-0.6b-q4_k_m.gguf: OK
sha256sum: qwen3-1.7b-q4_k_m.gguf: No such file or directory
qwen3-1.7b-q4_k_m.gguf: FAILED open or read
sha256sum: WARNING: 3 listed files could not be readFAILED open or read means sha256sum could not open the file, here because it is not there; it does not mean the file is wrong. A wrong file looks different. Here a copy of the model was cut short at its first 1,000,000 bytes and checked against the full file's checksum with sha256sum -c partial.sums:
partial.gguf: FAILED
sha256sum: WARNING: 1 computed checksum did NOT matchFAILED with nothing after it means the bytes differ from the published file. Delete the file and download it again; do not use it.
GGUF is the model file format of llama.cpp and Ollama. One GGUF file holds the whole model. It starts with a header, then metadata: named settings such as the model's n
Practice Chapter 1. What a model is in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.