Learn › BMC & Server Management › OpenBMC on Real Firmware
Walk the whole chain and then open the lid. SSH into the OpenBMC service processor itself, find the ARM Linux kernel, the flash layout with the host firmware partition, and the Phosphor services behind every Redfish and IPMI answer you have collected, then file the handover note.
Four work orders in, you have driven this service processor through two protocols. You read its service root, took a token, moved its power state machine, and pulled its logs.
Every one of those answers was produced by software running on the other end. This lesson is about going to look at that software.
Here is how the machine you have been talking to comes up.
U-Boot 2019.04 (Jun 02 2026 - 01:44:00 +0000)
SOC : AST2500-A1
RST : Power On
2nd Boot : Enable
LPC Mode : SIO:Enable : SuperIO-2e
Eth : MAC0: RMII/NCSI, , MAC1: RMII/NCSI,
Model: Romulus BMC
DRAM: 192 MiB (capacity:256 MiB, VGA:64 MiB, ECC:off)
MMC:
Loading Environment from SPI Flash... SF: Detected n25q256a with page size 256 Bytes, erase size 4 KiB, total 32 MiBA bootloader, on an AST2500 service processor, loading its environment from a 32 MiB SPI flash part. That is the same sequence a physical BMC runs when you energise a rack.
The thing people miss about a BMC is the simplest thing about it. It is a whole computer. Its own processor, its own memory, its own flash, its own Linux kernel, its own service manager, running whether the big machine beside it is on or off.
The Linux distribution it runs is called Phosphor, and it is what the OpenBMC project builds. bmcweb serving Redfish and ipmid answering IPMI are both just services under systemd on that little machine.
And it has an SSH server. Which means you can go and stand inside it.
The BMC runs dropbear, a small SSH server built for embedded systems. Your workstation reaches it on $BMC_SSH_PORT with the same credentials you have used all module.
The full form is sshpass -p 0penBmc ssh -o StrictHostKeyChecking=no -p $BMC_SSH_PORT root@$BMC_HOST and then the command you want. bmc-ssh on your workstation is a wrapper around exactly that.
Ask it what kernel it is running.
prompt: ops@dc-east-fw01:~$ answer: bmc-ssh 'uname -a' ||| sshpass -p 0penBmc ssh -o StrictHostKeyChecking=no -p $BMC_SSH_PORT root@$BMC_HOST 'uname -a' output: Linux romulus 6.18.40-c2b9fc3 #1 Mon Jul 27 02:33:15 UTC 2026 armv6l GNU/Linux hint: bmc-ssh takes the command to run on the BMC as its argument, in quotes
Read that line field by field. The host is called romulus. The kernel is 6.18.40, a current Linux. The architecture is armv6l, a 32-bit ARM.
That is a full Linux system living on the motherboard of the server you were powering on and off.
Look at the processor underneath it.
prompt: ops@dc-east-fw01:~$ answer: bmc-ssh 'head -n 8 /proc/cpuinfo' ||| sshpass -p 0penBmc ssh -o StrictHostKeyChecking=no -p $BMC_SSH_PORT root@$BMC_HOST 'head -n 8 /proc/cpuinfo' output: processor : 0 model name : ARMv6-compatible processor rev 7 (v6l) BogoMIPS : 49.50 Features : half thumb fastmult edsp java tls CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x0 CPU part : 0xb76 hint: Run head -n 8 on /proc/cpuinfo through bmc-ssh
One core, an ARMv6-compatible processor, and about 180 MB of usable memory. The entire Redfish service you have been driving all module runs in that.
Hold that hardware in your head for a moment, because it changes how you are allowed to write automation against these machines.
>>> The BMC becomes the bottleneck. This is a single small core with a couple of hundred megabytes of memory, serving TLS. Hammering it is a real way to make a management plane unreliable, and the failure is usually slow timeouts rather than a clean error, which is much harder to diagnose. This is why fleet tools cache, back off and stagger their polls. Knowing what the processor is stops you from writing the script that takes out your own management network.
On a BMC, storage is flash, and flash is divided into named partitions. Ask the kernel for the layout.
prompt: ops@dc-east-fw01:~$ answer: bmc-ssh 'cat /proc/mtd' ||| sshpass -p 0penBmc ssh -o StrictHostKeyChecking=no -p $BMC_SSH_PORT root@$BMC_HOST 'cat /proc/mtd' output: dev: size erasesize name mtd0: 02000000 00010000 "bmc" mtd1: 00060000 00010000 "u-boot" mtd2: 00020000 00010000 "u-boot-env" mtd3: 00440000 00010000 "kernel" mtd4: 01740000 00010000 "rofs" mtd5: 00400000 00010000 "rwfs" mtd6: 08000000 00010000 "pnor" hint: The kernel publishes the flash layout at /proc/mtd. Read it through bmc-ssh
Four of those explain how BMC firmware updates survive a bad flash.
u-boot and u-boot-env are the bootloader you saw at the start and its settings.rofs is the read-only root filesystem, mounted full and unwritable.rwfs is a small writable overlay, where changes actually land.pnor is the big one, 128 MiB, and it is not the BMC at all. It is the host firmware flash, the chip the main processor boots from, wired to the BMC so the BMC can serve and reflash it.That last partition is why a BMC can recover a server whose main firmware is corrupt. It owns the chip the host boots from.
Everything you have used this module is a systemd service on this little machine. Here is part of the list that was running.
avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack
* [email protected] loaded failed failed Disable the AVS bus on the VRMs
* [email protected] loaded failed failed Enable the AVS bus on VRMs
bmcweb.service loaded active running bmcweb server
busybox-klogd.service loaded active running Kernel Logging Service
busybox-syslog.service loaded active running System Logging Service
dbus-broker.service loaded active running D-Bus System Message Bus
dropbearkey.service loaded active exited SSH Key Generation
* fsi-scan.service loaded failed failed Scan FSI devicesbmcweb.service is the process that answered every Redfish request you made. dbus-broker is the message bus every OpenBMC component talks over. dropbearkey generated the host key for the SSH session you are sitting in.
That capture is trimmed for width and it is a slice of a longer list, so read it as an illustration and go count your own. The three failed units are expected here: avsbus drives voltage regulators and fsi-scan probes a bus to the host processor, and an emulator does not model either. On physical hardware those come up clean.
answer: bmc-ssh 'systemctl list-units --type=service --no-pager --no-legend' | wc -l ||| bmc-ssh 'systemctl list-units --type=service --no-pager --no-legend | wc -l' hint: Run systemctl list-units --type=service --no-pager --no-legend through bmc-ssh, then pipe into wc -l
Read your own number off your own machine. Then look for bmcweb.service in that list and remember that it is the thing you have been talking to all module.
One job is left that you have not done: watching the host boot. IPMI has a feature for that called Serial over LAN, and on this firmware it does not work. Asking for it returns Error requesting SOL parameter 'Set In Progress (0)': Invalid data field in request.
OpenBMC solved the same problem differently. It runs a console server, obmc-console, and publishes the host serial console over SSH on its own port. Your endpoint has it as $BMC_CONSOLE_PORT.
Knock on that port and see what answers.
prompt: ops@dc-east-fw01:~$ answer: echo | nc $BMC_HOST $BMC_CONSOLE_PORT | head -n 1 output: SSH-2.0-dropbear_2026.92 hint: Pipe an empty echo into nc against $BMC_HOST and $BMC_CONSOLE_PORT, then keep the first line
An SSH banner. That port is a second SSH service whose only job is to hand you the host serial console. SSH into it and you are watching the machine boot, with all of SSH authentication and encryption around it.
This is why the industry moved. IPMI Serial over LAN was a bespoke protocol with bespoke problems. The console over SSH is one everybody already knows how to secure.
You have now met three limits on this build: no sensor readings, no Serial over LAN, and one more you have not tested. How you write those up matters.
>>> Say what you tested and what you found, with the version. A resource that returns 404 has not failed, it is absent from this build, which is a fact about the image and not a fault in the machine. The reason to write it down with a version is that the next build may add it. Engineers who report what is missing, and against what version, are the ones whose reports still mean something six months later.
You have driven real OpenBMC firmware end to end. You read the service root and took a session token, then proved the BMC enforces it. You moved the power state machine and watched it transition. You pulled the event log and the sensor catalog, and wrote down which numbers were real.
Then you logged into the service processor itself and found the kernel, the flash layout, and the services behind every answer you had been given.
Nothing in that chain was a simulation of a standard. It was the standard, running the code that ships on production fleets.
Work Order FW-05, six objectives. This one is a handover: everything you collect goes to the engineer who takes the machine after you.
1. The kernel. Save the BMC kernel banner to ~/bmc-uname.txt. 2. The firmware. Save the OS identity to ~/bmc-os.txt. 3. The flash. Save the partition layout to ~/bmc-mtd.txt. 4. The services. Save the running service list to ~/bmc-services.txt. 5. The console. Save the console port banner to ~/console-banner.txt. 6. The challenge. Write the handover note in ~/handover.txt.
The grader logs into the BMC itself to confirm the kernel and the web service, so the files have to match the machine.
Your workstation is booting below with the OpenBMC instance beside it. Open Work Order FW-05, get inside the firmware, and leave a handover note the next engineer can act on.
Practice Inside the Firmware in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.