Learn › BMC & Server Management › First Day on the Floor
Meet the BMC, the always-on computer inside every server. Learn out-of-band management with interactive diagrams, then verify a live Dell BMC with ipmitool on a real machine.
It is 08:14 at the DC-EAST facility in Ashburn, Virginia. A new Dell PowerEdge R750xs was wheeled into Bay 14, Row 7 an hour ago. It is racked and cabled, but it is switched off. There is no operating system running on it yet.
The work order on your tablet is simple: confirm the server is alive and reachable before it joins the production pool.
Here is the puzzle. The machine is powered off. It has no OS to SSH into. So how do you check that a computer is healthy when the computer itself is not even on?
This is a Practice Zone. The diagrams below react to you: click the boxes, flip the switches. The real machine comes at the end, under Ready to practice.
Before we answer the puzzle, look at what is actually in that bay. A modern server is not one computer. It is two.
{ "height": 400, "caption": "Two separate paths into one server. Click any hop.", "nodes": [ { "id": "you", "label": "You (NOC)", "kind": "admin", "x": 0, "y": 170, "detail": "You manage this facility remotely, over the internet or a VPN. Out-of-band management means you never need to be in the building." }, { "id": "mcore", "label": "Mgmt core sw", "kind": "net", "x": 220, "y": 40, "detail": "Core switch of the management network. This whole top lane exists only for managing hardware." }, { "id": "magg", "label": "Mgmt agg sw", "kind": "net", "x": 410, "y": 40, "detail": "Aggregation layer of the management network." }, { "id": "msw", "label": "Management sw", "kind": "net", "x": 600, "y": 40, "detail": "The management switch for this row. Every BMC port in the row uplinks here." }, { "id": "bmcport", "label": "BMC port", "kind": "nic", "x": 800, "y": 40, "detail": "A dedicated RJ45 on the back of the server, owned by the BMC. Its own cable, its own MAC address, alive on standby power." }, { "id": "bmc", "label": "BMC", "kind": "bmc", "x": 970, "y": 40, "detail": "The Baseboard Management Controller: a tiny always-on computer inside the server, with its own processor, memory, and power feed. It runs even when the big server is switched off." }, { "id": "pcore", "label": "Core sw", "kind": "net", "x": 220, "y": 300, "detail": "Core switch of the production network, where workload traffic lives." }, { "id": "pagg", "label": "Agg sw", "kind": "net", "x": 410, "y": 300, "detail": "Aggregation layer of the production network." }, { "id": "psw", "label": "Access sw", "kind": "net", "x": 600, "y": 300, "detail": "The production access switch this server uplinks through." }, { "id": "eth0", "label": "eth0", "kind": "nic", "x": 800, "y": 300, "detail": "The host's network port. It belongs to the operating system, and it only answers while the host is up." }, { "id": "host", "label": "Host OS", "kind": "host", "x": 970, "y": 300, "detail": "The big machine and its operating system, the part that runs real workloads. It needs power and a booted OS before it can answer you. The BMC has neither requirement." } ], "edges": [ { "from": "you", "to": "mcore", "label": "management traffic", "kind": "mgmt" }, { "from": "mcore", "to": "magg", "kind": "mgmt" }, { "from": "magg", "to": "msw", "kind": "mgmt" }, { "from": "msw", "to": "bmcport", "kind": "mgmt" }, { "from": "bmcport", "to": "bmc", "kind": "mgmt" }, { "from": "you", "to": "pcore", "label": "production traffic", "kind": "inband" }, { "from": "pcore", "to": "pagg", "kind": "inband" }, { "from": "pagg", "to": "psw", "kind": "inband" }, { "from": "psw", "to": "eth0", "kind": "inband" }, { "from": "eth0", "to": "host", "kind": "inband" }, { "from": "bmc", "to": "host", "label": "watches + controls", "kind": "plain" } ] }
Trace both lanes. The top lane is the management network: its own core, aggregation, and management switches, ending at a dedicated BMC port on the back of the server. The bottom lane is the production network, ending at eth0, the port the host OS owns. Two networks, two cables, two ports, one server.
The big server is the host. The small computer bolted onto its motherboard is the BMC. The BMC has one job: manage the host, from the outside, no matter what state the host is in.
>>> The BMC. It has its own power and never fully sleeps as long as the rack has standby power. That is the whole reason it exists: to give you a way in when the host cannot give you one.
There are two doors into any server. Which one you use depends on whether the host is healthy.
In-band means you reach the server *through* its running operating system, the ordinary way, like an SSH login. Out-of-band means you reach the BMC *beside* the OS, not through it.
Flip the power switch on the diagram and watch which door stays open.
{ "height": 300, "caption": "Flip the power. The out-of-band door never closes.", "nodes": [ { "id": "you", "label": "Your workstation", "kind": "admin", "x": 0, "y": 170 }, { "id": "host", "label": "Host OS", "kind": "host", "x": 620, "y": 40, "detail": "In-band: you reach the machine through its running OS. Fast and full-featured, but only while the OS is actually up." }, { "id": "bmc", "label": "BMC", "kind": "bmc", "x": 620, "y": 300, "detail": "Out-of-band: you reach the BMC directly. It answers when the OS is crashed, hung, mid-reboot, or the machine is powered off entirely." } ], "edges": [ { "from": "you", "to": "host", "label": "in-band (SSH to OS)", "kind": "inband" }, { "from": "you", "to": "bmc", "label": "out-of-band (IPMI)", "kind": "oob" }, { "from": "bmc", "to": "host", "label": "power + reset", "kind": "power" } ], "toggle": { "label": "Server power:", "on": "Powered ON", "off": "Powered OFF", "dimOff": ["host", "e:you-host"] } }
>>> Out-of-band. The OS is the thing that failed, so the in-band door is shut. The BMC is a separate computer that is still awake, so its door is open. From your workstation you can tell the BMC to power-cycle the host without leaving your desk. That is the superpower of out-of-band management, and it is what you are about to do for real.
A BMC (Baseboard Management Controller) is a small dedicated computer embedded on the server motherboard. It has its own firmware, its own network interface, and standby power, so it keeps running independently of the host.
You talk to a BMC using management protocols. Two matter for this track:
ipmitool command speaks it. Universal across vendors.Vendors also brand their BMCs: Dell calls theirs iDRAC, HPE calls theirs iLO. The server in your bay is a Dell, so its BMC is an iDRAC, and it answers Dell's racadm commands over SSH as well as universal IPMI.
One question you might be asking: if the server just arrived, how does it already have a management address? The BMC comes up on standby power the moment the rack is energized and pulls a management IP from DHCP, or the facility pre-stages a fixed one during racking. Either way the BMC is reachable before anyone ever powers the host on. In your lab, that address is waiting in $BMC_IP.
Out-of-band is why data centers can run with almost no one on the floor. Thousands of servers, each with a BMC, all reachable from a desk. When you hear "lights-out management," this is it.
Two ideas that sound alike but answer different questions:
In dedicated mode, the BMC has its own port and its own network, the two-lane picture you saw earlier. In shared mode there is no second cable. Management traffic arrives on eth0's wire, and a tiny internal side channel on the motherboard called NC-SI (the sideband, carried on RMII or SMBus lines between the NIC and the BMC) hands it over. One wire, two riders, each with its own MAC address.
Flip the port mode and watch the road change.
{ "height": 360, "caption": "Same BMC either way. What changes is the road its traffic takes.", "nodes": [ { "id": "inet", "label": "You (NOC)", "kind": "admin", "x": 0, "y": 180 }, { "id": "mnet", "label": "Management network", "kind": "net", "x": 280, "y": 40, "detail": "The separate management switch fabric from earlier, collapsed into one box. Only exists as the BMC's road in dedicated mode." }, { "id": "pnet", "label": "Production network", "kind": "net", "x": 280, "y": 320, "detail": "The production switch fabric the host uses for workload traffic." }, { "id": "bmcport", "label": "BMC port", "kind": "nic", "x": 600, "y": 40, "detail": "The dedicated management RJ45. In shared mode this port is simply not used." }, { "id": "eth0", "label": "eth0 (LOM)", "kind": "nic", "x": 600, "y": 320, "detail": "The host's LAN-on-motherboard port. In shared mode the BMC's management traffic rides this same wire, steered by MAC address." }, { "id": "bmc", "label": "BMC", "kind": "bmc", "x": 880, "y": 130, "detail": "Same BMC in both modes. In shared mode its traffic reaches it through the NC-SI sideband instead of a dedicated port." }, { "id": "host", "label": "Host OS", "kind": "host", "x": 880, "y": 330 } ], "edges": [ { "from": "inet", "to": "mnet", "label": "dedicated path", "kind": "mgmt" }, { "from": "mnet", "to": "bmcport", "kind": "mgmt" }, { "from": "bmcport", "to": "bmc", "kind": "mgmt" }, { "from": "inet", "to": "pnet", "label": "production traffic", "kind": "inband" }, { "from": "pnet", "to": "eth0", "kind": "inband" }, { "from": "eth0", "to": "host", "kind": "inband" }, { "from": "eth0", "to": "bmc", "label": "NC-SI sideband", "kind": "oob" } ], "toggle": { "label": "Port mode:", "on": "Dedicated BMC port", "off": "Shared LOM (NC-SI)", "dimOn": ["e:eth0-bmc"], "dimOff": ["mnet", "bmcport", "e:inet-mnet", "e:mnet-bmcport", "e:bmcport-bmc"] } }
>>> Dedicated. In shared mode the BMC's traffic rides eth0's wire, so when the production path dies, the BMC goes unreachable right along with the host. That failure coupling, plus the security problem of management traffic touching the production network, is why serious facilities pay for the second cable. Shared mode saves ports and cabling on cheaper deployments. You will build and defend both topologies later in this track.
You have two ways to reach the Dell BMC in Bay 14:
1. SSH into the BMC gives you the vendor shell (Dell racadm). Vendor-specific, powerful. 2. ipmitool from your workstation speaks universal IPMI over the network. The same command works against a Dell, an HPE, a Supermicro, anything.
Let us preview the universal method. This is the exact command you will run on the real machine in a moment. $BMC_IP is already set for you in the lab to the BMC's address.
prompt: ops@dc-east-ws01:~$ answer: ipmitool -I lanplus -C 3 -H $BMC_IP -U ADMIN -P ADMIN mc info output: Device ID : 0 Device Revision : 0 Firmware Revision : 9.08 IPMI Version : 2.0 Manufacturer ID : 674 Manufacturer Name : DELL Inc Product ID : 564 (0x0234) hint: Type it exactly: ipmitool -I lanplus -C 3 -H $BMC_IP -U ADMIN -P ADMIN mc info
Reading that command left to right: -I lanplus selects the IPMI 2.0 network interface, -C 3 picks the session cipher suite, -H $BMC_IP is the BMC address, -U ADMIN -P ADMIN are its credentials, and mc info asks the management controller to describe itself. The reply confirms a live Dell BMC running IPMI 2.0. That is proof of life.
Why name the cipher with -C 3? A cipher suite is the encryption a BMC session uses. Suite 3 is HMAC-SHA1, which every IPMI 2.0 BMC supports. Recent ipmitool builds default to suite 17 (HMAC-SHA256) and print a noisy warning when a BMC does not offer it, so naming suite 3 explicitly keeps the session clean. You will study cipher suites as a security topic later in this track.
The BMC in this lab is a standards-faithful emulator: it speaks real IPMI 2.0 over LAN, the Dell racadm shell, and Redfish, so every command you learn here is the exact command you would run against physical hardware. A real iDRAC returns more inventory and vendor-specific fields, but the protocol, the tools, and the muscle memory are identical.
Time to do it for real. On the machine you are about to open, close out Work Order DC-EAST-WO-1001 by proving the BMC is online. The BMC address is preset for you in $BMC_IP, and the credentials are ADMIN / ADMIN.
Six objectives stand between this server and the production pool:
1. Raise the controller. Reach the BMC's own Dell shell and pull its identity readout from inside. 2. Interrogate it from outside. Make the BMC identify itself over the network, no SSH session, using the IPMI walkthrough you just ran. 3. File the identity. The NOC wants that readout saved to ~/bmc-info.txt. 4. Read the heartbeat. Report the server's live power state straight from the BMC. 5. Paper trail. Auditors want just the firmware line, in ~/firmware-check.txt. 6. The challenge. The dayshift lead wants hard proof this box is genuinely off, saved to ~/power-proof.txt. No hints on this one.
Here is the deal: the workspace shows no commands. It hands you one objective at a time, and you recall the move. That recall is the whole point; it is how last hour's reading becomes your skill.
Stuck is normal. Hit Request a signal on any objective for a nudge, or ask Daemon. It knows exactly which task you are on and will nudge, not spoil.
Your workstation dc-east-ws01 is booting in the bay below, with the Dell BMC on the management network beside it. Launch it and close the work order.
Practice First Contact in a real Linux terminal at The Linux Camp. Progress is verified automatically as you type commands on the machine.