ethtool -S names the exact counter, but the names belong to the driver, not to Linux. Three families cover every drop: damaged on the wire, lost at the ring, or refused above the device. Read deltas, never totals.
Short version: run ethtool -S eth0 and filter for drop, miss, discard and err, then watch the deltas rather than the totals. CRC and symbol counters blame the cable. Missed, out-of-buffer and queue drops blame a full ring. On EC2, allowance counters blame the instance size. If the driver reports no stats, ip -s -s link still answers.
Every shell block below came off one of two real machines: a Firecracker Debian VM with a virtio NIC, prompt root@net-01, and a production EC2 host with an ENA NIC, prompt root@storage-01. Counter sets from hardware not in that pair appear in blocks marked with their source.
The counter name tells you where the packet died, and there are only three places it can happen.
On the wire. The frame arrived damaged and the NIC discarded it after the frame check sequence failed. rx_crc_errors, rx_frame_errors and rx_length_errors live here. Nothing you tune in software fixes this one.
At the ring. The frame was perfect, the NIC held it, and the host never took it in time. rx_missed_errors is the standard name, joined by rx_fifo_errors and rx_over_errors as legacy synonyms, plus whatever the driver calls its own version.
Above the device. The packet cleared the wire and the device, and something further up refused it. Standard rx_dropped sits here, and so do the cloud policer counters on EC2.
The thing that trips most engineers is that only the first list is portable. The ethtool manual is blunt about it, noting there is "little commonality between naming" of driver statistics (ethtool(8)). Intel, virtio, Broadcom and Mellanox all have a counter for a packet the host never processed and all four spell it differently. Learn the three families and then look up the dialect.
Identify the driver before reading a single counter, because the vocabulary belongs to the driver rather than to Linux.
root@net-01:~# ethtool --version
ethtool version 6.1
root@net-01:~# ethtool -i eth0
driver: virtio_net
version: 1.0.0
firmware-version:
expansion-rom-version:
bus-info: LNRO0005:01
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: nosupports-statistics: yes is the line that matters. It is the difference between -S returning counters and returning nothing at all. Now read them.
root@net-01:~# ethtool -S eth0
NIC statistics:
rx_queue_0_packets: 1607
rx_queue_0_bytes: 15367287
rx_queue_0_drops: 0
rx_queue_0_xdp_packets: 0
rx_queue_0_xdp_tx: 0
rx_queue_0_xdp_redirects: 0
rx_queue_0_xdp_drops: 0
rx_queue_0_kicks: 141
tx_queue_0_packets: 1238
tx_queue_0_bytes: 97326
tx_queue_0_xdp_tx: 0
tx_queue_0_xdp_tx_drops: 0
tx_queue_0_kicks: 1140
tx_queue_0_tx_timeouts: 0Fourteen lines is a small dump. A virtio NIC has no physical layer to report on, so there is no CRC counter here and there never will be. On an Intel or Mellanox card the same command can run to hundreds of lines, which is why the portable move is to filter on the fragments the families share.
root@net-01:~# ethtool -S eth0 | grep -Ei 'drop|miss|err|fifo'
rx_queue_0_drops: 0
rx_queue_0_xdp_drops: 0
tx_queue_0_xdp_tx_drops: 0Three zeros. That is what a healthy interface looks like, and it is worth saying plainly: a quiet box prints zeros and there is nothing clever hiding behind them. Drop hunting happens under load, against deltas.
Counters are cumulative since the driver last reset, so a large total proves nothing about right now. Take two snapshots and diff them.
root@net-01:~# ethtool -S eth0 | grep -Ei 'drop|err' > /tmp/s1
root@net-01:~# sleep 5
root@net-01:~# ethtool -S eth0 | grep -Ei 'drop|err' > /tmp/s2
root@net-01:~# diff /tmp/s1 /tmp/s2 && echo no-change
no-changeno-change is a real answer, not a failed test. It rules the device out for that window. Push traffic through and the packet counters move while the drop counters do not, which is the shape you are looking for.
root@net-01:~# GW=$(ip -4 route show default | awk '{print $3}')
root@net-01:~# ping -c 200 -i 0.01 $GW | tail -3
--- 10.0.1.1 ping statistics ---
200 packets transmitted, 200 received, 0% packet loss, time 3181ms
rtt min/avg/max/mdev = 0.052/0.112/0.180/0.039 ms
root@net-01:~# ethtool -S eth0 | grep -Ei 'packets|drops'
rx_queue_0_packets: 2319
rx_queue_0_drops: 0
rx_queue_0_xdp_packets: 0
rx_queue_0_xdp_drops: 0
tx_queue_0_packets: 2090
tx_queue_0_xdp_tx_drops: 0For a live incident, watch -n1 -d 'ethtool -S eth0 | grep -Ei "drop|miss|err"' highlights fields as they change. For a ticket, the two snapshots win, because a diff is evidence somebody else can read.
A nonzero rx_crc_errors means frames are arriving corrupted, which is a hardware conversation and not a tuning one. The NIC computed the frame check sequence, compared it to the one on the wire, and threw the frame away. No ring size, no interrupt affinity and no kernel parameter changes that outcome.
The standard view names the cause directly, whether or not the driver offers private counters.
root@net-01:~# ip -s -s link show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
link/ether aa:fc:00:00:00:01 brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
15364533 1574 0 0 0 0
RX errors: length crc frame fifo overrun
0 0 0 0 0
TX: bytes packets errors dropped carrier collsns
93506 1200 0 0 0 0
TX errors: aborted fifo window heartbt transns
0 0 0 0 2 The doubled -s is what produces the RX errors breakdown. One -s gives totals only, and the man page notes that repeating the option increases the detail (ip(8)). Those five columns are the three families in miniature: length, crc and frame are wire damage, fifo and overrun are the ring.
Work the physical path when crc moves: reseat or replace the cable, try a different switch port, swap the transceiver, look for electrical noise along the run. Two causes get missed. A duplex mismatch produces late collisions that surface as CRC errors, and an MTU mismatch, where one side sends jumbo frames the other will not accept, can register as CRC plus length errors rather than as anything mentioning MTU.
On Mellanox and NVIDIA cards the physical port counters are separate from the per-queue ones and carry a _phy suffix.
rx_crc_errors_phy packets dropped for FCS error on the physical port
rx_symbol_err_phy physical coding (symbol) errors, high bit error rate
rx_corrected_bits_phy bits corrected by FEC, an early warning before loss
rx_discards_phy packets dropped for lack of port buffer
Source: Linux kernel mlx5 counter documentation,
https://docs.kernel.org/networking/device_drivers/ethernet/mellanox/mlx5/counters.htmlrx_corrected_bits_phy is the useful one nobody looks at. Forward error correction is repairing the link before any frame is actually lost, so the counter climbs while throughput still looks fine. That is the window to schedule a cable swap instead of taking an outage.
These two counters describe the same pressure at different stages. On Intel hardware rx_no_buffer_count means the host has not posted free buffers fast enough, so packets are delayed. rx_missed_errors means the adapter FIFO filled and packets were discarded. No buffer is starving. Missed is losing.
rx_no_buffer_count adapter FIFO holds packets, host posted no free buffers
rx_missed_errors FIFO itself filled, adapter dropped the packet
Source: Intel e1000-devel maintainer threads,
https://e1000-devel.narkive.com/gs1U4mB9/e1000-driver-rx-missed-errors
https://e1000-devel.narkive.com/31D89C8J/rx-no-buffer-count-errorsThe kernel keeps the same split at the standard layer. rx_missed_errors is documented as packets "missed by the host" (kernel networking statistics), and buffer exhaustion belongs there rather than in rx_dropped. Blog posts that say rx_dropped means the ring was full have it backwards. Mellanox spells this family rx_out_of_buffer and rx_discards_phy, virtio spells it rx_queue_0_drops, and Broadcom spells it rx_discard_pkts.
When this family is moving, the first lever is the receive ring. Ask what the hardware allows.
root@net-01:~# ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX: 256
RX Mini: n/a
RX Jumbo: n/a
TX: 256
Current hardware settings:
RX: 256
RX Mini: n/a
RX Jumbo: n/a
TX: 256There is no lever here. Current equals the pre-set maximum, so ethtool -G eth0 rx N has nothing left to give and the next move would be CPU and interrupt work. The ENA host is the other case.
root@storage-01:~# ethtool -g ens5
Ring parameters for ens5:
Pre-set maximums:
RX: 2048
RX Mini: n/a
RX Jumbo: n/a
TX: 1024
Current hardware settings:
RX: 1024
RX Mini: n/a
RX Jumbo: n/a
TX: 1024Receive is at half of what the hardware supports, so ethtool -G ens5 rx 2048 is available. Two warnings. On many drivers a ring change resets the interface and briefly drops the link, so treat it as a maintenance window, especially when the console rides that NIC. And ethtool settings do not survive a reboot. Persist them through NetworkManager connection properties, a dispatcher script or a systemd .link file, or the change disappears at the worst moment.
A bigger ring buys tolerance for bursts, not throughput. If the host still cannot drain fast enough, the drops come back at a higher packet rate.
On EC2 these counters mean the instance hit its own network limits, so the NIC is innocent and no amount of ring tuning will help. Check the driver first, because the metrics only exist on a recent enough ENA driver.
root@storage-01:~# ethtool -i ens5
driver: ena
version: 2.16.1g
firmware-version:
expansion-rom-version:
bus-info: 0000:00:05.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: yesAWS documents 2.2.10 as the floor for the allowance metrics, and 2.8.1 plus a Nitro instance for conntrack_allowance_available. This host is well past both, so all six appear.
root@storage-01:~# ethtool -S ens5 | grep allowance
bw_in_allowance_exceeded: 3297631
bw_out_allowance_exceeded: 217581
pps_allowance_exceeded: 152031
conntrack_allowance_exceeded: 0
linklocal_allowance_exceeded: 0
conntrack_allowance_available: 51268Read those carefully. For the bandwidth and packets-per-second counters, AWS describes the value as packets "queued or dropped" once traffic exceeds an instance maximum, so an increment is not automatically a lost packet. Shaping delays first and drops second. The conntrack and linklocal counters are described as dropped outright. conntrack_allowance_available is not an error at all: it is remaining headroom, connections still available before tracking starts refusing.
The usual cause is a microburst, a spike lasting seconds or less. CloudWatch NetworkIn and NetworkOut are per-minute averages, which flatten exactly the peak that tripped the policer, so a calm graph and a climbing counter are not in conflict. The fixes are instance sizing, spreading load, or deciding the shaping is acceptable. Nothing in ethtool -G moves an AWS allowance.
no stats available means the driver implements no statistics callback, not that the interface is healthy. Loopback is the clean example.
root@net-01:~# ethtool -S lo
no stats availableSome virtual NICs answer the same way, which is why ethtool -i and its supports-statistics line belong before -S in the workflow. A second failure looks like a missing package and is not.
root@net-01:~# su - probe -c 'ethtool -S eth0 | head -5'
-sh: 1: ethtool: not found
root@net-01:~# su - probe -c 'ethtool -G eth0 rx 512'
-sh: 1: ethtool: not foundThat is a PATH result rather than a permission result. The binary lives in an sbin directory that an ordinary user's PATH does not include, so the shell gives up before the kernel is ever asked whether the read was allowed. Use sudo or the absolute path.
Either way, the standard counters are always there. ip -s -s link reads the kernel's own structure rather than the driver's private one, so it works on every interface on every host, including the ones ethtool cannot describe.
They disagree because they are two separate accounting systems, and neither is lying. The kernel exposes standard interface statistics with fixed meanings, which is what ip shows, alongside driver-defined statistics, which is what ethtool -S shows. A driver counter only appears in the ip view if the driver deliberately folds it into a standard field.
Here is the same ENA host from the standard side, trimmed to the receive rows.
root@storage-01:~# ip -s -s link show ens5
RX: bytes packets errors dropped missed mcast
370111847883 135890357 0 4070 0 0
RX errors: length crc frame fifo overrun
0 0 0 0 0 Millions of allowance events in ethtool -S, and the standard view shows zero errors and 4070 dropped out of 135890357 packets received. Both are correct. The allowance counters are ENA's own, and nothing in the standard structure represents an AWS policer, so they never appear on this side.
That leaves the question people actually arrive with. rx_dropped is increasing, is something broken? Usually not. The standard definition covers packets received but not processed, including unsupported protocols, and layer 2 filtering discards land there too. Unknown ethertypes and tagged frames for a VLAN nobody configured are the everyday culprits. Compare the drop rate against packets received before tuning anything. The reverse case is real too: some drivers fold physical port discards into rx_fifo_errors, so a fifo count on a Mellanox card can mean adapter congestion rather than anything a FIFO did.
One honest limit. Inside a VM, the guest sees only what the virtual NIC reports. Drops on the hypervisor side, in a tap device or a vhost queue, are invisible from the guest. A clean ethtool -S in a guest is evidence about the guest, not proof that no packet was lost on the way in.
| Counter | Family | What died | First check |
|---|---|---|---|
rx_crc_errors | wire | frame arrived corrupted | cable, port, transceiver, duplex, MTU |
rx_length_errors, rx_frame_errors | wire | malformed or misaligned frame | MTU mismatch, then the physical path |
rx_crc_errors_phy, rx_symbol_err_phy (mlx5) | wire | FCS or coding errors on the port | link quality, FEC counters |
rx_missed_errors | ring | device FIFO full, packet discarded | ethtool -g, then CPU and IRQ headroom |
rx_no_buffer_count (Intel) | ring | no free host buffers posted yet | ring size before it becomes missed |
rx_fifo_errors, rx_over_errors | ring | legacy synonyms, driver dependent | read the driver before believing either |
rx_out_of_buffer, rx_discards_phy (mlx5) | ring | software or port buffers exhausted | ring size, then port congestion |
rx_queue_0_drops (virtio) | ring | per-queue drop in the guest driver | host side queues, which the guest cannot see |
rx_dropped | above | not processed, mixed causes | usually benign, correlate with real loss |
bw_in_allowance_exceeded (ENA) | above | inbound shaped or dropped at the instance limit | instance size, microbursts |
bw_out_allowance_exceeded (ENA) | above | outbound shaped or dropped | instance size, egress pattern |
pps_allowance_exceeded (ENA) | above | packet rate over the instance limit | small-packet workloads |
conntrack_allowance_exceeded (ENA) | above | connection tracking limit, packets dropped | connection churn, security group state |
linklocal_allowance_exceeded (ENA) | above | rate to DNS, IMDS or Time Sync exceeded | metadata and resolver call rate |
Verified on ethtool 6.1 with the virtio_net driver reporting version 1.0.0, kernel 6.1.155+, Debian 12 bookworm, and on Amazon Linux 2023 with kernel 6.1.163-186.299.amzn2023.x86_64 and the ena driver at 2.16.1g. Counter sets differ by driver and by driver version, so confirm names with ethtool -i before matching them to anything written here.
Why is rx_dropped increasing when nothing is wrong? Because the standard definition is broad. It counts packets received but not processed, which includes unsupported protocols and layer 2 filtering discards such as unknown ethertypes or tagged frames for a VLAN that is not configured. Buffer exhaustion is not supposed to land here, it belongs in rx_missed_errors. Check whether anything is actually losing traffic before treating a slow climb as a fault.
What is the difference between rx_missed_errors and rx_no_buffer_count? Stage. rx_no_buffer_count says the host has not posted free receive buffers fast enough, so packets are waiting in the adapter. rx_missed_errors says the adapter ran out of room and discarded them. The first is a warning, the second is loss. Both point at ring size and at how quickly the CPU returns buffers.
Why does ethtool -S show drops that ip -s link does not? They read different sources. ip reports the kernel's standard interface structure, ethtool -S reports counters the driver defines for itself, and a private counter appears in the standard view only if the driver maps it there. ENA's allowance counters have no standard equivalent at all, so they exist in one view and not the other.
What does bw_in_allowance_exceeded mean on EC2? Inbound traffic went past the instance's aggregate bandwidth allowance and AWS queued or dropped the excess. It counts cumulatively since the last driver reset, so compare two readings rather than reacting to the total. The usual cause is microbursts that per-minute CloudWatch averages hide. The fix is a larger instance or spread load, not NIC tuning.
More from the Field Manual, or work through the training tracks.