Published 2026-10-17 · LuckyMDM Blog
Short answer: a queued command is a row in a server-side table, not work already running on a device. Three things decide how long the acknowledgement takes — whether the device can be reached at all, whether it just restarted and is refetching, and whether another bulk job is occupying the same window. None of those three is a failure, and treating them as failure is what fills a fleet ledger with duplicate rows.
The pattern looks like this in most fleets. An operator pushes a bulk action to 100 units at 9:00. By 9:30 only a handful of receipts have landed. Somebody declares the push failed, resends to the missing units, and goes to lunch. By 13:00 every receipt has arrived — including the second copy of every row that was resent. Nobody is harmed, but the command ledger now contains two versions of the same event, and reconciling them later costs more than waiting would have.
What makes this hard to diagnose is that the delay observations are real: some units really did return receipts much later than the fleet's normal timing. The question is which clock you compare against, and most teams compare against nothing at all.
A command that has been created on the server waits there. The device learns about it through the push channel, then connects back and collects pending work, then executes, then reports back. Each of those steps can introduce delay, and none of them writes an error anywhere. Most fleets configure check-in intervals anywhere between 15 minutes and 4 hours, so the first class of delay is bounded by roughly one interval — a unit that is simply asleep contributes delay without ever failing.
Reachability delay. The unit is powered down, out of coverage, or on a network that blocks the management channel. Its last check-in is older than the fleet average, and the receipt arrives on its next wake. Signature: a long gap between the queue row and any device activity at all.
Restart refetch delay. A reboot causes the device to re-evaluate its policy and its pending queue. Receipts produced in this window land later than normal — typically tens of minutes rather than minutes — because refetching competes with everything else the device has to do on the way back up. Signature: the receipt sits unusually close to a recorded restart event.
Contention delay. An update deployment wave and a bulk command overlap in the same window, or several hundred units sit behind one store router. The queue drains, but slowly, and the tail stretches. Signature: nothing wrong per unit, yet the completion time is a distribution spread over 60 to 90 minutes rather than a single moment.
Each class separates cleanly once you keep three timestamps per command: when the queue row was created, when the device last checked in before the receipt, and — if you collect it — when the device last restarted. Restart sitting within a few minutes of the receipt points at the refetch window. A last check-in older than the fleet's interval points at reachability. Both timestamps looking healthy while the whole batch is late points at contention. Without the restart field, the second class becomes invisible and everything gets blamed on the network, because that is the explanation that requires the least evidence.
Two neighbouring problems get folded into this one, so the line is worth drawing: choosing when to send a batch is a scheduling question, and sorting receipts into categories is a ledger question. This page covers neither. It is about explaining delay that has already been observed, before anyone decides it failed.
| Class | Evidence to look for | Typical magnitude | Wrong action to take | Right action |
|---|---|---|---|---|
| Reachability | Last check-in older than one interval; no activity at all | Up to one check-in interval, often 15 minutes to 4 hours | Resending immediately | Wait for the next check-in; mark the unit as unreachable, not failed |
| Restart refetch | Receipt within tens of minutes after a recorded restart | Tens of minutes beyond the fleet median | Treating it as a device fault | Schedule bulk jobs away from known restart batches |
| Contention | Fleet-wide tail stretched; per-unit timestamps healthy | 60 to 90 minutes across a large batch | Raising a defect ticket per device | Split the batch; compare completion as a distribution |
Export 20 entries whose receipts arrived later than your usual window. For each one, write down the queue creation time, the previous check-in time, and the last restart time if you have it. Classify each entry against the table above. In most fleets this exercise takes about 40 minutes and resolves the majority of "failed" pushes without touching a device.
Then look for the second signal: how many resends happened after the original row had already executed. Count them. If a meaningful share of the late entries got a duplicate row, your retry policy — not your devices — is what generated most of the noise.
Take the example above at face value: 60 units treated as failures and resent, at 3 minutes of operator time each, is 180 minutes, about 3 hours, roughly 108 USD at 36 USD an hour. The visible cost is small. The invisible cost is larger: 60 extra rows enter the command ledger, each one needing reconciliation during the next audit, and every duplicate makes it harder to prove what actually happened on a specific unit on a specific date. Compare that with scheduling bulk work one hour away from a known restart batch, which costs nothing beyond a calendar entry.
LuckyMDM (Sichuan Starlight Network LLC) stores queue creation time, device-side execution time, and the last recorded restart time for every command, and it distinguishes queued from acknowledged from executed rather than collapsing them into success and failure. That separation is the only thing that makes the three delay classes visible after the fact.
Why it is wrong: a queued row is not evidence of execution in either direction. Until the device checks in, absence of a receipt carries no information at all, which is precisely why the queue state and the receipt state need to be kept as two different fields.
Why it is wrong: a restart inserts another refetch cycle, and refetch windows push receipts later, not earlier. Restarting a unit whose only problem was a slow batch adds delay of the second class on top of the third.
Why it is wrong: contention is a property of the window, not of the units. Two modest jobs launched together across 300 devices produce a longer tail than either job alone, and then every individual metric looks fine.
Boundary: genuine failure exists and looks different. A returned error state, a permission that no longer applies, or a device whose enrolment has been released are not delay classes. This article is not applicable to those cases; they need the failure path, not patience.
Boundary: these delay classes are bounded. When a latency exceeds several check-in intervals, or the restart field cannot explain it, stop classifying and start diagnosing — there is a real fault somewhere, and continuing to explain it away is how small problems become expensive ones.
Anchor it to your own interval rather than to a round number. Anything past one configured check-in interval deserves attention; anything past two deserves investigation; anything past three is probably not a delay class.
Yes, if you can. It costs one field per record and it is the only evidence that separates the second class from the third. LuckyMDM carries it per command for exactly this reason: without it, refetch windows get misattributed to the network.
It helps when the original row genuinely failed — an error state, a permission change, a released enrolment. It does not help when the row is still queued, which is why the queue state matters before you press the button again.
Yes. Keeping them at least one hour apart costs nothing and removes the third class almost entirely, which is cheaper than any tooling change.
The mechanism applies broadly; the magnitudes here come from fleets where check-in is measured in minutes to hours. Fleets running very short intervals will see compressed versions of all three classes, and should re-derive their own thresholds rather than copying these numbers.