For many of us, when we suspect there may be an issue with our internet connectivity, we reach for the trusty ping utility provided by our operating system and type:

ping 8.8.8.8

This is a great first troubleshooting step that asks “can I reach Google DNS?” The command sends Internet Control Message Protocol (ICMP) echo request to the destination (8.8.8.8) and measures the time it takes from sending the request to getting a response to measure your latency. If the pings fail (or latency is very high or some higher percentage of packets are lost), it suggests there may be a connectivity problem worth investigating further.

Yet, as basic and fundamental as this sounds, why does ICMP ping often not correlate with our actual experience using real-world applications? Why can you get 15ms latency to 8.8.8.8 but your video calls continue to stutter? Or why is ping showing significant packet loss but your network seems to be working fine? To understand this, it is helpful to carve the architecture of the internet up into various functional “layers” and see where ICMP falls.

In reality, things do not always fit neatly into individual layers and can span multiple (e.g. the QUIC protocol spans both the Transport and Security layers), but this is a helpful exercise conceptually and a common practice in academia.

What’s important to observe here is that ICMP messages are actually encapsulated inside IP packets and are integral to IP’s operation. Therefore, ICMP operates at the Network Layer in our table above. The everyday applications we use, such as web browsing, social media, online gaming, real-time voice, video conferencing, and video streaming, all operate at the Application Layer, a much higher level of abstraction than the Network Layer.

The Security Layer of the internet exists between the Network Layer (where ICMP resides) and the Application Layer. In practice, real-world applications access cryptographically verified resources. You have certainty you're "talking" to the right service. ICMP operates below the Security Layer, so there's no cryptographic certainty. Any networked appliance between you and your destination can respond on behalf of that device without your knowledge. As a result, real-world application responsiveness and reliability does not always align with what ICMP measures, even to the same destination.

Given that ICMP is not used directly to service any of the applications we use, and is not cryptographically verified traffic, is utilizing ICMP really the best available approach for measuring internet application responsiveness and reliability?

The problems with using ICMP to measure responsiveness and reliability

The unfortunate truth is that while ICMP-based utilities are the most popular and most utilized tools by systems administrators, performance monitoring tools, and network benchmarking applications for measuring responsiveness and reliability, they can provide misleading results that do not reflect real-world experience. These tools can often under-report latency, over-report latency or provide false-negatives and false-positives regarding your ability to access services on the wider internet.

Under-reported latency

ICMP-based latency measurements can often paint a rosier picture than reality. There are a myriad of reasons that this is the case.

ICMP traffic is often prioritized separately from other traffic. While your actual traffic to Microsoft Teams or Fortnite may be stuck in a queue on a router between your device and the server, ICMP echo requests may be sailing through. Many network devices “fast-path” process ICMP traffic in hardware, dedicated ASICs, or via XDP/eBPF, while real application traffic is “stuck” in user space processed by slower SoCs or held behind in queues and buffers.

Some network equipment responds to ICMP locally without forwarding the packet to the actual destination. Middleboxes, proxies, or carrier equipment might intercept your ping and reply themselves, showing excellent latency to what you think is the remote host while your actual application traffic still has to traverse the full path with all its real delays. For example, the Colima project, a popular container runtime, always has the local host reply to ICMP echo requests on macOS regardless of which internet endpoint you think you are pinging.

For many networks, transfer rates are not only impacted by physical limitations, but also by traffic policies and rate limits enforced by traffic shapers. Traffic shapers often exempt ICMP from rate limiting and shaping policies. Your actual TCP connections might be throttled to certain bandwidth limits, experiencing associated latency, while ICMP bypasses these controls entirely.

Over-reported latency

Conversely, using ICMP to measure latency can sometimes provide values that are much higher than responsiveness to real applications using transport and application layer protocols.

While many routers and networks prioritize or fast-track ICMP traffic as previously discussed, others actively deprioritize ICMP traffic in their queuing systems or implement ICMP rate-limiting. During congestion, ICMP packets get delayed or dropped first while TCP/UDP traffic flows normally. This means your ping measurements can show terrible latency and packet loss while your actual applications perform fine - or it can mask real congestion affecting your apps. This has been reported many times for specific router make and models in the Orb Discord community.

Connectivity false-positives and false-negatives

If you're using ICMP ping to determine whether you have internet access for your everyday applications, the result may be misleading.

Ping succeeds, but you can't browse

You might be on an airplane, at a coffee shop, or at a hotel and successfully ping 8.8.8.8. But you could be behind a captive portal, have service limited to Wi-Fi text messaging, or have an active VPN connection that completes ICMP pings while all HTTP traffic fails.

Your router or local host may be responding to your pings and making it seem like the internet is working, but there's a real problem with your connection to the internet.

Ping fails, but the internet works fine

Many networks and destinations intentionally block or deprioritize ICMP traffic. Firewalls, routers, and servers often drop ICMP packets as a security measure. A failed ping doesn't necessarily mean the service is unreachable—it might just mean ICMP is blocked.

Ping takes a different path

ICMP packets may take different routes through the network than actual application traffic. A successful ping doesn't guarantee your web browser or application will work, since those protocols might encounter issues or traffic shaping that ICMP doesn't.

No cryptographic verification

Because ICMP operates below the Security Layer of the internet, there's no cryptographic certainty that the response is coming from the endpoint you're attempting to measure. Sometimes you want to confirm there's a ping responder somewhere in the network, but you have no way to verify it's the one you intended to reach.

Given these limitations, how should we approach measuring what users actually experience?

What Orb is doing about it

We want to ensure Orb remains the best-available tool for helping everyone, regardless of their level of technical competency, to understand the Responsiveness and Reliability of the internet from the devices they care about most. Orb measures Responsiveness and Reliability by having a set of configured endpoint (Cloudflare, Fastly, etc.) and protocol (HTTP/3, ICMP, etc.) combinations available, testing them, and keeping the two best-performing measurement instruments active. Two are always kept active to provide failover should the upstream service encounter an issue.

Beginning with Orb 1.4 applications and sensors, the default behavior will be to measure Responsiveness and Reliability exclusively with application layer, cryptographically verified sources to most accurately measure real-world latency and internet reliability. Moving forward, the default behavior will be for ICMP measurements to be disabled to ensure measurement traffic is not intercepted and manipulated by intermediary routers and packet shapers.

Should you wish to continue to utilize Orb’s ICMP measurements, or wish to enable them for a specific use case such as validating Responsiveness when you are using in-flight Wi-Fi texting with no expectation of broader internet connectivity, you can now explicitly re-enable ICMP measurements in the Orb apps for version 1.4 or above.

When you select “Include ICMP”, you will see an “!” icon on the Responsiveness and Reliability cards. You can tap or click them for more details.

In addition to enabling ICMP in the Orb app, you can also do so via Orb Cloud configurations.

Since launching Orb in late April, 2025, we’ve been thrilled to continue the work of countless others before us to raise broader consumer awareness about the importance of Responsiveness. But tools that purport to measure your experience are only trustworthy when the values they report correlate with your actual, perceived, experiences. This small yet important change to Orb’s default behavior is another step on our path of making Orb the best tool for accurately measuring internet Responsiveness and Reliability.