Unlocking Traceroute: A Deep Dive into Network Path Analysis and Common Pitfalls

Overview

1.1 What is Traceroute

Traceroute is a tool that detects the number of hops between a source host and a destination host by displaying the IP addresses of intermediate routers.

Three main characteristics of Traceroute:

Cross‑platform : Available on macOS, Windows, Linux, Android, iOS and other operating systems.

Easy to use : Simply append an IP address or domain name to the Traceroute command.

Comprehensive information : Shows hop count, packet loss, latency and other metrics.

However, using Traceroute alone does not guarantee that network problems can be identified.

1.2 Reasons Traceroute may fail to pinpoint issues

For professionals, improved carrier network quality reduces the probability of simple faults; most problems are complex and cannot be diagnosed with a single Traceroute. For non‑professionals, limited understanding of Traceroute and abundant false reports make accurate diagnosis difficult.

Traceroute Principles

2.1 How Traceroute works

Send a probe packet from SRC to DST with TTL=1.

TTL decrements at each hop; when TTL reaches 0 the router discards the packet and returns an ICMP "TTL Exceeded" message.

SRC records the information from the ICMP reply.

Repeat steps 1‑3, increasing TTL by 1 each round, until DST returns an ICMP "Destination Unreachable" message.

2.2 Implementation details

Traditional UNIX uses UDP probes (default port 33434, incremented each hop); Windows "tracert" and MTR use ICMP Echo Request.

If DST does not send an ICMP "Destination Unreachable" (e.g., due to firewalls), the trace cannot reach the destination.

Traceroute can be performed with UDP, TCP or ICMP packets; TCP is rarely used because many routers filter it.

Typical Traceroute sends three probes per hop; missing replies are shown as "*". MTR sends probes continuously.

Each probe carries a unique identifier (incremented port number for UDP/TCP, sequence number for ICMP).

Under ECMP, different paths may be observed; under LAG, paths are not visible.

2.3 Latency calculation

Traceroute can be used to compute round‑trip time (RTT) by timestamping the probe when sent and the ICMP reply when received.

Note 1: Routers simply forward packets; they do not process timestamps. Note 2: The displayed latency is the RTT of the forward path, not the return path.

2.4 How each hop is generated

SRC sends a packet with TTL=1 to Router 1.

Router 1 decrements TTL, discards the packet when TTL=0, and sends an ICMP "TTL Exceeded" back to SRC.

SRC displays the hop information (e.g., 172.16.2.1, 10.3.2.2).

The return path of the ICMP packet is not visible to Traceroute.

Thought and verification: RFC 1812 requires the source address of an ICMP message to be one of the router's physical interface IPs. In practice, some routers send the ICMP reply out a different interface, which can violate the RFC.

Using DNS Reverse Lookup in Traceroute

Reverse DNS can reveal router location, interface type, bandwidth, routing role, and autonomous system relationships, all of which are valuable for fault diagnosis.

3.1 Router geographic location

Knowing the location helps optimize routes, compare expected latency, and build network topology maps.

3.2 Interface type and bandwidth

DNS often encodes interface information (e.g., "xe-11-1-0.edge1.NewYork1.Level3.net" indicates a Juniper 10 GE port). This data may be outdated but can aid in identifying equipment models.

3.3 Router role

Common naming conventions: Core routers (CR, GBR, CCR), Peering routers (BR, Edge, Peer), Customer routers (AR, Cust, GW).

3.4 Autonomous System boundaries

Reverse DNS helps locate AS boundaries, understand policy changes, and identify which party to contact for issues.

Network Latency Types

Serialization delay : Packet size divided by link speed (e.g., 1500 bytes over 1 Gbps ≈ 0.012 ms).

Queueing delay : Time a packet waits in a router's buffer; negligible when utilization is low, but grows sharply near saturation.

Propagation delay : Determined by the speed of signal in the medium (≈200 000 km/s in fiber); 1 ms RTT corresponds to ~100 km distance.

Traceroute Delay Analysis

5.1 Factors affecting Traceroute latency

Traceroute latency consists of three components: time for the probe to reach a router, router processing time to generate the ICMP reply, and time for the reply to return. Only the first and third are network‑dependent; the second can be misleading due to router CPU load.

5.2 Router operation

Routers have a fast‑path (hardware) for data‑plane forwarding and a slow‑path (software) for control‑plane tasks such as ICMP generation. Limited CPU resources and vendor‑specific ICMP rate limits (e.g., 400 pps per interface) can cause artificial latency spikes.

5.3 Eliminating false latency

If a particular hop shows high latency, subsequent hops will usually exhibit equal or greater latency. Consistent spikes often indicate router rate‑limiting or asymmetric return paths.

Asymmetric Forwarding Paths

Forward and reverse paths may differ, making Traceroute only show the forward direction. Running a reverse Traceroute can help identify issues on the return path.

6.1 Asymmetry at AS boundaries

Asymmetric paths often begin at AS borders due to differing routing policies, leading to congestion or latency on one direction only.

6.2 Multipath interconnection

Multiple parallel paths can cause later hops to appear faster than earlier ones when one path is congested.

6.3 Verifying the problem location

Using /30 mask techniques to force the return path through a specific provider can confirm whether the issue lies on that provider's network.

Equal‑Cost Multipath (ECMP) Effects

7.1 ECMP routing

Flow‑based hashing can cause Traceroute probes (which increment destination ports) to follow different paths, producing seemingly inconsistent hop sequences.

7.2 Unequal‑length ECMP

ECMP may result in varying hop counts, making the trace appear to jump or loop.

7.3 Forcing a single path

Traceroute options (e.g., fixing the destination port with

-U

and

-p

) can keep probes on the same path; varying the target IP can explore alternative paths.

MPLS Considerations

8.1 MPLS ICMP tunneling

When routers forward ICMP packets based on MPLS labels, the ICMP reply may travel through the MPLS tunnel and emerge at the egress, causing identical latency values for multiple hops.

<code>te2-4.ar5.PAO2.gblx.net (69.22.153.209) 1.160 ms 1.060 ms 1.029 ms
192.205.34.245 (192.205.34.245) 3.984 ms 3.810 ms 3.786 ms
...</code>

Conclusion

Traceroute contains many layers of networking knowledge; despite its limitations, advanced tools like Paris Traceroute can provide more accurate topology discovery for troubleshooting.

References

<code>1. A Practical Guide to (Correctly) Troubleshooting with Traceroute
2. Avoiding traceroute anomalies with Paris traceroute
3. RFC1812
4. 百度百科
5. 维基百科</code>