Diagnoses and Treatment of Black Hole Routers

ID: Q159211

The information in this article applies to:

SUMMARY

On a TCP/IP wide area network (WAN), communication over some routes may fail if intermediate network segments have packet sizes smaller than the communicating hosts, and routers do not send appropriate ICMP responses to this condition. A router that causes this condition is sometimes known as a "black hole" router. The Ping utility, a standard utility installed with the Microsoft Windows NT TCP/IP protocol, can be used to find black hole routers. Some recommendations are provided to work around or fix problems with black hole routers.


MORE INFORMATION

When a network router receives a packet larger than the Maximum Transfer Unit (MTU) of the next network segment, and that packet's IP layer "don't fragment" bit is flagged, the router should send an ICMP destination unreachable message back to the sending host. When this does not happen, packets can be dropped, causing a variety of errors that will vary with the application that is communicating over the failed link. These errors will not occur when an application connects to a computer on a local subnet. The problem may seem intermittent, but on closer examination, it can be duplicated, such as in having a client read a large file from a remote host.

The Ping utility can be used to find black hole routers by using the -f (do not fragment) and the -l (buffer size) parameters. Setting the -f parameter will cause the Ping utility to send an ICMP echo packet with the IP "do not fragment" bit set. The -l parameter sets the buffer, or payload, size of the ICMP Echo packet. The largest buffer that can be sent unfragmented equals the MTU minus the IP and ICMP headers (MTU-28) of the smallest MTU along a route. For example, because Ethernet has an MTU of 1500 bytes, under the best circumstances, Ping could echo an unfragmented packet with an ICMP buffer of 1472 bytes. The syntax for Ping in this case would be:

Ping <computer name or IP address> -f -l 1472

This should work on all local IP addresses. If the MTU of all segments of a routed connection are 1500 or larger, the packet should be returned as well. If there are intermediate segments with smaller MTUs, and routers return the appropriate ICMP Destination Unreachable packet, the utility should display "Packet needs to be fragmented but DF set". If there are segments along the route with smaller MTUs, and the appropriate ICMP packet is not returned, the Ping utility should display "Request timed out." The default MTUs of common network media are described in Knowledge Base article Q140375.

By changing the -l parameter on successive Pings, the largest unfragmented packet that will travel a specific route can be found. The smallest MTU in general use is 576 bytes, so you should be able to safely start with an ICMP buffer of 548, then work up from there. For example, if Ping <host name or address> -f -l 972 returns packets and Ping <host name or address> -f -l 973 fails, the largest MTU that can be used over that route is 1000 (972+28).

To fix or work around black hole routers, there are four possible solutions:

  1. Enable PMTU Black Hole Detection on Windows NT hosts that will be communicating over a wide area connection, as documented in Knowledge Base article Q136970. In this case, Windows NT 3.51 Service Pack 2 or greater or Windows NT 4.0 should be used.


  2. Configure intermediate routers to send ICMP type 3 code 4 (destination unreachable don't fragment (DF) bit sent and fragmentation required) messages. This may require upgrading router software or firmware, router configuration or router replacement.


  3. Disable PMTU discovery on Windows NT hosts that communicate over troublesome routes. This will configure the default MTU to 576 bytes. This could cause significant degradation in network performance.


  4. Set the MTU of the host interface to be the largest the black hole router can handle. This guarantees the largest possible packet size will be sent over that connection, but will cause local traffic, and traffic over routed connections without problems, to use smaller packets than they would otherwise. This workaround assumes that you have determined the MTU and the state of all possible links that could be used by the host in question.


For additional information, please see the following articles in the Microsoft Knowledge Base:
ARTICLE-ID: Q120642
TITLE : TCP/IP & NBT Configuration Parameters for Windows NT 3.5

ARTICLE-ID: Q128797
TITLE : Unable to Transfer Files Across DEC 250 and DEC 500 Routers

ARTICLE-ID: Q136970
TITLE : PMTU Black Hole Detection Algorithm Change for Windows NT 3.51

ARTICLE-ID: Q138575
TITLE : Communication Fails Through Ethernet Segment Between FDDI Rings

ARTICLE-ID: Q140375
TITLE : Default MTU Size for Different Network Topology

For further information, see Internet RFC 1191 and RFC 1435, available from http://www.internic.net/ and Microsoft Windows NT 3.5/3.51: TCP/IP Implementation Details, a white paper available from http://www.microsoft.com/.

Additional query words: prodnt 


Keywords          : kbnetwork nthowto nttcp 
Version           : WinNT:3.5,3.51,4.0
Platform          : winnt 
Issue type        :

Last Reviewed: January 29, 1999