Verify the IP routing table

The example in the previous section was based on a directly connected device. To illustrate verifying the IP routing table and subsequent CEF troubleshooting, Example 4-5 is based on a router with several routes, as illustrated in Figure 4-3. Devices connected to Router 2 in the 10.1.1.0/24 subnet are unable to reach a Secure File Transfer Protocol (SFTP) server with the address 10.18.118.184, as shown in the first steps of Example 4-5; specifically, the host 10.1.1.100 is unable to ping 172.18.118.184. For the purpose of this example, assume that all the IP routing configuration and host configurations, such as IP address and default gateways, are configured correctly. In addition, assume that our issue is strictly limited to Router 2 and not Router 1, the Layer 3 switch.

Figure 4-3 Troubleshooting IP Connectivity and Verifying the IP Routing Table

Example 4-5 IP Connectivity Issue from the Host, 10.1.1.100, to the Target, 172.18.118.184, Based on Figure 4-3

The first step in troubleshooting is to access the first-hop router, Router 2, and verify IP connectivity to the SFTP server, 10.18.118.184. Example 4-6 also confirms the route to the host, 10.18.118.184, and verifies the ARP entry for the next hop for 10.18.118.184, which is 10.18.114.1. The host entry, 10.18.118.184, is known through a static route with a metric of 0 according to the show ip route command. This is the gateway of last resort.

Example 4-6 Verifying IP Routing Table and Next-Hop Information

Although the ping from the router was successful, the next step is to verify the MAC (Hardware Addr) of the next-hop router. This step verifies that the correct ARP entry exists for the next-hop router. Example 4-7 illustrates obtaining Router 1's MAC address for the interface configured with the IP address 172.18.114.1.

Example 4-7 Verifying the MAC Address of an Interface in Cisco IOS

The next-hop router's (Router 1's) MAC address is indeed 0008.a37f.cb7c. Therefore, at this point in troubleshooting, both the routing entry and ARP entry for the next-hop router are correct.

The next step is to verify IP connectivity to the next-hop router, Router 1, from both the host, 10.1.1.100, and Router 2, as shown in Example 4-8.

Example 4-8 Verifying IP Connectivity to the Next Hop from Router 2 and the Host, 10.1.1.100

Example 4-8 also issued an ICMP ping of a larger packet size for continued verification of accessibility of the next-hop router. You must verify various packet sizes because defaultsized packets of ICMP echoes can be routed and switched correctly in the network, whereas larger packets can be dropped for a multitude of reasons. These reasons include encoding errors in WAN circuits, fragmentation configuration issues, hardware issues, Virtual Private Network (VPN) misconfiguration, software features such as encryption and compression, and so on. Some network routers' security policies can also prevent the routers from responding to very high rates of ICMP packets. Therefore, you should understand your network policies when troubleshooting or installing systems. In Example 4-8, the router is indeed reachable through ICMP.

Example 4-8 proved that the next hop is reachable from both Router 2 and the host, 10.1.1.100. Therefore, at this point, the ARP and IP routing tables appear to be correct on Router 2. The next step is troubleshooting CEF (Step 6), which is discussed in the next section using the same example as that in Figure 4-3.

Sending ICMP echoes, requests, or responses from the CLI of the router is software switched, while ICMP echoes from end devices through many Cisco IOS routers and all current-generation Catalyst switches are hardware switched. The software-switching path might be correct, but the hardware-switching path might not be correct. In this example, the software-switching path on Router 2 might be correct, but the hardware-switching path might not be correct. Troubleshooting the hardware-switching path is outside the scope of this chapter. Chapter 5 discusses troubleshooting hardware-switching paths on a Cisco Catalyst 6500 platform. However, you must troubleshoot CEF from the software-switching perspective first because the hardware-switching tables are built from the softwareswitching tables.

The section "Troubleshooting the CEF FIB Table," later in this chapter, continues the investigation of the software-switching path by troubleshooting the Cisco IOS CEF table on Router 2 for Figure 4-3. As noted previously, because the hardware-switching path gets built from the software-switching CEF and adjacency table, you should investigate the software-switching path first.

Using IOS Ping with the Record Option to Rule Out CEF

CEF does not support all IP packet types and must process-switch specific types of packets. One such packet includes the ICMP echo with the record option. As such, you can rule out CEF as a cause of an IP connectivity problem with some certainty using the ICMP echo with record option.

For example, in Figure 4-3, sending an ICMP echo with the record option forces all routers along the path to use the process-switching method of forwarding a frame. If an ICMP echo with the record option is successful and a standard ICMP echo is not, you can assume with some certainty that CEF is indeed a cause of your IP connectivity issue somewhere along the path.

In Cisco IOS, use the ping command with extended commands option to send ICMP echoes with the record option. Example 4-9 illustrates the use of the ICMP echo with record option.

Example 4-9 Sending ICMP Packets Using the Record Option

NOTE When troubleshooting CEF, always note that certain packets, mainly those requiring special handling, are not CEF switched.

Learn how to troubleshoot Cisco's Express Forwarding network switching technology in "Basic IP Connectivity and CEF Troubleshooting," Chapter 4 from the book Cisco Express Forwarding by Nakia Stringfield, Russ White and Stacia McKee.

Basic IP Connectivity and CEF Troubleshooting

  Introduction
  Accurately describe the problem
  Scope the network topology
  Review the OSI model for troubleshooting
  Verify the ARP table
  Verify the IP routing table
  Verify the CEF FIB table
  Verify the adjacency table
  Conduct hardware-specific troubleshooting

Reproduced from the book Cisco Express Forwarding. Copyright 2007, Cisco Systems, Inc. Reproduced by permission of Pearson Education, Inc., 800 East 96th Street, Indianapolis, IN 46240. Written permission from Pearson Education, Inc. is required for all other uses.