Explanation: TunnelX temporarily disabled due to recursive routing

I wanted to take a few minutes to share a scenario that some seem to struggle with. This scenario is a routing issue that sometimes occurs when an interior routing protocol allows routes to leak back through a tunnel. To demonstrate this, I’ve built a lab with three routers. R1 and R3 are participating in EIGRP and have a GRE tunnel configured directly between them.

Topology

TunnelRecurse

 

Router Configurations

R1

hostname R1
!
interface FastEthernet0/0
 ip address 192.168.12.1 255.255.255.0
!
interface Tunnel0
 ip address 192.168.13.1 255.255.255.0
 tunnel source 192.168.12.1
 tunnel destination 192.168.23.3
!
router eigrp 1
 network 192.168.0.0 0.0.255.255
!
ip route 0.0.0.0 0.0.0.0 192.168.12.2

R2 (hub)

hostname R2
!
interface FastEthernet0/1
 ip address 192.168.12.2 255.255.255.0
!
interface FastEthernet0/1
 ip address 192.168.23.2 255.255.255.0
!

R3

hostname R3
!
interface FastEthernet0/1
 ip address 192.168.23.3 255.255.255.0
!
interface Tunnel0
 ip address 192.168.13.3 255.255.255.0
 tunnel source 192.168.23.3
 tunnel destination 192.168.12.1
!
router eigrp 1
 network 192.168.0.0 0.0.255.255
!
ip route 0.0.0.0 0.0.0.0 192.168.23.2

At first glance, this seems to be a valid configuration. Routers R1 and R3 can reach one another using the static default route. This should establish reachability between the two devices and allow the tunnel to come up. As a matter of fact the tunnel DOES come up and even allows EIGRP to form an adjacency. However, traffic doesn’t pass consistently and the adjacency is subsequently lost.

Bouncing the interfaces only seems to allow them to come online for a short period of time. Routers R1 and R3 cycle through the same process and produce the following logs with each iteration.

*Mar  1 00:05:00.095: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up
*Mar  1 00:05:22.843: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 192.168.13.3 (Tunnel0) is  up: new adjacency
*Mar  1 00:05:39.871: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 192.168.13.3 (Tunnel0) is down: holding time expired
*Mar  1 00:06:38.575: %TUN-5-RECURDOWN: Tunnel0 temporarily disabled due to recursive routing
*Mar  1 00:06:39.575: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to down
*Mar  1 00:06:39.583: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 192.168.13.3 (Tunnel0) is down: interface down

Explanation

I see this issue come up pretty regularly and the phenomenon introduces some confusion. The problem has to do with the routes between the tunnel source and destination. Before EIGRP converges, the best (and only) route between R1 and R3 is the static default route assigned by the administrator. Let’s examine what happens when the EIGRP adjacency is established between the two routers.

R1#debug ip routing
R1#configure terminal
R1(config)#interface tunnel 0
R1(config-if)#shut
R1(config-if)#no shut
R1(config-if)#
*Mar  1 00:12:01.775: RT: is_up: Tunnel0 1 state: 4 sub state: 1 line: 0 has_route: False
*Mar  1 00:12:01.779: RT: SET_LAST_RDB for 192.168.13.0/24
  NEW rdb: is directly connected
*Mar  1 00:12:01.779: RT: add 192.168.13.0/24 via 0.0.0.0, connected metric [0/0]
*Mar  1 00:12:01.779: RT: NET-RED 192.168.13.0/24
*Mar  1 00:12:01.783: RT: interface Tunnel0 added to routing table
*Mar  1 00:12:03.767: %LINK-3-UPDOWN: Interface Tunnel0, changed state to up
*Mar  1 00:12:03.767: RT: is_up: Tunnel0 1 state: 4 sub state: 1 line: 0 has_route: True
*Mar  1 00:12:04.055: RT: NET-RED 0.0.0.0/0
*Mar  1 00:12:04.767: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up
*Mar  1 00:12:04.767: RT: is_up: Tunnel0 1 state: 4 sub state: 1 line: 0 has_route: True
*Mar  1 00:12:34.443: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 192.168.13.3 (Tunnel0) is up: new adjacency
*Mar  1 00:12:39.491: RT: SET_LAST_RDB for 192.168.23.0/24
  NEW rdb: via 192.168.13.3

*Mar  1 00:12:39.491: RT: add 192.168.23.0/24 via 192.168.13.3, eigrp metric [90/297270016]
*Mar  1 00:12:39.495: RT: NET-RED 192.168.23.0/24

Recalling the configuration of R1, the tunnel destination is 192.168.23.3. After Tunnel0 comes up, the EIGRP adjacency is established and routes exchanged. As is shown above, R1 now has a new route to 192.168.23.0/24. This route is actually THROUGH the tunnel to 192.168.13.3. To reiterate the point, the tunnel destination would be reached through itself.

At this point, the logic would be as follows:

  1. R1 routes a packet out Tunnel0
  2. Tunnel0 adds a GRE and additional IP Headers
  3. Outer packet header has a source of 192.168.12.1 and destination of 192.168.23.3
  4. Router needs to route new packet to 192.168.23.3
  5. Route table indicates 192.168.23.3 should be sent out Tunnel0
  6. Tunnel0 adds a GRE and additional IP Headers
  7. Outer packet header has a source of 192.168.12.1 and destination of 192.168.23.3
  8. Go to step 4

This is a condition that would continue an infinite number of times. Fortunately the router recognizes this as recursion and doesn’t allow it to continue.

Resolution

There are more than one ways to resolve this type of issue. One way would be to simply configure a static host route between R1 and R3.

R1

ip route 192.168.23.3 255.255.255.255 192.168.12.2

R3

ip route 192.168.12.1 255.255.255.255 192.168.23.2

This would be a better route than what is learned through EIGRP.

Another option might be to tighten up the network statements so there is no advertisements of the physical interfaces used to facilitate the GRE tunnels.

R1

router eigrp 1
 no network 192.168.0.0 0.0.255
 network 192.168.13.0

R3

router eigrp 1
 no network 192.168.0.0 0.0.255
 network 192.168.13.0

A final option might be to use a separate VRF for the physical interfaces. This would provide isolation between the physical and tunnel interfaces.

R1

ip vrf FVRF
!
interface FastEthernet0/0
 ip vrf forwarding FVRF
 ip address 192.168.12.1 255.255.255.0
!
no ip route 0.0.0.0 0.0.0.0 192.168.12.2
ip route vrf FVRF 0.0.0.0 0.0.0.0 192.168.12.2
!
interface tunnel 0
 tunnel vrf FVRF

R3

ip vrf FVRF
!
interface FastEthernet0/1
 ip vrf forwarding FVRF
 ip address 192.168.23.3 255.255.255.0
!
no ip route 0.0.0.0 0.0.0.0 192.168.23.2
ip route vrf FVRF 0.0.0.0 0.0.0.0 192.168.23.2
!
interface tunnel 0
 tunnel vrf FVRF

Conclusion

The concept of recursive routing through a tunnel is something most of us have witnessed at one time or another. When I’m describing it, I sometimes use the analogy of a snake eating its own tail. If the route a tunnel takes is through the tunnel itself, a logic loop exists and must be broken.

Understanding the problem requires thinking through the process from two perspectives. There are transit packets that use the tunnel. Additionally, the tunnel itself creates packets used to carry the transit packets. When tunnel packets are routed back through the tunnel, a serious problem exists and the tunnel is rendered unusable.

About Paul Stewart, CCIE 26009 (Security)

Paul is a Network and Security Engineer, Trainer and Blogger who enjoys understanding how things really work. With over 15 years of experience in the technology industry, Paul has helped many organizations build, maintain and secure their networks and systems.
This entry was posted in Certification, General, Network, Technology and tagged , . Bookmark the permalink.

2 Responses to Explanation: TunnelX temporarily disabled due to recursive routing

  1. reaper81 says:

    Great post, Paul. This was one of the things I found confusing when I started out my CCIE studies.

  2. that1guy15 says:

    Interesting idea on the VRF for tunnels. I might need to play with that a little for my studies. Im a big fan of KISS for as much as possible so I prefer to not advertise PTP links or tunnels into IGPs unless there is a good reason too.

    Who needs to talk with a PTP segment anyways?

Leave a Reply