Notes: Load Balancing with CEF

Introduction

• IOS 11.1(17)CC introduced CEF for Cisco 7200 and 7500 series routers
• CEF can efficiently use multiple parallel links without additional hardware multiplexers.
• More information: "Alternatives for high bandwidth connections using parallel T1/E1 links."

Load Balancing

• A functionality that distributes packets across multiple links based on routing information.
• If multiple paths to the same destination are discovered, they are installed into the routing table according to the route selection process (discussed previously).
• Only parallel links of the same metric are discussed here (although, the same principles apply over unequal cost paths).
• Load balancing does not have to be supported. A router can use a single path, even if additional equal-cost paths exists (maximum-paths 1).
• By default, a router will try to do load balancing.
• The default in IOS is 4 entries for most IP routing protocols.
• For BGP, the default is only one entry.
• The maximum number that can be configured is 6 different paths.
• Two modes of load balancing: per-destination and per-packet basis.

Per-Destination

• All packets for a given destination are forwarded along the same path.
• Preserves packet order.
• Potential unequal usage of links.
• A larger number of destination addresses leads to more equally used links.
• A route-cache entry for every destination address vs. every destination network as done when only a single path exists.
• Therefore, traffic for two different hosts on the same destination network can use different paths.
• Core routers memory and processing requirements become demanding.

Per-Packet

• Guarantees equal load across all links.
• Packets may arrive out-of-order at the destination (due to differential delay in the network).
• Disables the forwarding acceleration by a route cache, which includes the outgoing interface information.
• Instead, determines the outgoing interface for each packet by looking up the routing table and picking the least used interface.
• Ensures equal link utilization.
• Process-intensive and impacts overall forwarding performance.
• Not well-suited for higher speed interfaces.

Load Balancing with CEF for IP

• Includes enhancements that allow to use load balancing without sacrificing forwarding performance even for per-packet load balancing, which previously required disabling of route caching mechanisms like fast switching or optimum switching.
• Supports encapsulations such as Frame Relay, Ethernet, PPP, HDLC, and tunnels.

How CEF Load Balancing Works

• CEF is an advanced Layer 3 switching technology inside a router.
• The route cache is filled on demand when the first packet for a specific destination needs to be forwarded.
• CEF takes a different approach as it calculates all information necessary in advance.
• CEF also decouples the forwarding information from the next hop adjacency.
• Two main CEF components are:

1. Forwarding Information Base (FIB)
2. Adjacency Table

 

Forwarding Information Base (FIB)

• Used to make IP destination prefix-based switching decisions.
• Conceptually similar to the routing table.
• Maintains a mirror image of the forwarding information of the IP routing table.
• When a routing or topology change occurs, the IP routing table is updated, and those changes are reflected in the FIB.
• One-to-one correlation between FIB entries and routing table entries, so the need for route cache maintenance is eliminated.

Adjacency Table

• Network nodes are adjacent if they can reach each other with a single hop.
• CEF uses the adjacency table to prepend Layer 2 addressing information.
• Maintains Layer 2 next-hop addresses for all FIB entries.
• Populated as adjacencies are discovered.
• Each time an adjacency entry is created (through the ARP protocol, for example), a link-layer header for that node is precomputed and stored in the adjacency table.
• Once a route is determined, it points to a next hop and corresponding adjacency entry.
• For each resolved path to a destination prefix, a pointer is added for the adjacency corresponding to the next-hop interface for that path.
• For per-destination load balancing a hash is computed from the source and destination IP addresses.
• This hash points to exactly one adjacency entry, providing the same path for all packets with this source/destination IP address pair.
• For per-packet load balancing the packets are distributed in round-robin fashion over the available paths.
• The FIB and adjacency tables provide all the necessary forwarding information.

Example:

Destination      Adjacencies    #    Interface    Encapsulation
192.168.25.0/24  34, 45         34   Serial0      PPP
                                45   Serial1      PPP

For each packet a match of the destination address against the FIB is made. In this example the FIB entry has multiple adjacencies for the destination 192.168.25.0, one of the adjacencies needs to be selected. For per-packet load balancing packets are encapsulated and forwarded by alternating between entry #34 and entry #45. For load balancing on IP address basis a hashcode is generated out of the addresses. For any given pair of source and destination addresses the hashcode is always the same, thus guaranteeing that always the same path is used.

CEF Configuration

• Straightforward, as the only configurable option is the load balancing mode.
• The global command to enable CEF is ip cef.
• The command automatically enables CEF on all interfaces that use supported encapsulations.
• The default load balancing mode: per-destination.
• To change the load balancing mode from per-destination to per-packet use the interface command ip load-sharing per-packet.
• Enabling CEF is sufficient in most cases.
• Per-packet load balancing is only rarely needed, e.g. when the majority of traffic is between two hosts.

Summary

• CEF supports load balancing for TCP/IP over parallel links without impacting performance even if the traffic patterns require per-packet load balancing.

References

Load Balancing with CEF