Design Quick Notes: BGP

Introduction / Summary

• BGP split horizon rule: do not advertise prefixes learned from another iBGP router.
• Full mesh iBGP does not scale.
• As tmore BGP routers are added, the number of sessions quickly increase.
• Solution: use route reflectors or confederations.
• Route Reflectors allow a router to reflect routes learned from one iBGP peer to another iBGP peer. The concept is similar to the Designated Router in OSPF.
• Confederations build eBGP sessions inside your AS.

iBGP Scalability Issues

• Because of the BGP split horizon rule, iBGP communication requires full mesh design.
• The BGP split horizon rule is used to prevent information from circulating endlessly between iBGP in a loop.
• The total number of BGP sessions increase quickly as more routers are added to the network. 
• The formula n * (n - 1) / 2 can be used to find the exact number of required BGP sessions.
• Further, adding new nodes is a configuration-intensive and time-consuming task, prone to errors.
• Route Reflectors modify the BGP split horizon rule and Confederations modify iBGP AS path processing.
• Route Reflectors allow a particular router to forward incoming iBGP updates to other iBGP peers.
• Confederations introduce the concept of multiple smaller autonomous systems within the original AS. The smaller autonomous systemes exchange BGP updates between them by using intra-confederation eBGP sessions.

Route Reflectors

• Route Reflectors reflect learned routes from iBGP peers to other iBGP peers.
• Enabled on the route reflector itself by defining its iBGP peers as route reflector clients.
• Route reflector clients need no additional configuration.
• The number of required BGP sessions drops significantly.
• Easier to manage and troubleshoot.
• A non-client: any route reflector iBGP peer that is not a route reflector client.
• Route reflectors are non-clients with each other.
• Route reflectors must be fully meshed with non-clients.
• A client: any iBGP peer to which the route reflector reflects routes.
• To configure a route reflector client on a route reflector, the peer must specifically be designated as a client by a special BGP configuration option.
• A non-client is a regular peering definition on a route reflector, with a peer that is not specifically configured as a route reflector client.

Confederations

• The outer AS is called the confederation AS; it is all that is visible to the outside world.
• Each inner AS is a sub-autonomous system.
• Each sub-autonomous system has its own private AS number (range: 64512 through 65534).
• BGP confederations contain the full-mesh requirement within a subautonomous system.
• To further scale the network, route reflectors can be used within a sub-autonomous system.
• BGP confederations can be used to partition a network based on structural or geographic requirements. Each sub-AS can run a different IGP if that is what is needed.
• Confederations would also be used during a company merger, when two private AS BGP networks need to merge into a consolidated network.

BGP Split Horizon Rule

• The split horizon rule prevents routing loops.
• EBGP and IBGP manage the split horizon rule differently.
• EBGP relies on the AS path to prevent to loops. If a router sees its own AS in the AS path, the route is discarded.
• IBGP relies on the behavior that IBGP routes will not be readvertised to other IBGP peers.
• IBGP peers are in the same AS, so they do not add anything to the AS path.
• IGP split horizon rule: do not advertise a route back onto the interface from which it was learned.

Route Reflector Split Horizon Rule

• The split horizon rule is modified with route reflectors.
• If a route reflector receives a route from an eBGP peer, it reflects the route to all clients and non-clients.
• If a route reflector receives a route from a client, it reflects the route to all clients, non-clients, and all eBGP peers.
• If a route reflector receives a route from a non-client, it reflects the route to all clients and eBGP peers but not to non-clients. Non-clients need to be fully meshed.
• As any BGP route, the route needs to be the best to be reflected.

Redundant Route Reflectors

• If the only route reflector in the network fails, the clients can no longer send BGP updates to, or receive them from, the rest of the AS. The route reflector is, therefore, a single point of failure.
• To avoid this scenario, redundant route reflectors are typically used.
• Clients should establish iBGP sessions with at least two route reflectors by using different physical connections.
• Both route reflectors will receive the same iBGP update from their clients, and they will both reflect the update to the rest of the clients.
• As a result, each client will get two copies of all routes. 
• Under certain circumstances, improper route reflection can result in an iBGP routing loop that is impossible to detect. 
• Additional BGP attributes are thus necessary to prevent these routing loops.

Route Reflector Clusters

• Within the cluster, each client peers with all route reflectors.
• Route reflectors from different clusters need to be fully meshed with each other.The exception to this rule is when there is a hierarchical design where a route reflector is a client of another route reflector.
• Routers that do not support route reflector functionality have to be fully meshed with route reflectors from all clusters and with other non-clients.

Route Reflector Clusters: Cluster ID

• IBGP behavior depends on the cluster ID configuration.
• The cluster ID can be manually configured with the bgp cluster-id command.
• If not explicitly configured, the router ID is used as the cluster ID.
• Clients should not be configured with this information.
• The cluster ID is prepended to the cluster list attribute, which works as a loop prevention mechanism similar to AS path.
• The BGP route selection process is modified so that it includes the criterion of the length of the cluster list. Routes with a shorter cluster list are preferred to those routes that have a longer list.
• Shared (manually configured) cluster ID is less resilient but can use fewer hardware resources.
• No cluster ID is preferred for more resiliency.

Additional Loop Prevention Mechanisms

• Two non-transitive optional BGP attributes are used to prevent loops in BGP networks that use route reflectors: the originator ID and the cluster list.
• Originator ID: router ID of the route originator.
• Cluster list: sequence of cluster IDs.
• If the router finds its own originator ID or cluster ID, the route is discarded.
• RFC 4456 modies BGP path selection:
• shorter cluster list is preferred
• makes cluster ID configuration obsolete
• The route reflector sets the originator ID attribute of the route to the router ID of the originating router.
• The cluster list is a sequential list of cluster IDs, which prepends the router IDs (or cluster IDs if configured) of the route reflectors that have reflected the route along the way.
• Modified BGP path selection:
• The traditional BGP path selection parameters, such as weight, local preference, origin, and MED, are compared first.
• If these parameters are equal, the routes that are received from EBGP neighbors are preferred over routes that are received from IBGP neighbors.
• When a router receives two IBGP routes, the nonreflected routes (routes with no originator ID attribute) are preferred over reflected routes.
• The reflected routes with shorter cluster lists are preferred over routes with longer cluster lists. If the additional route-reflector-oriented selection criteria do not yield a decision, the rest of the traditional BGP path selection rules are followed.

Loop Prevention: Cluster List

• When the route is reflected, the route reflector adds its cluster ID (or router ID) to the cluster list single-cluster topology example:
• The same cluster ID is configured on both route reflectors.
• An identical cluster ID prevents a route exchange between the route reflectors, thus preventing loops.
• Overlapping-cluster example:
• Each route reflector has a different cluster ID, so route exchange is possible.
• Routes with a shorter cluster list are preferred.
• The cluster list length prevents routing loops by influencing the best route selection.
• When a route is reflected, the route reflector adds its cluster ID to the cluster list attribute. The cluster ID value of a route reflector can be taken from one of the following:
• BGP cluster ID: If explicitly configured
• BGP router ID: If the cluster ID is not explicitly configured
• Because the cluster list attribute of the route from client B (empty) is shorter than the same attribute of the route that is coming from RR3 (10.10.10.1), the route from client B is considered the best path for that destination.

Network Design with Route Reflectors

• Route reflectors can be one of the following:
• Dedicated and not routing any traffic.
• In the path of the traffic.
• The preferred way of designing the route reflector network is by using a redundant cluster, which relies on the underlying physical network redundancy. 
• When using multiple clusters, the route reflectors should be fully meshed, unless there is a hierarchical route reflector design.
• A BGP network can be designed to use dedicated route reflectors that are not actually participating in user data packet processing, but they manage route distribution tasks only.

Hierarchical Route Reflector Design

• With hierarchies, a router that serves as a route reflector in one cluster can act as a client in another cluster.
• The hierarchy can be as deep as needed.
• When a client in the lowest level receives an EBGP update, it will forward the update on all configured IBGP sessions to a route reflector. The route reflector recognizes BGP updates that are received from configured clients and will forward these updates to all other clients that use normal IBGP sessions. The update, sent on a normal IBGP session, will be a second-level client update to the second-level route reflector. The second-level route reflector will recognize that the update was received from a client, and it will forward it to all other clients and into the full mesh.

Potential Network Issues

Here are some of the problems that could occur if you deviate from route reflector network design rules:

• If route reflectors are not connected with IBGP sessions in a full mesh, some clusters will not have all the routes.
• If a client has IBGP sessions with some route reflectors in a cluster, but not with all of them, the client might miss some BGP routes.
• If a client has IBGP sessions to route reflectors that belong to different clusters, the BGP update from the client will be forwarded by the client into the full mesh with different cluster IDs in the cluster list attribute. When the BGP update enters the mesh, it will reach the other route reflector, which will, unnecessarily, accept the route as valid and forward it into its cluster. This situation, in turn, causes unnecessary duplication of updates to the clients.
• If a client has IBGP sessions to other clients in the same cluster, those clients will receive unnecessary duplications of updates.
• Modification of route attributes on route reflectors before reflecting the routes can lead to suboptimal routing or even potentially broken networks.