Redundancy: providing multiple alternative paths for the traffic.
Symmetry: having traffic that leaves the AS from an exit point return through the same point.
Load balancing: dividing traffic optimally over multiple links. Extremely challenging because multiple providers control and manipulate traffic that transits any AS.
REDUNDANCY
Connectivity is not the responsibility of one entity.
At any time, human error, software errors, physical errors, or adverse unforeseen conditions can jeopardize connectivity. Hence, redundancy is desirable.
However, the more redundancy a network has, the more unpredictable the traffic entrance and exit points are.
Traffic might exit on one point and come back on another.
Alternate routes translate into additional routing overhead.
Default routes become viable because they minimize resource consumption and provide "maximum" redundancy.
Any router advertising the universally known default route 0.0.0.0/0 represents itself as a gateway of last resort for other systems.
Default routes can be static or dynamic and they can point to a network or an exit interface.
It is important for defaults to disappear dynamically if what they point to disappears.
Selected default networks should not be specific subnets.
A subnet that is flip-flopping might cause your default to come and go constantly. It is much better to point the default to a major aggregate or supernet that reflects the stability of a whole provider rather than a particular link.
Multiple default routes can be used simultaneously.
One way to set multiple static defaults is to point to multiple networks (using aggregates if possible for stability reasons) and establish a degree of preference by using the local preference BGP attribute.
Another way is to use the distance parameter (the lower distance wins).
The distance parameter is not exchanged between routers, so this would work only in the case of one router connected via multiple connections.
If a route is associated with an interface, the interface must be unavailable before the route becomes invalid. In some cases, the interface will still be active, even if the virtual circuit has failed.
SYMMETRY
Traffic leaving from a given exit point and coming back through the same point.
Easy to achieve with a single point.
With redundancy (multiple connections), traffic tends to be asymmetrical.
In some situations, asymmetrical traffic is acceptable and does not pose significant problems.
In general, symmetrical routing is better because it minimizes potential delays that could result otherwise.
To accommodate symmetry, you should designate a primary link and make the utmost effort to direct the majority of traffic to flow on this link.
LOAD BALANCING
The capability to divide data traffic over multiple connections.
A common misconception about balancing is that it means an equal distribution of the load.
Equal distribution is difficult, if not impossible to achieve in most scenarios.
Load balancing tries to achieve a traffic distribution pattern that will best utilize the multiple links that provide redundancy.
Traffic should be thought of as two separate entities: inbound and outbound.
Inbound traffic is affected by how the AS advertises its networks to the outside world, whereas outbound traffic is affected by the routing updates coming in from outside ASes.
SCENARIO 1: SINGLE-HOMING
A single connection to a service provider.
Only a default route required.
The least expensive and the most effective method.
No need for BGP between the customer and the service provider.
Symmetric routing but no redundancy.
Obviously, not suitable for satisfying mission-critical data communication requirements.
SCENARIO 2: MULTIHOMING TO A SINGLE PROVIDER
Multiple connections to the same service provider.
Although not necessary in all cases, BGP will provide the most flexibility.
Possible to use a private AS.
Default Only, One Primary Link, and One Backup Link
Preference over the default route can be set with administrative distance or local preference if BGP is used.
Load balancing is not an option.
Inbound traffic is out of the customer's control.
The customer could affect incoming traffic by advertising different metrics.
Default, Primary and Backup, Plus Partial Routing
Same scenario as before but the customer also accepts partial routing information from the service provider.
Local preference is used to manipulate the preference based on AS path or IP prefix.
For all other Internet routes not known to the customer, default will be taken in the primary and backup default manner.
A totally different approach is for the provider to send its metrics (MED) toward the customer.
Less configuration on customer site but totally dependent on the provider delivering accurate MEDs.
The customer can influence inbound traffic by advertising different metrics on different links.
Some providers encourage their customers to send their internal IGP metrics as BGP metrics.
Default, Primary and Backup, Plus Full and Partial Routing
Multihomed to a single provider.
Local preference is used to control the customer's outbound traffic, and the metric is used to control the inbound traffic.
CAUTION: When dealing with outbound traffic, manipulating exit points for specific routes is dangerous. Routing loops can occur if outbound traffic following an IGP default toward the customer's BGP router is directed toward another router following default to the BGP router.
AUTOMATIC LOAD BALANCING
Cisco IOS software supports dynamic load balancing on a single router for identical destinations learned via EBGP that are sourced from the same autonomous system.
A Cisco router will keep (locally in its IP routing table) up to six identical BGP routes to the same destination (dictated by the maximum-paths command).
However, BGP installs only a single best route by default. BGP Multipath can be used to install multiple paths.
When passing on the EBGP updates to the IBGP peers, however, the router will pass only its singular best route for a particular destination.
By default, a Cisco router will load balance on a per-destination (host) basis in a round-robin fashion.
Does not work in a multiprovider environment.
SCENARIO 3: MULTIHOMING TO DIFFERENT PROVIDERS
Redundancy and geographical restrictions are strong motivations for multihoming.
Default Only, Primary and Backup
A customer can set or learn the default routes to the two providers either through static routes or through both providers' dynamic advertisements of default routes.
The customer can prefer one default over another by using administrative distance or local preference.
Default, Primary and Backup, Plus Partial Routing
By accepting partial routing from the providers, the customer does not need to see all Internet routes and can still make best routing decisions.
For the more specific routes, BGP will choose the provider link closer to the destination.
For other Internet routes, the basic principle of primary and backup default route can be used.
Default, Primary and Backup, Plus Full and Partial Routing
Accepting full routes from either or both providers is not really necessary unless the customer plans to be a provider itself and advertise full routes to its customers.
For effective load balancing, the customer can accept full routing from one or both providers.
In some cases, the customer might want to accept full routing from one provider and implement partial/default routing with the other provider
Customer Inbound Traffic
In a multiprovider scenario, sending different metrics from the customer's end will not have any effect because the MED value is non-transitive.
Only the customer's direct upstream provider will learn the MED value but it is not passed along.
Instead, the AS path attribute could be manipulated. By adding bogus AS entries to affect the length would make the route less desirable.
A BGP neighbor connection can be established within the same AS (internal BGP or iBGP) or between routers in different ASs (external BGP or eBGP).
During the OPEN message exchange, peer routers compare ASs to determine if they are in the same or different AS.
The difference between iBGP and eBGP is how BGP would process the routing updates and which attributes are carried in internal vs. external BGP.
The neighbor negotiation process is mainly the same for internal and external BGP.
IP connectivity must exist via a protocol different from BGP.
PHYSICAL vs. LOGICAL CONNECTIONS
Indirectly connected external neighbors require extra configuration (multihop eBGP).
On the other hand, neighbors within the same autonomous system (internal neighbors) have no restrictions whatsoever on whether the peer is physically connected or separated by multiple IP hops.
As long as there is IP connectivity between the two neighbors, BGP requires no additional configuration.
Session stability depends on the stability of neighbor IP address.
Binding the BGP neighbor connection to a loopback interface will ensure that the BGP session is not reliant on any hardware interface that might be problematic.
Other than route reflection, BGP does not readvertise iBGP routes learned from other iBGP peers.
Thus, a full mesh within the AS is required.
SYNCHRONIZATION WITHIN AN AS
By default, BGP requires that it must be synchronized with the IGP before BGP may advertise transit routes to external ASs.
Whenever a router receives an update about a destination from an iBGP peer, the router tries to verify internal reachability for that destination before advertising it to other eBGP peers.
The router does this by checking the destination prefix first to see if a route to the next-hop router exists and second to see if a destination prefix in the IGP exists.
Otherwise, the router treats the destination prefix as not being synchronized with the IGP and does not advertise it.
The BGP rule states that a BGP router should not advertise to external neighbors destinations learned from IBGP neighbors unless those destinations are also known via an IGP. This is known as synchronization.
One option is to use default routes on non-BGP routers but this could lead to suboptimal routing or even routing loops.
no synchronization will tell BGP to override the synchronization requirement and allow it to advertise routes learned via IBGP, irrespective of the existence of an IGP route.
That said, by far the most common configuration in Internet-connected networks is to disableBGP synchronization and rely on a full mesh of IBGP routers. The thought of injecting tens of thousands of routes into an IGP is quite frightening.
SOURCES OF ROUTING UPDATES
DYNAMICALLY INJECTING INFORMATION INTO BGP
Two options:
Purely dynamic: redistribution from an IGP to BGP (the redistribute command).
Semidynamic: only certain IGP routes are injected into BGP (the network command).
If BGP does not find an exact match for routes redistributed via the network command, they will not be advertised.
If the mask is not specified with the network command and auto-summarization is enabled (default), existence of any subset of the classful prefix will result in the advertisement of the classful prefix.
INJECTION OF UNWANTED OR FAULTY INFORMATION
Redistributing the whole IGP into BGP could result in the advertisement of private addresses or internal addresses that are supposed to be used within the AS only.
In mutual redistribution, information that was injected from the outside into the AS could be sent back to the Internet as having originated from the AS.
Careful filtering is required to guard against these undesired consequences.
Some IGPs can inherently differentiate between internal and external routes.
For other IGPs, special route tagging can distinguish between internal and external routes.
UNSTABLE ROUTES
A process called route dampening penalizes and ultimately discontinues advertisement of fluctuating routes, depending on their degree of instability. Route advertisements may be suppressed for minutes or even hours (until they stabilize).
One way to minimize route instability is through aggregation. When an aggregate route announcement represents more than one route, the fluctuation of any single route does not cause fluctuation in the aggregate itself.
STATICALLY INJECTING INFORMATION INTO BGP
The most effective method of ensuring route stability.
IGP routes (or aggregates) to be advertised to BGP peers are manually defined as static routes.
If the route is advertised to the Internet from a single point, advertising a route that is actually down is not a big issue.
Hosts trying to access that destination will fail irrespective of whether the route is advertised.
On the other hand, if a route is advertised to the Internet from multiple points, advertising the route statically at all times might end up black-holing the traffic.
ORIGIN OF ROUTES
BGP considers the networks advertised via the network command or via aggregation as being internal to the AS and will include the ORIGIN attribute in each route as being IGP (i).
On the other hand, whenever a route is injected into BGP via redistribution (whether statically or dynamically), the ORIGIN of the route will be INCOMPLETE because the redistributed routes could have come from anywhere.
Aggregated routes will assume the worst ORIGIN value of all the component routes.
The ORIGIN attribute is used by the BGP decision process to favor one route over another.
BGP ROUTES: ADVERTISEMENT AND STORAGE
A route is defined as a unit of information that pairs a destination with the attributes of a path to that destination.
Routes are stored in the Routing Information Bases (RIBs): namely, the Adj-RIBs-In, Loc-RIB, and the Adj-RIBs-Out.
Adj-RIB-Out: Routes that will be advertised to other BGP speakers.
Loc-RIB: Routes that will be used by the local BGP speaker, and the next hop for each of these routes must be present in the local BGP speakers forwarding information base (FIB).
Adj-RIBs-In: Routes that are received from other BGP speakers.
If a BGP speaker chooses to advertise a route, it may add to or modify the route's path attributes before advertising it to the peer.
THE BGP RIB
Three distinct parts: Adj-RIB-In, Loc-RIB, and Adj-RIB-Out.
If a route is not placed in the Loc-RIB, it cannot be placed in the Adj-RIB-Out for advertisement to peers.
BGP DECISION PROCESS
Based on the attribute values.
Only one best path is chosen, in the following preference order:
If the next-hop is inaccessible, the route is ignored.
WEIGHT (Cisco proprietary attribute, local to the router).
LOCAL PREFERENCE.
AS_PATH.
ORIGIN type (IGP < EGP < incomplete).
The lowest MED value.
eBGP paths over iBGP paths.
The route via the closes IGP neighbor. Follow the shortest path to the BGP NEXT_HOP.
The lowest BGP ROUTER_ID.
If BGP Multipath is enabled, some steps could be bypassed, and all paths with the same AS_PATH length and MED value could be installed into the routing table.
BGP PATH ATTRIBUTES
A set of parameters that describe characteristics of a prefix (route).
Well-known mandatory: must be recognized by all BGP implementations and included in every UPDATE message.
Well-known discretionary: must be recognized by all BGP implementations but may or may not be sent in a particular UPDATE message.
Optional transitive:
Optional non-transitive:
Unrecognized non-transitive optional attributes must be quietly ignored and not passed along to other BGP peers.
ORIGIN (Type Code 1)
A well-known mandatory attribute that defines the origin of the path information.
0: IGP - NLRI that is interior to the originating AS.
1: EGP - NLRI learned via EGP.
2: INCOMPLETE - NLRI learned by some other means.
AS_PATH (Type Code 2)
A well-known mandatory attribute that is composed of a sequence of AS path segments.
Inside the AS, the AS_PATH information is intact. However, when sending routes to eBGP peers, the AS that originates the route adds its own AS number.
Thereafter, each AS that receives the route and passes it on to other eBGP peers will prepend its AS number to the list.
Prepending is the act of adding the AS number to the beginning of the list.
Loop prevention: If the route is advertised to an AS whose AS number is already present in the AS_PATH, the UPDATE is ignored.
NEXT_HOP (Type Code 3)
A well-known mandatory attribute that defines the next-hop IP address.
For eBGP, the next-hop is the IP address of the neighbor that announced the route.
For iBGP, for routes that originated inside the AS, the next-hop is the IP address of the neighbor that announced the route.
A recursive IP lookup might be required for a router to know where to send the packet.
If a next-hop cannot be reached, BGP considers the route inaccessible.
MULTI_EXIT_DISC (MED) (Type Code 4)
An optional non-transitive attribute that is used to discriminate among multiple exits points to a neighboring AS.
A hint to external neighbors about the preferred path into an AS that has multiple entry points.
When BGP passes the routing update to another AS, the MED is reset to 0 (unless the outgoing MED is explicitly set to a specific value).
When the route is originated by the AS itself, the most common practice is for the MED value to follow the internal IGP metric of the route.
MEDs from different ASs are not comparable.
Still, for administrators who have a reason to do so, Cisco offers the bgp always-compare-med router command, which tells BGP to compare MEDs from different ASs for the same route.
When providers don't use MEDs or other attribute manipulation techniques to prefer the best-exit path, it's often referred to as closest-exit or hot potato routing.
LOCAL_PREF (Type Code 5)
A well-known discretionary attribute that is used to inform other BGP speakers in its own AS of the preference for an advertised route.
A higher local preference is preferred.
Exchanged between iBGP peers only.
Cisco proprietary WEIGHT attribute is similar.
The difference is that the weight parameter is local to the router and is not exchanged between routers, even internal to an AS.
ATOMIC_AGGREGATE (Type Code 6)
A well-known discretionary attribute that is used to inform other BGP speakers that the local system chose a less-specific route.
AGGREGATOR (Type Code 7)
An optinal transitive attribute that contains the last AS number that formed the aggreagate and the IP address of the BGP speaker that formed the aggregate route.
COMMUNITY (Type Code 8)
An optional transitive attribute that specifies all routes with this attribute belonging to the communities listed in the attribute.
ROUTE FILTERING AND ROUTE MANIPULATION
A BGP speaker can choose what routes to send and receive.
Route filtering is essential in defining routing policies.
Routes permitted through a filter can have their attributes manipulated.
Manipulating the attributes affects the BGP decision process used to identify the best routes to a given destination.
Both the inbound and outbound filtering can be applied to the peer and protocol levels.
Inbound filtering indicates that the BGP speaker is filtering routing updates received from other peers, whereas outbound filtering limits the routing updates advertised from the BGP speaker to other peers.
Both eBGP and iBGP filtering behavior is the same.
Three actions:
Identifying the routes.
Permitting or denying the routes.
Manipulating the attributes.
After a route matches an instance, it is considered identified and will not be compared to any further instances. Hence, the order in which the instances are checked is important.
If the route proceeds to be compared against the entire list of instances and there is still no match, the route is discarded.
Prefix lists are currently the preferred mechanism to be used for filter routing information.
Trying to identify the AS_PATH list consists of comparing the list to what is called a regular expression.
PEER GROUPS
A group of BGP neighbors that share the same update policies.
Saves the administrator from repetitive configuration and the BGP router itself from parsing the policies sequentially for each individual neighbor.
With peer groups, the router formulates the update once, based on the policies of the peer group, and then floods the same update to all the neighbors that fall within the group.
BGP-4 AGGREGATION
Aggregation applies to routes that do not exist in the BGP routing table (contrary to the network command, which applies to rotes that do exist in the IP routing table).
Aggregation can be performed if at least one more-specific route of the aggregate exists in the BGP routing table.
Examples:
Sending the aggregate route only and suppressing more specific routes.
Sending the aggregate route and the more specific routes. Typical in a multihomed ISP scenario. Normally, the ISP does not want to advertise the more-specific routes further to the NAP, so it has the originating AS tag all the more-specific routes with the community attribute NO_EXPORT.
Sending the aggregate route and a subset of more-specific routes.
BGP is a path vector protocol used to carry routing information between autonomous sytems.
Path vector signifies that BGP carries a sequence of AS numbers that identifies the AS path. Also used for loop prevention.
BGP uses TCP port 179.
All the transport reliability is taken care of by TCP. No need to design this feature into the protocol itself.
BGP speakers = routers that run BGP.
BGP neighbors or peers = routers that form a TCP connection to exchange routing information.
Initially, all BGP routes are exchanged. Only incremental updates are sent as the network information changes.
Four message types: OPEN, NOTIFICATION, KEEPALIVE, and UPDATE.
BGP neighbors exchange OPEN messages to determine the connection parameters.
NOTIFICATION is sent in case of disagreement between the peers, and as a result, the session is not established or is torn down if already established.
UPDATE messages advertise BGP routes. Indicate a list of destinations that can be reached through that BGP speaker, and the related path attributes.
KEEPALIVE messages are sent periodically (every 60 seconds by defaut) between the BGP peers to ensure that the connection stays up.
A table version number keeps track of the current instance of the BGP routing table. Incremented at each change. A rapidly incrementing table version is usually an indication of network instability.
BGP MESSAGE HEADER FORMAT
+++++++++++++++++++++++++++++++++++++++++++ | Marker | Length | Type | +++++++++++++++++++++++++++++++++++++++++++
Depending on the message type, there might or might not be a data portion following the header.
For example, KEEPALIVE messages consist of the message header only.
The Marker field (16 bytes) is used to either authenticate incoming BGP messages or detect loss of synchronization between two BGP peers.
If the type is OPEN or OPEN+no authentication, the Marker field must be all 1s.
Otherwise, the Marker field will be computed based on part of the authentication mechanism.
The Length field (2 bytes) indicates the total BGP message length.
The Type field (1 byte) indicates the message type (OPEN, UPDATE, NOTIFICATION, KEEPALIVE).
BGP NEIGHBOR NEGOTIATION
The OPEN Message.
BGP peers agree on a BGP version number. BGP tries to negotiate the highest common version.
The AS number is advertised in the OPEN message.
The Hold time indicates the maximum amount of time in seconds that may elapse between the receipt of successive KEEPALIVE or UPDATE messages.
The BGP neighbors negotiate to select the Hold time value which is lower.
The Hold timer could be 0, in which case the timers are never reset. The minimum Hold timer is 3 seconds.
BGP ID is also transported in the OPEN message. Typically the highest loopback IP address.
Optional parameters include the authentication information parameter.
BGP FINITE STATE MACHINE
1. Idle.
BGP initializes its resources, initiates a TCP transport connection, and starts listening for a connection that may be initiated by a remote peer.
2. Connect.
BGP is waiting for the transport protocol connection to be completed. If successful, the state transitions to OpenSent (the OPEN message is sent). If unsuccessful, the state transitions to Active.
3. Active.
BGP tries to acquire a peer by initiating a transport protocol connection. If expired, BGP restarts the ConnectRetry timer and falls back to the Connect state. The state might go back to Idle in case of other events, such as Stop event. A neighbor state that is oscillating between Connect and Active typically indicates a problem with the TCP transport connection.
4. OpenSent.
BGP is waiting for an OPEN message from its peer. In case of errors, an error NOTIFICATION is sent and the state goes back to Idle. If no errors, BGP starts sending KEEPALIVE messages.
5. OpenConfirm.
BGP waits for a KEEPALIVE message. If a KEEPALIVE is received, the state goes to Established, and the neighbor negotiation is complete.
6. Established.
The final stage in the neighbor negotiation. BGP starts exchanging UPDATE packets with its peers.
NOTIFICATION MESSAGE
Always sent whenever an error is detected.
After that, the peer connection is closed.
++++++++++++++++++++++++++++++++++++++++++ | Error | Error subcode | Data | ++++++++++++++++++++++++++++++++++++++++++
The Error code indicates the type of the notification.
The Error subcode provides more information about the nature of the error.
The Data field contains data relevant to the error, such as a bad header or an illegal AS number.
KEEPALIVE MESSAGE
Periodic messages exchanged between peers to determine whether peers are reachable.
The Hold time is the maximum amount of time that may elapse between the receipt of successive KEEPALIVE or UPDATE messages.
A recommended KEEPALIVE rate is 1/3 of the Hold time.
If the Hold timer is 0, periodic KEEPALIVE messages are not sent.
The KEEPALIVE message is a BGP header with no data following it.
The NLRI (IP prefix route) indicates the networks being advertised.
The Path Attribute list enables BGP to detect routing loops and gives it flexibility to enforce local and global routing policies.
NETWORK LAYER REACHABILITY INFORMATION (NLRI)
The NLRI is the part of the BGP UPDATE message that lists the set of destinations about which BGP is trying to inform its other BGP neighbors.
The NLRI consists of one or more instances of the 2-tuple format <length, prefix>, where length is the number of the subnet mask that a particular prefix has.
Example: <19, 198.24.160.0> = 198.24.160.0/19.
Withdrawn routes have the same format.
An UPDATE message with no new NLRI or Path Attribute information is used to advertise only routes to be withdrawn from service.
PATH ATTRIBUTES
A set of parameters used to keep track of route-specific information.
Well-known mandatory: Has to exist in the BGP UPDATE packet. Must be recognized by all BGP implementations. If missing, a NOTIFICATION error is generated, and the session is closed.
Well-known dicretionary: Recognized by all BGP implementations but might or might not be sent in the BGP UPDATE message.
Optinal transitive: The BGP implementation should accept the attribute and pass it along to other BGP speakers.
Optional non-transitive: The attribute is quietly ignored and not passed along to other BGP peers.
BGP CAPABILITIES NEGOTIATION
The purpose is to introduce a new optional parameter to BGP-4 called Capabilities.
Supported capabilites are included in the OPEN message.
If a peer supports a given capability, the speaker can use the capability with the peer.
If not supported, the BGP speaker receives a NOTIFICATION error message and attempts to reestablish the connection without sending the Capabilities parameter to the peer.
MULTIPROTOCOL EXTENSIONS FOR BGP
MP-BGP is negotiated via BGP capabilities.
Provides a backward-compatible extension to the BGP-4 protocol that let it carry information fro network layer protocols other than IPv4, such as IPv6 and IPX.
Two new attributes: Multiprotocol Reachable NLRI (MP_REACH_NLRI) and Multiprotocol Unreachable NLRI (MP_UNREACH_NLRI).
MP_REACH_NLRI and MP_UNREACH_NLRI are optional non-transitive attributes.
Interdomain multicast routing is the most common use of BGP's multiprotocol extensions.
TCP MD5 SIGNATURE OPTION
Help BGP protect itself from spoofed TCP segments and TCP resets in particular.
The TCP segment carries a digest message based on information known only to the connection end points.
When TCP receives a signed segment, the receiver must validate it by using its local key to calculate its own digest and compare the value with that of the received digest.
If the comparison results in unequal values, the segment should be discarded and must not produce any response to the sender.
The MD5 key is never exchanged via the connection and that only the end points should be aware of the value.
There are performance implications that result in significantly increased delay with BGP message processing and generation.