Wednesday, December 30, 2015

CCIE 400-101: Network Principles - Latency, Windowing, BDP, Global Synchronization

Latency

  • Latency is most often used to describe the delay between the time that data is requested and the time when it arrives (also known as round-trip time or RTT for short).
  • However, there are essentially four components that contribute to total latency:
    1. Transmission delay: the time to transmit data on the link.
    2. Propagation delay: the time for a packet to travel accross the link.
    3. Processing delay:  the time for a device to process the data.
    4. Queuing delay: the time the data spent waiting in queue (memory) inside a device.
  • The combination of transmission, propagation, processing, and queuing delays often produces a complex and variable network latency profile.


  • Different delays can be calculated as follows:
    • Transmission delay (t) = number of bits to send (b) / link data rate (r). (t = b/r)
    • Propagation delay (p) = link distance (d) / speed of signal (s). (p = d/s)
      • Unless otherwise stated, s = c = 3 x 10^8 (the speed of light)
    • Processing delay and queuing delay are variable and depend on many factors.

Example: The latency is indicated in the 'time' column of the ping output. The ping statistics also list the minimum, maximum, and average latencies.

Pinging www.google.com [74.125.224.82] with 1500 bytes of data:
Reply from 74.125.224.82: bytes=1500 time=68ms TTL=52
Reply from 74.125.224.82: bytes=1500 time=68ms TTL=52
Reply from 74.125.224.82: bytes=1500 time=65ms TTL=52
Reply from 74.125.224.82: bytes=1500 time=66ms TTL=52
Reply from 74.125.224.82: bytes=1500 time=70ms TTL=52

Ping statistics for 74.125.224.82:
    Packets: Sent = 5, Received = 5, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 65ms, Maximum = 70ms, Average = 67ms


  • A networking technology with very high throughput and bad (high) latency can be worse for some applications than one with relatively low throughput but good (low) latency. 
  • See: Ping vs. Latency: What's the Difference?

Windowing

  • Essentially, the TCP window size is the amount of ("in flight") data that a sender can send before it needs to wait for an acknowledgement from the recipient. 
  • If the sender has not received acknowledgement, it will stop and wait. If this wait time exceeds a certain limit, a retransmission will occur. This is how TCP achieves reliable data transmission. 
  • Even if there is no packet loss in the network, windowing can limit throughput. 
  • Because TCP transmits data up to the window size before waiting for the acknowledgement, the full bandwidth of the network may not always get used. 
  • The window size is communicated via a 16-bit field in the TCP header, giving it a maximum size of 64 KB. 
  • Window scale provides a scale factor which is required for TCP to support window sizes that are larger than 64 KB. 
  • Essentially, window scaling simply extends the 16-bit window field to 32 bits. 
  • This is done by specifying a 1-byte shift count in the header options field. 
  • The true receive window size is left shifted by the value in the shift count. 
  • The maximum value of the shift count is 14, resulting in the maximum window size of 1 GB!


  • The window scaling option may be sent only once during a connection by each host, in its SYN packet.
  • The window size can be dynamically adjusted by modifying the value of the window field in the TCP header, but the scale factor remains static for the duration of the TCP connection.
  • Scaling is only in effect if both ends include the option; if only one end of the connection supports window scaling, it will not be enabled in either direction.

 

Bandwidth Delay Product (BDP)

  • In short, bandwidth delay product (BDP) is the optimum window size to fully utilize link bandwidth. 
  • The best possible network performance is achieved when the link between the sender and the receiver forwards a constant stream of data. 
  • Analogy: a water hose. To achieve maximum water flow, the hose should be full. As the hose increases in diameter and length, the volume of water to keep it full also increases. 
  • In networks, diameter equates to bandwidth, length is measured as round-trip time, and the TCP window size is analogous to the volume of water necessary to keep the hose full. 
  • On fast networks with large round-trip times, the TCP window size must be increased to achieve maximum TCP bandwidth. 
  • Theoretically, the TCP window size should be set to the BDP, which is simply computed as bandwidth * delay. 
  • Example: For 1 Gbps link with 70 ms delay, the BDP (optimum TCP window size) is 1,000,000,000 * 0.07 = 70,000,000 (bits). Convert that to bytes, you get 70,000,000 / 8 = 8,750,000 bytes = 8,75 MB.  
  • In order to fully utilize 1 Gbps link with 70 ms, the TCP window size must be 8,75 MB.  
  • Based on these calculations, it is easy to see why the typical default buffer size of 64 KBytes would be completely inadequate for this connection. With 64 KBytes you would get only 0.1% of the available bandwidth.

Global Synchronization

  • During times of congestion, routers can use a technique called tail drop to simply discard packets when the interface queue is filled to its maximum.
  • When tail drop happens, TCP has an automatic recovery from the dropped packets, which causes the sender to reduce its sending rate.
  • Bandwidth utilization drops significantly and interface queue becomes less congested.
  • After a certain amount of time, the sender starts increasing the rate again.
  • The problem is that almost all the senders will use the same time delay. When these delays expire, at the same time, all senders will send additional packets, the interface queue will overflow again, more packets will be dropped, the senders will all back off for a fixed delay.. and the cycle repeats.
  • This pattern of each sender decreasing and increasing transmission rates at the same time as other senders is referred to as "global synchronization".
  • The result is a sawtooth-shaped bandwidth utilization, causing inefficient use of available bandwidth and inconsistent application throughput.


References

Transmission and Propagation Delay
Congestion Avoidance
Tuning TCP for High Bandwidth-Delay Networks
A User's Guide to TCP Windows
TCP Windows and Window Scaling

Wednesday, December 23, 2015

CCIE 400-101: Network Principles - IPv4 and IPv6 PMTU, MSS

MSS

  • The TCP Maximum Segment Size (MSS) defines the maximum amount of data that a host is willing to accept in a single TCP/IP datagram.
  • The MSS value is sent as a TCP header option only in TCP SYN segments.
  • Each side of a TCP connection reports its MSS value to the other side.
  • Contrary to popular belief, the MSS value is not negotiated between hosts. The sending host is required to limit the size of data in a single TCP segment to a value less than or equal to the MSS reported by the receiving host.
  • The way MSS now works is that each host will first compare its outgoing interface MTU with its own buffer and choose the lowest value as the MSS to send.
  • The hosts will then compare the MSS size received against their own interface MTU and again choose the lower of the two values.
  • Often the send MSS value will be the same on each end of a TCP connection.
  • By default, the MSS = 1460 bytes in an Ethernet network (1500 bytes - 20-byte IP header - 20-byte TCP header = 1460 bytes).
  • TCP MSS takes care of fragmentation at the two endpoints of a TCP connection, but it doesn't handle the case where there is a smaller MTU link in the middle between these two endpoints.

PMTU

  • Path MTU (PMTU) is the smallest MTU in the path from source to destination.
  • Path MTU Discovery (PMTUD) was developed to avoid fragmentation in the path between the endpoints. It is used to dynamically determine the lowest MTU along the path from a packet's source to its destination.  
  • PMTUD is only supported by TCP. UDP and other protocols do not support it. 
  • If PMTUD is enabled on a host, and it almost always is, all TCP/IP packets from the host will have the DF bit set. 
  • If a router tries to forward an IP datagram, with the DF bit set, onto a link that has a lower MTU than the size of the packet, the router will drop the packet and return an Internet Control Message Protocol (ICMP) "Destination Unreachable" message to the source of this IP datagram, with the code indicating "fragmentation needed and DF set" (type 3, code 4).
  • A router returning an ICMP message indicating "fragmentation needed and DF set" should include the MTU of that next-hop network in the low-order 16 bits of the ICMP additional header field that is labeled "unused" in the ICMP specification. The high-order 16 bits remain unused, and MUST be set to zero.  Thus, the message has the following format:

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |   Type = 3    |   Code = 4    |           Checksum            |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |           unused = 0          |         Next-Hop MTU          |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |      Internet Header + 64 bits of Original Datagram Data      |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+



  • When the source station receives the ICMP message, it will lower the packet size to reflect the smaller MTU. 
  • There are three things that can break PMTUD, two of which are uncommon and one of which is common.
    • A router can drop a packet and not send an ICMP message. (Uncommon)
    • A router can generate and send an ICMP message, but the sender ignores the message. (Uncommon)
    • A router can generate and send an ICMP message but the ICMP message gets blocked by a router or firewall between this router and the sender. (Common) 
  • The success or failure of PMTUD hinges upon ICMP unreachable messages getting through to the sender of a TCP/IP packet. 
  • The following commands will enable the necessary ICMP messages to pass through a firewall:

access-list 101 permit icmp any any unreachable
access-list 101 permit icmp any any time-exceeded
access-list 101 deny icmp any any
access-list 101 permit ip any any


 

References

Resolve IP Fragmentation, MTU, MSS, and PMTUD Issues with GRE and IPSEC

Tuesday, December 22, 2015

CCIE 400-101: Network Principles - IPv4 and IPv6 Fragmentation, TTL, IP MTU

IPv4 and IPv6 fragmentation

  • Although the maximum length of an IP datagram is 64K, most transmission links enforce a smaller maximum packet length limit, called an MTU (maximum transmission unit). 
  • The design of IP accommodates MTU differences by allowing routers to fragment IP datagrams as necessary. 
  • IP fragmentation involves breaking a datagram into a number of pieces that can be reassembled later. 
  • Fragmentation causes more overhead for the receiver when reassembling the fragments because the receiver must allocate memory for the arriving fragments and coalesce them back into one datagram after all of the fragments are received. 
  • Reassembly is very inefficient on a router whose primary job is to forward packets as quickly as possible. 
  • Another fragmentation issue involves handling dropped fragments. If one fragment of an IP datagram is dropped, then the entire original IP datagram must be resent, and it will also be fragmented.
  • Example: The original IP packet is 12,000 bytes. To transmit this data over the first local link, Device A splits it into four fragments. The first router must fragment them further into smaller packets to send them over the 1,300-byte MTU link. Note that the packets are reassembled only at the final destination.



  • Unlike in IPv4, IPv6 routers never fragment IPv6 packets.
  • Packets exceeding the MTU are dropped and this condition is signaled by a "Packet Too Big" ICMPv6 message (similar to the IPv4 method when the DF bit is set).
  • IPv6 hosts are expected to perform path MTU discovery to determine the maximum size of packets to send, and the upper-layer protocol is expected to limit the payload size.

 

TTL 

  • TTL, which stands for Time To Live, is an 8-bit field in the IP header.
  • TTL determines the lifetime of an IP packet.
  • The purpose of TTL is to prevent a data packet from circulating in the network indefinitely.
  • The initial TTL value is set by the sender, and is reduced by every router on the route to the packet's destination. Hence, the TTL can also be considered as a hop count.
  • Because of the 8-bit field size, TTL can support a maximum of 255 hops.
  • If the TTL reaches 0 before the packet reaches its final destination, the packet is discarded and an ICMP error message is sent back to the sender.
  • Different operating systems can use different default TTL values. For example, most Windows systems use a default TTL of 128.


IP MTU

  • Maximum Transmission Unit (MTU) is the size of the largest packet that the device can forward. 
  • If the packet is bigger than the MTU, the packet will be fragmented (unless the DF bit is set, in which case the packet is dropped). If a packet is fragmented, it is reassembled at its final destination. 
  • There are basically two MTU values: system or interface MTU and IP MTU. 
  • The interface MTU specifies the maximum size of the datagram that the interface can send (including the Layer 2 header). 
  • IP MTU defines the maximum size of the IP packet. 


  • The value of the MTU depends on the type of the transmission link. For example, the default MTU for Ethernet is 1500 bytes. In a normal transmission case (no additional encapsulation, such as GRE or IPsec), the sender may use the maximum payload length of 1460 bytes (1500 bytes - 20-byte IP header - 20-byte TCP header = 1460 bytes) without any potential risk of packet fragmentation/drop.
  • Sometimes the IP MTU must be lowered from its default value to support extra headers like GRE or IPsec. 
  • A larger MTU brings certain benefits like processing of fewer packets for the same amount of user data.  
  • Example: NFS, by default, has a read and write block size of 8192, so a NFS IP/UDP datagram will be approximately 8500 bytes (including NFS, UDP, and IP headers). A sending station connected to an Ethernet (MTU 1500) will have to fragment the 8500 byte datagram into six pieces; five 1500 byte fragments and one 1100 byte fragment. If any of the six fragments is dropped, the complete original datagram will have to be retransmitted. This creates more overhead and requires more processing resources from the router, as compared to sending one big packet of 8500 bytes (MTU 9000). 
  • On the contrary, larger packets can occupy the link for too long, thereby increasing delay.

 

References

The TCP/IP Guide: IP Datagram Size, MTU, and Fragmentation Overview

Tuesday, September 1, 2015

CCIE 400-101: Network Principles - IPv4 Options and IPv6 Extension Headers

IPv4 Options

  • In addition to the standard 20-byte IP header, IP options provide more flexibility in how the datagrams are handled. 
  • IPv4 options are, unsurprisingly, optinal. However, all IP devices must be capable of properly reading and handling them.
  • The IP datagram may contain zero, one or more options.
  • This makes the IP header variable in size. Since the IP header must be a multiple of 32 bits, a Padding field is included if all options together is not a multiple of 32 bits.

IP Option Format

 

  • Option Type: This 8-bit field is divided into three sub-fields.
    • Copied (1-bit flag): 1 if the option is to be copied into all fragments; 0 if the option should not be copied. 
    • Option Class (2 bits): Only two values are used - 0 is for Control options, and 2 for Debugging and Measurement. 
    • Option Number (5 bits): Specifies the kind of option. 32 different values can be specified for each of the two option classes. 
  • Option Length: Indicates the size of the entire option in bytes. 
  • Option Data: Contains data to be sent as part of the option.

List of IPv4 Options 

References

TCP/IP Guide: IP Option Format
TCP/IP Guide: IP Options


IPv6 Extension Headers

  • Much like IPv4 options, IPv6 extension headers carry optional information.
  • Rather than a single variable header that contains all fields (including options), the IPv6 datagram consists of the main header and the extension headers.
  • Only the fields that are truly required for all datagrams remain in the main header; others are put into extension headers and used as needed.

++++++++++++++++++++++++++++++++++++++++++++++
|           Main Header (40 bytes)           |
++++++++++++++++++++++++++++++++++++++++++++++
|           Extension Header #1              |
++++++++++++++++++++++++++++++++++++++++++++++
|                  ...                       |
++++++++++++++++++++++++++++++++++++++++++++++
|           Extension Header #N              |
++++++++++++++++++++++++++++++++++++++++++++++
|                                            |
|                 Data                       |
|                                            |
++++++++++++++++++++++++++++++++++++++++++++++


  • The only field in common to all extension headers is the Next Header field, which logically links all the headers in the IPv6 together.
    • The Next Header field in the main header contains a reference number for the first extension header type.
    • The Next Header field in the first extension header contains the number of the second extension header type. The second header's Next Header points to the third one, and so on.
    • The Next Header field of the last extension header contains the protocol number of the encapsulated higher-layer protocol.


  • If there are no extension headers, the Next Header field points straight to the upper-layer protocol in the Data field.


List of Extension Headers

 


References

TCP/IP Guide: IPv6 Header Chaining Using the Next Header Field
TCP/IP Guide: Summary of IPv6 Extension Headers

 

Bonus: IPv4 Header vs. IPv6 Header



Cisco: IPv6 Extension Headers Review and Considerations

Monday, August 24, 2015

CCIE 400-101: Network Principles - ICMP Unreachable and ICMP Redirect

ICM General Overview

  • IP in itself is not absolutely reliable, sometimes packets get dropped.
  • The purpose of ICMP messages is to provide feedback about problems in the communication path.
  • There is still no guarantee that packets will be delivered to their final destination. Some packets may even be undelivered without any report of their loss.
  • The higher level protocols (like TCP) must implement their own reliability procedures if reliable communication is required.

ICMP Unreachable

  • RFC 792 defines six different ICMP Destination Unreachable messages.
    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     Type      |     Code      |          Checksum             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             unused                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Internet Header + 64 bits of Original Data Datagram      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


  • The Type is 3 and the Code varies from 0-5 depending on why the packet was discarded:
    • 0 = net unreachable;
    • 1 = host unreachable;
    • 2 = protocol unreachable;
    • 3 = port unreachable;
    • 4 = fragmentation needed and DF set;
    • 5 = source route failed;
  • Codes 0, 1, 4, and 5 may be received from a gateway. Codes 2 an 3 may be received from a host.
  • Internet Header + 64 bits of Original Data Datagram are used by the host to match the message to the appropriate process. Port numbers are assumed to be in the first 64 bits of the original datagram's data.

Network Unreachable (0)

  • When a router sends a Network Unreachable message, it is saying that it has no way to reach the destination IP address.
  • If a device tries to ping 192.168.1.142 but there is a router in the path with no route to the destination network, it will send the Network Unreachable message.
  • This can also happen if a link to reach the network is down and there are no alternate paths.
  • A Network Unreachable message could also be the result of a misconfigured route. Using the previous IP address 192.168.1.142 as an example, assume that the administrator wanted to add a static route 192.168.1.0/24 but he accidentally configured 192.168.1.0/25 (default route is not set). The packet would not match this route, it would be dropped, and a Network Unreachable message would be sent to the source.

Host Unreachable (1)

  • The Host Unreachable message is sent by the last router in the path - directly connected to the destination network.
  • This message is generated when the router sends an ARP request on the directly attached network looking for the MAC address of the destination IP but never receives a reply, and hence, is not able to forward the packet to its final destination.
  • The router then issues a Host Unreachable message back to the originator.
  • The Host Unreachable message implies that the path across the communicating devices is working. The packet has been forwarded all the way through the network and has arrived successfully at the last router in the path.

Port Unreachable (3)

  • Unlike the Network Unreachable and Host Unreachable messages which come from routers, the Port Unreachable message comes from a host.
  • The Port Unreachable message indicates that the packet got all the way to its final destination but when the packet was processed by a higher layer protocol, the port process did not exist.
  • The protocol handler then reported the situation with a Port Unreachable message.
  • The nice thing about a Port Unreachable message is that it confirms proper end-to-end IP operation and the presence of the destination host.

Protocol Unreachable (2), Fragmentation Needed and DF Set (4), and Source Route Failed (5)

  • These three ICMP Unreachable messages are rare, as compared to the previous three, which are the most common. 
  • The Protocol Unreachable message simply means that the destination host did not support that protocol. This could happen, for example, when OSPF packets are sent to a RIP router. Since the router doesn't have any OSPF processes enabled, the entire protocol is unreachable. 
  • The Fragmentation Needed and DF Set is self-explanatory. The originating device set the DF bit in the IP flag field, so the receiving router cannot fragment the packet. If the router has a smaller MTU and fragmentation is not allowed, the packet is discarded and the ICMP message is returned. 
  • Did not find much information about the Source Route Failed message but this message might occur when source routing is implemented. Normally, packet forwarding is destination-based, but IP options can be used to specify the route a packet takes through the network. This is called source routing. The Source Route Failed message is usually generated when a router cannot forward the packet because source routing is enforced in the packet but the required route does not exist.
  • Bonus: There are two IP options that support source routing. Both options include a list of IP addresses specifying the routers that must be used to reach the destination. Strict source routing means that the path specified in the option must be used exactly, in sequence, with no other routers permitted to handle the packet at all. In contrast, loose source routing specifies a list of IP addresses that must be followed in sequence, but having intervening hops in between the devices on the list is allowed.

ICMP Redirect

  • ICMP Redirect message is sent by a router to notify a host that a better route is available for a particular destination.
  • A host should never send an ICMP Redirect message; Redirects must be sent only by gateways.
  • For example, in the following diagram, Host H's default gateway is R1. To reach the remote PC, Host H would send the traffic to R1 but clearly a better path in the same segment, through R2 directly, is available. R1 realizes this situation and sends an ICMP Redirect message to Host H, who then updates its gateway information accordingly.



Example debug message:  *Mar 18 06:28:54: ICMP:redirect sent to 172.16.1.1 for dest 10.1.1.1, use gw 172.16.1.200

  • R1 cannot, however, send Redirect messages at its own will. The router must fulfill specific requirements first.
  • Cisco routers send ICMP Redirect messages when all of the following conditions are met:
    • The incoming interface is the same interface on which the packet gets routed out.
    • The subnet of the source IP address is on the same subnet of the next-hop IP address of the routed packet.
    • The packet is not source routed.
    • The kernel is configured to send Redirects (By default it is but can be disabled on a per-interface basis with no icmp redirects.)
  • ICMP Redirects are disabled by default if HSRP is configured on the interface.
  • ICMP Redirects generally indicate a badly designed network.

Tuesday, August 18, 2015

CCIE 400-101: Network Principles - General Network Challenges

Asymmetric routing

  • Quite simply, asymmetric routing is when a packet takes one path to the destination and returns using another path.
  • Asymmetric routing in itself is not a problem from a TCP/IP communication perspective but it does create trouble with certain firewall setups.
  • With stateful inspection, the firewall keeps track of active connections in its state table. Packets that do not have an entry in the state table or are not explicitly permitted by an access-list, will be discarded. Due to this behavior, asymmetric routing can cause packets to be dropped.


  • Assume that the Client sends an HTTP Request to the Web Server. The packet exits the network through ASA1, which builds and records the connection state entry. 
  • Now, the HTTP Response is forwarded to ASA2 but it does not have any connection information for the packet or an access-list that would allow the packet to pass, so the HTTP Response is dropped.

Unicast flooding

  • Unicast flooding is the unintentional behavior of a switch treating a unicast packet like a broadcast. This occurs when the destination MAC address is not found in the CAM table. The switch then floods the packet (broadcast). 
  • This can happen when a device wants to send a packet to a host for which it has a valid ARP entry but the MAC address has timed out from the switch's CAM table.  
  • The primary causes for unicast flooding are asymmetric routing, STP topology changes, and CAM table overflow.

Asymmetric Routing

  • Consider the traffic flows in the diagram below:
  1. S1--VLAN 1--Switch A--Router A--VLAN 2--Switch B--VLAN 2--S2 (blue line)
  2. S2--VLAN 2--Switch B--Router B--VLAN 1--Switch A--VLAN 1--S1 (red line)


  • Let's think about what happens when traffic flows from S1 to S2. The servers are in different VLANs. Router A is the default gateway for S1, and Router B is the default gateway for S2. 
  • S1 sends an ARP request for its default gateway (Router A) and receive a reply. Meanwhile, Switch A tracks both MAC addresses (S1 and Router A) in its CAM table.
  • Router A then pushes out an ARP request and saves the MAC address of S2 in its ARP cache. Switch A also records the MAC address of S2.
  • But what happens after this point? Router A will not receive packets from S2 directly as they will traverse through Router B. Therefore, the Switch A will actually learn the MAC address of Router B, which rewrites the frame after receiving it from S2. 
  • Note that Router A still has a valid ARP entry for S2 (4 hours by default) but the MAC address will time out after 300 seconds (default). So after 5 minutes, Switch A will have no record of S2 in its CAM table but Router A will still build the frame correctly due to the valid ARP entry. This causes Switch A to receive a packet to a destination MAC address to which it has no entry. Switch A must then flood the packet.
  • The same problem will occur in the reverse direction as well.

 

STP Topology Changes

  • TCN BPDUS shorten the CAM aging time to the ForwardDelay (15 seconds by default). If the MAC address is not relearned within that interval, the entry is deleted. 
  • Flooding will occur as the switch rebuilds its CAM table.
  • Again, the end hosts likely still have valid ARP entries to build the Layer 2 packet but the switch does not know how to reach the destination MAC addresses.
  • Ports with the STP PortFast feature enabled will not cause TCNs when going to or from the forwarding state.

CAM Table Overflow

  • The CAM table is full and new MAC addresses cannot be learned. 
  • Packets destined to such MAC addresses are flooded until  space becomes available in the CAM table.
  • This is possible but rare, since most modern switches have large enough forwarding tables to accommodate MAC addresses for most designs. 
  • CAM table exhaustion can also be caused by an attack where one host starts generating frames each sourced with different MAC address - eating up all CAM table resources. Once the CAM table becomes saturated, other traffic will be flooded because new learning cannot occur. This kind of attack can be prevented by limiting the number of MAC addresses learned on untrusted ports by using the port security feature.  


Out of Order Packets

  • Out of order delivery simply means packets arriving to destination in a different order from which they were sent. 
  • Out-of-order packets can be caused by a per-packet load balancing using multiple different speed paths to reach the target, poorly configured queuing along a path or asymmetric routing.
  • In the case of queuing, out-of-order packets can occur when the packets are not forwarded in a first-in/first-out (FIFO) order. 
  • Packet reordering can affect performance and efficiency of packet in the following ways:
    1. Cause unnecessary retransmission: When the TCP receiver gets packets out of order, it sends duplicate ACKs to trigger fast retransmit algorithm at the sender. These ACKs makes the TCP sender infer a packet has been lost and retransmit it.
    2. Limit transmission speed: When fast retransmission is triggered by duplicate ACKs, the TCP sender assumes it is an indication of network congestion. It reduces  its congestion window to limit the transmission speed, which needs to grow larger from a “slow start” again.
    3. Reduce the recipient's efficiency: TCP receiver has to hand in data to the upper layer in order. When reordering happens, TCP has to buffer all the out-of-order packets until getting all packets in order. Meanwhile, the upper layer gets data in burst rather than smoothly, which also reduce the system efficiency as a whole. 

Impact of Microburst

  • Microbursting is a phenomenon where rapid bursts of data traffic are sent in quick succession, leading to periods of full line-rate transmission that can overflow packet buffers.
  • Interface ignores and overruns typically point to microbursts. This is indicative of the Rx ring and the corresponding packet buffer being overwhelmed due to data bursts over extremely short periods of time (microseconds).
  • Microbursts are difficult to detect because the interface counters typically average the traffic over a 5-minute interval by default.
  • Example: The interface utilization is only 20-30% of the available bandwidth but the output drop count is high. 

Switch#show interface Fa0/1 | include duplex|output drops|rate
  Full-duplex, 10Mb/s, media type is 10/100BaseTX
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 7228
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 2921000 bits/sec, 477 packets/sec


  • If the output drops increase with interface utilization closing on line speed, more bandwidth is needed.
  • If the output drops increment, but the overall traffic utilization of that interface is otherwise low, most likely the network is experiencing some type of bursty traffic. 
  • To troubleshoot microbursts, a packet sniffer that can capture traffic over a long period of time and allow analyzing of the results in a graph with very short time intervals is needed.

Sunday, August 16, 2015

CCIE 400-101: Network Principles - CEF Load Balancing and Polarization

Overview

  • Cisco IOS software supports two modes of load balancing: per-destination or per-packet. 
  • In per-destination mode, all packets for a given path are forwarded along the same path. This preserves packet order but might utilize the links unequally.
  • Per-packet load balancing guarantees equal utilization across all links but packets may arrive out of order and different delay may occur. 

CEF Load Balancing Algorithms

  • For per-destination load balancing, the router computes a hash out of the source and destination IP addresses. This hash points to exactly one of the entries in the adjacency table. 
  • If per-packet load balancing is used, the packets are distributed round robin over the available paths. 
  • Per-destination load balancing is enabled by default with CEF. 
  • Per-destination load balancing is the method of choice in most situations. 
  • Use per-packet load-balancing only between a single source-destination host pair to ensure equal link usage. 
  • The CEF load balancing algorithm is selected with the ip cef load-sharing algorithm command. 
  • Original algorithm: uses the same algorithm on every router and causes CEF polarization. 
  • Universal algorithm: allows each router to make a different decision for each source-destination address pair, which resolves load sharing imbalances. The default. 
  • Tunnel algorithm: designed to balance the per-packet load when only a few source and destination pairs are involved. 
  • Include-ports algorithm: uses the Layer 4 source and destination ports as part of the load balancing decision.

CEF Load Balancing Internal Mechanics

  • The CEF table points to 16 hash buckets, which point to the adjacency table for parallel paths.
  • Active paths are assigned internally to the 16 hash buckets. The path-to-bucket assignment varies with the type of load balancing and the number of active paths.
  • The 16 buckets are evenly filled with the active paths. If 16 isn't divisible by the number of active paths, the last few buckets that represent the remainder are disabled. 
  • The result of the hash function is used to pick one of the enabled buckets, and thus which path to use for the traffic flow.
  • Example: The hash buckets for two and three active paths. With three links (16 is not divisible by 3), the last bucket is disabled.



The following output shows the hash bucket assignments when two active paths are available.
RouterB#show ip cef  192.168.20.0 internal 
 192.168.20.0/24, version 66, per-destination sharing 
0 packets, 0 bytes 
  via 20.20.20.1, Serial2, 0 dependencies 
    traffic share 1 
    next hop 20.20.20.1, Serial2 
    valid adjacency 
  via 30.30.30.1, Serial3, 0 dependencies 
    traffic share 1 
    next hop 30.30.30.1, Serial3 
    valid adjacency 
  0 packets, 0 bytes switched through the prefix 
  Load distribution: 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 (refcount 1) 

 The following output shows the hash bucket assignments when three active paths are available.
RouterB#show ip cef  192.168.20.0 interface 
 192.168.20.0/24, version 64, per-destination sharing 
0 packets, 0 bytes 
  via 20.20.20.1, Serial2, 0 dependencies 
    traffic share 1 
    next hop 20.20.20.1, Serial2 
    valid adjacency 
  via 30.30.30.1, Serial3, 0 dependencies 
    traffic share 1 
    next hop 30.30.30.1, Serial3 
    valid adjacency 
  via 10.10.10.1, Serial1, 0 dependencies 
    traffic share 1 
    next hop 10.10.10.1, Serial1 
    valid adjacency 

  0 packets, 0 bytes switched through the prefix 
  Load distribution: 0 1 2 0 1 2 0 1 2 0 1 2 0 1 2 (refcount 1) 
 

CEF Polarization

  • In short, CEF polarization refers to the utilization of a single ECMP link due to the use of the same hash algorithm and the same hash input. 
  • When there are two paths available, the router performs exclusive OR (XOR) operation on the lower-order bits of the source IP (SIP) and destination IP (DIP). 
  • Consider the graph below. Say hosts behind R1 want to reach R6's directly connected network 6.6.6.6/32. Traffic flows, whose hash result in 0, traverse through R2. Similarly, if the hash is equal to 1, packets are sent to R3. Now, if the routers use the same hash algorithm and the same hash input, the result will always be 1 for traffic that arrived on R3 from R1. Therefore, the equal-cost path through R4 will remain unused (the hash result is never 0). 



 Avoiding CEF Polarization

  1. Alternate between default (SIP+DIP) and full (SIP+DIP+Layer 4 ports) hashing input configuration at each layer of the network.
  2. Alternate between an even and odd number of ECMP links at each layer of the network.
  3. Use the unique-ID/universal-ID. The hash function is seeded with a 32-bit randomly generated, router-specific value, which ensures that the same source/destination pair hash into a different value on different routers along the path.

Monday, August 10, 2015

CCIE 400-101: Network Principles - CEF

1.1.b Identify Cisco express forwarding concepts

  • The switching process is responsible for moving packets between the interfaces.
  • There are three different switching path methods available: process switching, fast switching, and Cisco Express Forwarding (CEF).
  • With process switching, every packet is inspected and forwarded by the CPU. This method is very tedious, resource-intensive, and slow.
  • Fast switching also uses process switching, but only for the first packet of a given traffic flow. Subsequent packets are forwarded using a cache entry instead of an individual lookup for each packet.
  • CEF pre-builds and pre-computes forwarding information based on the routing table and ARP before receiving any packets. The information is stored in separate data structures (FIB and adjacency table), which are designed for quick lookups without involving the CPU.

Routing Information Base (RIB)

  • Essentially, the RIB aka the routing table is a collection of best paths to each network that the router knows about.
  • Each routing protocol has its own data structures (OSPF database, EIGRP topology table, BGP table etc.) that store information from routing updates.
  • Each routing protocol also runs its own route selection algorithm to find its own best path to each unique prefix.
  • The same network can be learned from different routing protocols or different neighbors but only one path is installed into the RIB based on the lower administrative distance and metric values. (Not considering load balancing here.)
  • Basically, the RIB contains only the best of the best routes.
  • The RIB can be viewed with the show ip route command.

Router#show ip route

Gateway of last resort is 10.142.124.1 to network 0.0.0.0

S*    0.0.0.0/0 [1/0] via 10.142.124.1
      10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
C        10.142.124.0/24 is directly connected, FastEthernet0/0
L        10.142.124.151/32 is directly connected, FastEthernet0/0
S        10.142.125.0/24 [1/0] via 10.42.24.1



Forwarding Information Base (FIB)

  • The FIB is conceptually similar to the RIB.
  • The FIB maintains a mirror image of the RIB.
  • When routing or topology changes occur in the network, the IP routing table is updated, and those changes are reflected in the FIB. 
  • Think about a "normal" routing traffic flow: decapsulate the packet to find the destination IP address, do a routing table lookup, select the best match, find the exit interface, and resolve Layer 2 next-hop information. This is rather inefficient, needing to consult different tables to find the necessary information to rebuild and forward the packet.
  • Now, the FIB eliminates the need to do recursive RIB lookups because it resolves next-hops to outgoing interfaces and computes the Layer 2 information prior to sending any packets.
  • The show ip cef command displays the contents of the FIB.

Router#show ip cef
Prefix                 Next Hop             Interface
0.0.0.0/0              10.142.124.1         FastEthernet0/0
0.0.0.0/8              drop
0.0.0.0/32             receive
10.142.124.0/24        attached             FastEthernet0/0
10.142.124.0/32        receive              FastEthernet0/0
10.142.124.1/32        attached             FastEthernet0/0
10.142.124.4/32        attached             FastEthernet0/0
10.142.124.8/32        attached             FastEthernet0/0
10.142.124.107/32      attached             FastEthernet0/0
10.142.124.151/32      receive              FastEthernet0/0
10.142.124.203/32      attached             FastEthernet0/0
10.142.124.218/32      attached             FastEthernet0/0
10.142.124.222/32      attached             FastEthernet0/0
10.142.124.254/32      attached             FastEthernet0/0
10.142.124.255/32      receive              FastEthernet0/0
10.142.125.0/24        10.142.124.1         FastEthernet0/0
127.0.0.0/8            drop
224.0.0.0/4            drop
224.0.0.0/24           receive
240.0.0.0/4            drop
255.255.255.255/32     receive



  • There are different kinds of CEF adjacencies: null, glean (attached), punt, drop, and discard.
  • Null adjacency: Packets destined to the Null0 interface are dropped.
  • Glean (attached) adjacency: When a router is directly connected to a multiple hosts, the router maintains a prefix for the subnet rather than individual hosts. The subnet prefix points to a glean adjacency. When packets need to be forwarded to a specific host, the adjacency database is gleaned for the specific prefix.
  • Punt adjacency: Features that require special handling or features that are not yet supported in conjunction with CEF are forwarded to the next switching layer for handling (the router CPU).
  • Drop adjacency: Packets are dropped, but the prefix is checked.
  • Discard adjacency: Purposefully drops packets as a result of a configured feature that discards the packets.


Adjacency Table

  • Network nodes are adjacent if they can reach each other with a single hop. 
  • CEF uses the adjacency table to prepend Layer 2 addressing information. 
  • Maintains Layer 2 next-hop addresses for all FIB entries. 
  • Populated as adjacencies are discovered. 
  • Each time an adjacency entry is created (through the ARP protocol, for example), a link-layer header for that node is precomputed and stored in the adjacency table. 
  • Once a route is determined, it points to a next hop and corresponding adjacency entry. 
  • For each resolved path to a destination prefix, a pointer is added for the adjacency corresponding to the next-hop interface for that path.
  • The adjacency table can be examined with the show adjacency command.

Router#show adjacency
Protocol Interface                 Address
IP       FastEthernet0/0           10.142.124.1(11)
IP       FastEthernet0/0           10.142.124.4(7)
IP       FastEthernet0/0           10.142.124.8(7)
IP       FastEthernet0/0           10.142.124.107(7)
IP       FastEthernet0/0           10.142.124.203(7)
IP       FastEthernet0/0           10.142.124.218(7)
IP       FastEthernet0/0           10.142.124.222(7)
IP       FastEthernet0/0           10.142.124.254(7)



Let's look at a specific entry for 10.142.124.8.

Router#show adjacency 10.142.124.8 detail
Protocol Interface                 Address
IP       FastEthernet0/0           10.142.124.8(7)
                                   0 packets, 0 bytes
                                   epoch 0
                                   sourced in sev-epoch 1
                                   Encap length 14
                                   002590956536CCEF48117D180800
                                   ARP



The last line says that the information was drawn from the ARP cache. The most information part is the seemingly random string 002590956536CCEF48117D180800, which actually lists the destination MAC address (002590956536), the source MAC address (CCEF48117D18), and the protocol type (0x800). This information can be verified by referring to the ARP cache and checking the MAC address of the exit interface.

Router#show arp 10.142.124.8
Protocol  Address       Hardware Addr   Interface
Internet  10.142.124.8  0025.9095.6536  FastEthernet0/0


Router#show interface FastEthernet0/0
FastEthernet0/0 is up, line protocol is up
  Hardware is i82546, address is ccef.4811.7d18 (bia ccef.4811.7d18)
  Internet address is 10.142.124.151/24



Saturday, August 8, 2015

CCIE 400-101: Network Principles - IOS and IOS XE

1.1.a Describe basic software architecture differences between IOS and IOS XE

  • IOS XE is an evolution of IOS.
  • Historically, an overwhelming majority of the internal software processes are run using Cisco IOS memory. 
  • IOS XE introduces a distributed software architecture that moves many operating system responsibilities out of the IOS process. 
  • This architecture allows for better allocation of memory so the router can run more efficiently and allows increased high availability at all times. 
  • IOS XE 15.0 runs as a single daemon within a modern Linux OS. Additional system functions run as separate processes.
  • IOS XE retains the exact same look and feel of IOS, while providing improved functionality.
  • Some of the IOS XE benefits include:
  1. Multi-Core CPU: Running IOS and separate processes enables load balancing the multi-core CPU, each process using a different core.
  2. Control Plane and Data Plane Separation: The Control Plane and Data Plane are separated with the introduction of the Forwarding Feature Manager (FFM) and its standard interface to the Forwarding Engine Driver (FED).
  3. Platform Abstraction: IOS XE allows the platform dependent code to be abstracted from a single monolithic image to move to a more purely protocol independent IOS process.
  4. Application Integration: prior to IOS XE, the only way to integrate functionality into an IOS was to either port the functionality into the IOS or run it as a service blade outside of IOS.

  • The actual IOS XE comes in seven different software modules (sub-packages).
  • Each sub-package provides specific functions:
  1. RPBase provides the operating system software for the Route Processor.
  2. RPControl controls the control plane processes that interface between the IOS process and the rest of the platform.
  3. RPAccess provides software required for router access.
  4. RPIOS provides the Cisco IOS kernel, which is where IOS features are stored and run. Each consolidated package has a different RPIOS.
  5. ESPBase provides the ESP operating system and control processes, and the ESP software.
  6. SIBBase controls the SIP operating system and control processes.
  7. SIPSPA provides the SPA driver and Field Programmable Device (FPD) images.

Friday, May 22, 2015

CCIE OCG: Classification and Marking

IP Precedence and DSCP Compared

  • IP header includes a 1-byte Type of Service (ToS) field.
  • The ToS was intended to be used to mark a packet for treatment with QoS tools.
  • The ToS field is further divided: the high-order 3 bits defined as IP Precedence (IPP).
  • Bits 3 through 6 of the ToS byte included flag fields which were not used very often.
  • Along came Differentiated Services (DiffServ).
  • DiffServ needed more than 3 bits to mark packets, so it redefined the ToS byte.
  • The ToS was renamed Differentiated Services (DS) field, and IPP was replaced with a 6-bit field called Differentiated Services Code Point (DSCP).
  • The low-order 2 bits is used with QoS Explicit Congestion Notification (ECN).
 
IP Precendence Values
Name Decimal Value Binary Value
Routine Precedence 0 000
Priority Precedence 1 001
Immediate Precedence 2 010
Flash Precedence 3 011
Flash Override Precedence 4 100
Critical Precedence 5 101
Internetwork Control Precedence 6 110
Network Control Precedence 7 111


DSCP Settings and Terminology

  • For example, DSCP 46 is Expedited Forwarding (EF). 
  • EF should be given queuing preference for minimal latency but the packets should be prevented from congesting the link.
  • The suggested settings and associated QoS behavior are called Per-Hop Behaviors (PHB). 
  • IPP and DSCP overlap.
  • Class Selector (CS) PHB provide backward compatibility between DSCP and IPP. 
  • IPP can still understand DSCP values.
  • Packets with larger CS DSCPs should be given better queuing preference than packets with lower CS DSCPs.


Default and Class Selector DSCP Values

DSCP Class Selector  Binary DSCP Value IPP Value IPP Names
Default/CS0 00000000 000 Routine
CS1 00100000 001 Priority
CS2 01000000 010 Immediate
CS3 01100000 011 Flash
CS4 10000000 100 Flash Override
CS5 10100000 101 Critical
CS6 11000000 110 Internetwork Control
CS7 11100000 111 Network Control


  • The Assured Forwarding (AF) defines four classes for queuing purposes and three levels of drop probability inside each queue.
  • AF conforms to the following format:
  1. AFxy, where x implies one of the queues (1-4) and y implies one the three drop priorities (1-3).
  2. The higher the x, the better queuing treatment a packet should get.
  3. The higher the y, the worse the drop treatment for those packets (more likely to get dropped).
  • For example, AF11 should get worse queuing treatment than AF23. However, AF11 has better drop treatment than AF23.


Assured Forwarding DSCP Values

Queue Class  Low Drop Probability Medium Drop Probability High Drop Probability

Name/Decimal/Binary Name/Decimal/Binary Name/Decimal/Binary
1 AF11/10/001010 AF12/12/001100 AF13/14/001110
2 AF21/18/010010 AF22/20/010100 AF23/22/010110
3 AF31/26/011010 AF32/28/011100 AF33/30/011110
4 AF41/34/100010 AF42/36/100100 AF43/38/100110


  • Conversion Formula: If AFxy, then 8x-2y.
  • Example: AF41 = 8*4+2*1 = 34.
  • Expedited Forwarding (EF):
  1. Queue EF packets so that they get scheduled quickly, to give them low latency.
  2. Police the EF packets so that they do not consume all bandwidth on the link or starve other queues. 
  • IP packets get encapsulated inside other headers, which have their own QoS fields.

 

Ethernet LAN Class of Service (CoS)

  • Ethernet supports a 3-bit QoS marking field but it only exists when the Ethernet header includes either a 802.1Q or ISL header.
  • 802.1Q: the 3 most significant bits of the 2-byte Tag Control field (user-priority bits).
  • ISL: the 3 least significant bits from the 1-byte User field. 
  • Frame Relay and ATM have a single bit to define drop probability: Discard Eligibility (DE) and Cell Loss Priority (CLP), respectively.
  • MPLS defines a 3-bit field called the MPLS Experimental (EXP) bit that is intended for general QoS marking.

 

Location for Marking and Matching

  • For classification: On ingress only, and only if the interface supports that particular header field.
  • For marking: On egress only, and only if the interface supports that particular header field.

Marking Field Summary
Field Location Length
IP Precedence IP Header 3 bits
IP DSCP IP Header 6 bits
DS field IP Header 1 byte
ToS byte IP Header 1 byte
CoS ISL and 802.1Q header 3 bits
Discard Eligibel (DE) Frame Relay header 1 bit
Cell Loss Priority (CLP) ATM cell header 1 bit
MPLS Experimental MPLS header 3 bits



Cisco Modular QoS CLI

  • Intended to simply QoS configuration. 
  • MQC-based tools all begin with the phrase "class-based".
  • Three major commands:
    1. The class-map command defines the matching parameters for classifying packets into service classes.
    2. The PHB actions (marking, queuing, and so on) are configured under a policy-map command.
    3. The policy-map is enabled on an interface by using a service-policy command. 
  • Packets are classified using the match command inside the class-map. 
  • The match command has many options for matching packets, including QoS fields, ACLs, and MAC addresses. 
  • Class-map names are case-sensitive. 
  • The match protocol commands means that IOS uses NBAR to perform that match. 
  • The match any command matches any and all packets. 
  • CEF is required for class-based marking. Without it, the class-map and policy-map configuration would be allowed, but the service-policy command would be rejected. 
  • Class-maps can use multiple match commands, and even nest class-maps inside other class-maps.
  • Up to four (CoS and IPP) or eight (DSCP) values can be listed on a single match cos, match precedence, or match dscp command. If any is found in the packet, the statement is matched.
  • If the class-map has multiple match commands, the match-any or match-all (default) defines whether a logical OR or a logical AND is used, respectively.
  • The match class command refers to another class-map by name, nesting the logic. It is considered a match if the referenced class-map also results in a match.

! This class-map matches packets that are permitted by ACL 101 and have an IPP of 5.
class-map match-all EXAMPLE-A
 match access-group 101
 match precedence 5


! This class-map matches packets that are permitted by ACL 101 or have a DSCP of AF21.
class-map match-any EXAMPLE-B
 match access-group 101
 match dscp AF21


! This class-map will never match anything because a packet cannot have two DSCP values.
class-map match-all EXAMPLE-C
 match dscp 0
 match dscp 1


! This example shows the use of nested class-maps.
class-map match-all CHILD-CLASS
 match access-group 101
 match precedence 5
!
class-map match-any PARENT-CLASS
 match class CHILD-CLASS
 match cos 5


Mach: Packets permitted by ACL 101 AND marked with IPP 5 OR frames with CoS 5.


Classification Using NBAR

  • NBAR classifies packets that are normally difficult to classify.
  • Some applications use dynamic port numbers, so static match command will not work.
  • NBAR can do deep packet inspection, looking at the host name, URL, MIME, and so on.
  • NBAR can look past the TCP and UDP headers to recognize application-specific information.

 

Configuration

  • CB marking requires CEF.
  • Packets are classified based on the logic in MQC class-maps.
  • An MQC policy-map refers to one or more class-maps; packets classified into that class are then marked.
  • CB marking is enabled in either ingress or egress direction.
  • A policy-map is processed sequentially; after a packet has matched a class, it is marked based on the set command(s) defined for that class.
  • Multiple set commands can be configured in one class to set multiple fields, for example to set both DSCP and CoS.
  • Packets that do not explicitly match a defined class are considered to have matched a special class called class-default.
  • For any class inside the policy-map for which there is no set command, packets in that class are not marked.

 Example:
  • Frames entering with CoS 5 will be marked with DSCP EF.
  • Frames entering with CoS 1 will be marked with DSCP AF11.
  • Frames entering with any other CoS will be marked with DSCP 0.
  • Packets exiting with DSCP EF will be marked with CoS 5.
  • Packets exiting with DSCP AF11 will be marked with CoS 1.
  • Packets exiting with any other DSCP will be marked with CoS 0.

class-map COS1
 match cos 1
!
class-map COS5
 match cos 5
!
class-map AF11
 match dscp af11
!
class-map EF
 match dscp EF
!
policy-map COS-TO-DSCP-IN
 class COS1
  set dscp af11
 class COS5
   set dscp ef
 class class-default
  set ip dscp default
!
policy-map DSCP-TO-COS-OUT
 class AF11
  set cos 1
 class EF
  set cos 5
 class class-default
  set cos 0
!
interface FastEthernet0/0.10
 encapsulation dot1q 10
 service-policy input COS-TO-DSCP-IN
 service-policy output DSCP-TO-COS-OUT


Only interfaces configured for 802.1Q accept service-policy commands that reference policy-maps that either classify or mark based on CoS.


CB Marking Design Choices

  • Packets should generally be marked as close to the ingress point of the packet as possible. 
  • However, the earliest point might not be a trusted device.
  • Rule: "Mark as close to the ingress edge of the network as possible, but not so close to the edge that the marking is made by an untrusted device." 
  • Cisco recommends not to use more than four or five different service classes for data traffic.


Recommended Values For Marking

Type of Traffic CoS IPP DSCP
Voice Payload 5 5 EF
Video Payload 4 4 AF41
Voice/Video Signaling 3 3 CS3
Mission-Critical Data 3 3 AF31, AF32, AF33
Transactional Data 2 2 AF21, AF22, AF23
Bulk Data 1 1 AF11, AF12, AF13
Best Effort 0 0 BE
Scavenger 0 0 2, 4, 6


Marking Using Policers

  • Traffic policers measure entering/exiting an interface -determining whether a configured traffic contract has been exceeded.
  • Two components: a traffic rate (bits/second), and a burst size (bytes).
  • Traffic within the limits is conformed to the contract.
  • Traffic over the rate or burst is exceeding the contract.
  • The simplest form of policing discards packets that exceeded the contract.
  • Another option: mark down packets instead of dropping them.
  • Example: a policer could re-mark AF11 packets that exceeded the contract with a new DSCP value of AF13, but not discard the packet.

QoS Pre-Classification

  • When packets are tunneled or encrypted, the original headers and packet contents are unavailable for inspection.
  • QoS pre-classification allows a VPN endpoint router to make egress QoS decisions based on the original traffic, before encapsulation.


AutoQoS

  • A macro that helps automate class-based QoS configuration.
  • It creates and applies QoS configurations based on Cisco best practice recommendations.
  • Two flavors: AutoQoS for VoIP and AutoQoS for the Enterprise.

AutoQoS for VoIP

  • Automatic QoS configuration for voice and video applications.
  • Assumes two types of ports on switches: user access and uplink.
  • Also assumes that a user access port might or might not have an IP phone.
  • When enabled for any interface, the command starts a macro that globally enables QoS, configures interface ingress and egress queues, configures class-maps and policy-maps, and applies the policy-map to the interface.
  • Enabled with the interface-level command auto qos [cisco-phone | cisco-softphone].
  • If no phone is found, all traffic is marked down to DSCP 0 and treated as best effort.
  • If a phone is found, the received QoS markings are trusted.
  • Voice and video control traffic, real-time video traffic, voice traffic, routing protocol traffic, and STP BPDU traffic has priority (expedited forwarding).
  • Uplink ports use the auto qos voip trust command.

AutoQoS for the Enterprise

  • Automates the QoS configuration for VoIP and other network applications.
  • Meant to be used for WAN links.
  • It detects the types and amounts of network traffic and then creates policies based on that.
  • Two steps:
  1. Discover the traffic.
  2. Provide recommended QoS configuration.
  • Enable discovery: auto discovery qos [trust].
  • Make sure CEF is enabled, the interface bandwidth is configured, and no QoS configuration is applied.
  • Use the trust keyword to trust received markings.
  • Traffic discovery uses NBAR to learn the types and amounts of traffic on each enabled interface.
  • After traffic discovery has collected enough information, run auto qos on the interface.
  • This runs a macro that creates templates based on the traffic collected, creates class-maps to classify traffic, and creates a policy-map to allocate bandwidth and mark the traffic.

Monday, May 11, 2015

400-101: BGP, Part VI:II

3.7e Implement and Troubleshoot Scalability


3.7e(ii) Confederations

BGP confederations introduce the concept of creating a number of smaller autonomous systems within an original autonomous system. The sub-ASs exchange BGP update between them using intra-confederation eBGP sessions.

With BGP confederations, member-ASs are never visible from outside the confederation. This allows private AS numbers (64512 to 65535) to be used inside the confederation. BGP routers inside the member-AS must still maintain a full mesh of iBGP sessions. EBGP sessions are established between member-ASs. These eBGP sessions behave slightly differently from classic or real eBGP sessions (referred to as intra-confederation eBGP sessions to differentiate them from true eBGP sessions).

When a router sends a BGP update over an intra-confederation eBGP session, it prepends the member-AS number to the AS_PATH attribute. This information is maintained by the routers within the confederation. When a router sends a BGP update over a true eBGP session, it removes the part of the AS_PATH that describes the member-AS numbers and prepends the official AS number to the AS_PATH. As a result, the confederation appears as one single AS to the outside world.

When a router sends an update over an intra-confederation eBGP, it prepends the member-AS number in parenthesis. Routers do not modify the AS_PATH when transmitting over an iBGP session. When advertising the route to a true eBGP peer, the router removes the parenthesized information and replaces it with the official AS number of the confederation.

All else being equal, the BGP best path decision process chooses normal eBGP routes over confederation eBGP over iBGP routes.

A router that does not support BGP confederations cannot operate inside a BGP confederation. A router not supporting BGP confederations will reject an AS_PATH with unknown segement type.

Although intra-confederation eBGP sessions have eBGP-like properties, they still run inside a real AS and therefore share some properties with iBGP sessions to achieve the same results. Similar to iBGP, the BGP attributes of local preference, MED, and next-hop are not changed across intra-confederation eBGP sessions. All routers in all member-ASs inside the confederation consequently see the same next-hop address when doing recursive routing. The entire confederation should then use the same IGP to resolve the BGP next-hop address.

As a summary, the route reflectors modify the BGP split horizon rule and confederations modify AS_PATH processing. Intra-confederation eBGP sessions act like eBGP sessions from a session establishment perspective, and they act like iBGP sessions from the BGP attribute-propagation perspective.

 

Configuration Example


The confederation AS 12 is divided into two sub-AS 65001 and 65002. Routers inside each sub-AS are configured with full-mesh BGP peering. Each router uses 150.1.x.x/32 as its router-ID and advertises a Loopback network (as depicted in the topology diagram). Peer groups are used to cut down on the configuration. OSPF is enabled for full IP connectivity inside the confederation.

R1#
router bgp 65001
 no synchronization
 bgp log-neighbor-changes
 bgp confederation identifier 12
 bgp confederation peers 65002
 network 1.1.1.0 mask 255.255.255.0
 neighbor CONFED-PEER peer-group
 neighbor CONFED-PEER remote-as 65001
 neighbor CONFED-PEER update-source Loopback0
 neighbor 10.1.17.7 remote-as 78
 neighbor 10.1.18.8 remote-as 78
 neighbor 150.1.2.2 remote-as 65002
 neighbor 150.1.2.2 ebgp-multihop 2
 neighbor 150.1.2.2 update-source Loopback0
 neighbor 150.1.3.3 peer-group CONFED-PEER
 neighbor 150.1.4.4 peer-group CONFED-PEER
 no auto-summary


 R2#
router bgp 65002
 no synchronization
 bgp log-neighbor-changes
 bgp confederation identifier 12
 bgp confederation peers 65001
 network 2.2.2.0 mask 255.255.255.0
 neighbor CONFED-PEER peer-group
 neighbor CONFED-PEER remote-as 65002
 neighbor CONFED-PEER update-source Loopback0
 neighbor 150.1.1.1 remote-as 65001
 neighbor 150.1.1.1 ebgp-multihop 2
 neighbor 150.1.1.1 update-source Loopback0
 neighbor 150.1.5.5 peer-group CONFED-PEER
 neighbor 150.1.6.6 peer-group CONFED-PEER
 no auto-summary


 R8#
router bgp 78
 no synchronization
 bgp log-neighbor-changes
 network 8.8.8.0 mask 255.255.255.0
 neighbor 10.1.18.1 remote-as 12
 no auto-summary



Let's see what the routers (R5) inside the sub-AS are learning.

R5#show ip bgp
BGP table version is 9, local router ID is 150.1.5.5
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*>i1.1.1.0/24       150.1.1.1                0    100      0 (65001) i
*>i2.2.2.0/24       150.1.2.2                0    100      0 i
*>i3.3.3.0/24       150.1.3.3                0    100      0 (65001) i
*>i4.4.4.0/24       150.1.4.4                0    100      0 (65001) i
*> 5.5.5.0/24       0.0.0.0                  0         32768 i
*>i6.6.6.0/24       150.1.6.6                0    100      0 i
*>i7.7.7.0/24       10.1.17.7                0    100      0 (65001) 78 i
*>i8.8.8.0/24       10.1.18.8                0    100      0 (65001) 78 i



Note that the next-hop and local preference values are retained across confederation-EBGP peering (example route 8.8.8.0/24 from R8). The AS number inside the parenthesis signifies that the route has traversed through a confederation sub-AS.

R5#show ip bgp 8.8.8.0/24
BGP routing table entry for 8.8.8.0/24, version 9
Paths: (1 available, best #1, table Default-IP-Routing-Table)
  Not advertised to any peer
  (65001) 78
    10.1.18.8 (metric 30) from 150.1.2.2 (150.1.2.2)
      Origin IGP, metric 0, localpref 100, valid, confed-internal, best



According to BGP confederation rules, true eBGP peers should not see the sub-AS numbers. They don't. From R8's perspective, the routes originated from AS 12.

R8#show ip bgp
BGP table version is 8, local router ID is 150.1.8.8
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 1.1.1.0/24       10.1.18.1                0             0 12 i
*> 2.2.2.0/24       10.1.18.1                              0 12 i
*> 3.3.3.0/24       10.1.18.1                              0 12 i
*> 4.4.4.0/24       10.1.18.1                              0 12 i
*> 5.5.5.0/24       10.1.18.1                              0 12 i
*> 6.6.6.0/24       10.1.18.1                              0 12 i
*> 8.8.8.0/24       0.0.0.0                  0         32768 i