Friday, January 15, 2016

CCIE 400-101: Layer 2 Technologies - dot1q, Native VLAN

dot1Q

  • IEEE 802.1Q (dot1q) is a industry standards based implementation of carrying traffic for multiple VLANs over a single trunk link between two Ethernet switches.
  • This means adding an identifier (tag) to frames to indicate what VLAN the frame belongs to.
  • 802.1Q does not encapsulate the original frame. Instead, for Ethernet frames, it adds a 32-bit (4-byte) field between the source MAC address and the EtherType/Length fields of the original frame.

  • TPID - Tag Protocol Identifier (16 bits): set to a value of 0x8100 to identify the frame as an IEEE 802.1Q tagged frame. 
  • Priority (3 bits): indicates the priority level (0 through 7) used for QoS. 
  • CFI - Canonical Format Indicator (1 bit): specifies if the MAC address is in noncanonical (1) or canonical (0) format. 
  • VID - VLAN Identifier (12 bits): uniquely identifies the VLAN which the frame belongs to.

 

Native VLAN

  • In short, the ONLY difference between a regular VLAN and the native VLAN is that frames from/to the native VLAN are carried untagged. 
  • As explained in dot1q section, VLANs are identified by adding a tag to the frame. 
  • What happens if a trunk port receives a frame without this tag? Which VLAN does the frame belong to? 
  • Answer: the native VLAN. 
  • If a switch receives an untagged frame, it associates that frame to the native VLAN. 
  • When a switch sends out a frame belonging to the native VLAN, it removes the tag. 
  • The native VLAN is configurable on a per port basis with the switchport trunk native vlan <vlan-id> command. 
  • The default native VLAN is VLAN 1.

References

ISL and IEEE 802.1Q Frame Format
VLAN Trunking Using IEEE 802.1Q
Network Engineering Stack Exchange - 19377

Sunday, January 10, 2016

CCIE 400-101: Layer 2 Technologies - CDP, LLDP, UDLD

CDP 

  • The Cisco Discovery Protocol (CDP) is a proprietary Layer 2 protocol used to share information about other directly connected Cisco equipment.
  • CDP runs on all of the following Cisco manufactured equipment: routers, switches, bridges, and access servers.
  • A Cisco device enabled with CDP sends out periodic interface updates every 60 seconds to a multicast address 01-00-0c-cc-cc-cc in order to make itself known to neighbors.
  • The holdtime specifies the lifetime of an entry in the table - if no announcements are received from a device for a period in excess of the holdtime, the device information is discarded (default 180 seconds).
  • Cisco devices never forward CDP packets.
  • CDP can also be used for On-Demand Routing (ODR), which is a method of including routing information in CDP announcements so that dynamic routing protocols do not need to be used in simple networks.
  • CDP is enabled by default, but can be globally disabled with the no cdp run command or per interface with the no cdp enable command.
  • The show cdp neighbors command displays this information:
    • type of device that is discovered
    • name of the device
    • number and type of the local interface (port)
    • number of seconds the CDP advertisement is valid for the port
    • device type
    • device product number
    • port ID
  • The details contained in CDP announcements is easily extended due to the use of the type-length-value (TLV) frame format.

Example:

Router#show cdp neighbors
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
                  S - Switch, H - Host, I - IGMP, r - Repeater

Device ID        Local Intrfce     Holdtme    Capability  Platform  Port ID
R2-AGS           Ser 1              129          R        2500      Ser 0
R6-2500          Eth 0              144          R        4000      Eth 0
Router#


  • The show cdp neighbors detail and show cdp entry commands displays additional information about the neighboring devices that include network-layer protocol information and version.

Example:

router#show cdp neighbors detail
-------------------------
Device ID: lab-7206
Entry address(es):
IP address: 172.19.169.83
Platform: cisco 7206VXR, Capabilities: Router
Interface: Ethernet0, Port ID (outgoing port): FastEthernet0/0/0
Holdtime : 123 sec
Version :
Cisco Internetwork Operating System Software
IOS (tm) 5800 Software (C5800-P4-M), Version 12.1(2)
Copyright (c) 1986-2002 by Cisco Systems, Inc.
advertisement version: 2
Duplex: half
-------------------------
Device ID: lab-as5300-1
Entry address(es):
IP address: 172.19.169.87
Platform: cisco AS5300, Capabilities: Router
--More--

 

router#show cdp entry lab-7206
-------------------------
Device ID: lab-7206
Entry address(es):
IP address: 172.19.169.83
Platform: cisco 7206VXR, Capabilities: Router
Interface: Ethernet0, Port ID (outgoing port): FastEthernet0/0/0
Holdtime : 123 sec
Version :
Cisco Internetwork Operating System Software
IOS (tm) 5800 Software (C5800-P4-M), Version 12.1(2)
Copyright (c) 1986-2002 by Cisco Systems, Inc.
advertisement version: 2
Duplex: half



  • CDP can also be enabled/disabled per interface with the [no] cdp enable command.
  • To enable Cisco Discovery Protocol Version 2 (CDPv2) advertising functionality on a device, use the cdp advertise-v2 command.
  • CDPv2 has three additional type-length values (TLVs): VTP Management Domain Name, Native VLAN, and full/half-Duplex.

LLDP

  • The Link Layer Discovery Protocol (LLDP) is basically a standards-based (IEEE 802.1AB) equivalent of the Cisco proprietary CDP. 
  • Similar to CDP, LLDP also advertises and receives information about identity and capabilities between Layer 2 peers. 
  • LLDP Data Units (LLDPDUs) are sent to the multicast destination MAC address01:80:c2:00:00:0e.
    This address is defined within a range of addresses reserved by the IEEE for protocols that are to be constrained to an individual LAN. 
  • LLPDUs are not forwarded to other switches. 
  • See: Tutorial on the Link Layer Discovery Protocol.

UDLD

  • Problem:  If a blocking port stops receiving BPDUs from its upstream (designated) switch on the port, STP eventually ages out the STP information for the port and moves it to the forwarding state. This creates a forwarding loop. 
  • How is it possible for the switch to stop receiving BPDUs while the port is up? The reason is unidirectional link. 
  • A link is unidirectional when both sides of the connection are up but the local side is not receiving the packets sent by the remote side while remote side receives packets sent by local side. 
  • To detect a unidirectional link before a forwarding loop is created, Cisco developed the (aptly named) UDLD protocol. 
  • UDLD is a Layer 2 protocol that works with the Layer 1 mechanisms to determine the physical status of a link. At Layer 1, auto-negotiation takes care of physical signaling and fault detection. UDLD performs tasks that auto-negotiation cannot perform, such as detecting the identities of neighbors and shutting down misconnected ports. 
  • UDLD works by exchanging protocol packets between the neighboring devices using the multicast MAC address 01:00:0C:CC:CC:CC. 
  • Both devices on the link must support UDLD and have it enabled on respective ports. 
  • Each switch port configured for UDLD sends UDLD protocol packets that contain the port's own device/port ID, and the neighbor's device/port IDs seen by UDLD on that port.  
  • Neighboring ports should see their own device/port ID (echo) in the packets received from the other side. 
  • If the port does not see its own device/port ID in the incoming UDLD packets for a specific duration of time, the link is considered unidirectional. 
  • UDLD can operate in two modes: normal and aggressive. 
  • In normal mode, if the link state of the port was determined to be bi-directional and the UDLD information times out, no action is taken by UDLD. The port state for UDLD is marked as undetermined. The port behaves according to its STP state. 
  • In aggressive mode, if the link state of the port is determined to be bi-directional and the UDLD information times out while the link on the port is still up, UDLD tries to re-establish the state of the port by sending a UDLD frame 8 times every 1 second. If not successful, the port is put into the errdisable state. 
  • UDLD should be able to detect an unidirectional link before STP would unblock the port due to missed BPDUs. Thus, when you configure UDLD timers, make sure your values are set so that unidirectional link is detected before “STP MaxAge + 2xForwardDelay” expires.
  • In order for a link to become “Bidirectional”, UDLD process should first hear an echo packet with it’s own ID from a peer on the other side.
  • This prevents link from becoming errdisabled when you configure aggresive mode just on one side. The UDLD state of such link will be “Unknown”.

Example:

Switch# show udld f0/13

Interface Fa0/13
---
Port enable administrative configuration setting: Enabled
Port enable operational state: Enabled
Current bidirectional state: Unknown
Current operational state: Advertisement
Message interval: 7
Time out interval: 5
No neighbor cache information stored


  • The problem of unidirectional links usually occurs on fiber-optic links and is not common on copper connections.
  • To enable UDLD on all fiber-optic interfaces, use the global command udld enable (normal mode) or udld aggressive (aggressive mode).
  • To enable UDLD individually on specific ports (copper or fiber), use the udld port [aggressive] command.

Example:

Switch(config)# interface f0/13
Switch(config-if)# udld port aggressive

 

Switch# show udld f0/13

Interface Fa0/13
---
Port enable administrative configuration setting: Enabled / in aggressive mode
Port enable operational state: Enabled / in aggressive mode
Current bidirectional state: Bidirectional
Current operational state: Advertisement - Single neighbor detected
Message interval: 7
Time out interval: 5



  • To restore interfaces from the error-disabled state, you can use either the udld reset command or configure errdisable autorecovery as described in a previous post.

References
CDP on Cisco Routers and Switches Running IOS
Link Layer Discovery Protocol (LLDP, IEEE 802.1AB)
Understanding and Configuring UDLD 
UDLD Modes of Operation
UDLD

Wednesday, January 6, 2016

CCIE 400-101: Layer 2 Technologies - Managing MAC address table, errdisable recovery, L2 MTU

Managing MAC address table

  • The MAC address table contains address information that the switch uses to forward traffic between ports. 
  • Two types of addresses: 
    • Dynamic: a source MAC address that the switch learns from incoming frames and then ages out when it is not in use. 
    • Static: a manually entered address that does not age and is not lost when the switch reloads. 
  • The MAC address table also lists the associated VLAN ID and the port number with the MAC address. 
  • By default, MAC address learning is enabled on all interfaces and VLANs on the switch. 
  • MAC address learning can be disabled (except for internal VLANs) but it is not recommended. 
  • Cisco guides that you should disable MAC address learning only in VLANs with two ports. If you disable MAC address learning on a VLAN with more than two ports, every packet entering the switch is flooded in that VLAN domain. 
  • MAC address learning can be disabled for a single VLAN or a range of VLANs with one command. 
  • The same principle applies to interfaces. 
  • Example: no mac address-table learning vlan 223 or no mac address-table learning vlan 1-10,15.
  • The show mac address-table command with different parameters displays information about the MAC address table.

errdisable recovery

  • Errdisable is a feature that automatically disables a port if the software detects an error situation on the port.
  • This means that the configuration shows the port to be enabled but IOS has effectively shut it down and no traffic is sent or received on the port.
  • To determine if a port has been error disabled, use the show interfaces command.

cat6knative#show interfaces gigabitethernet 4/1 status

Port    Name       Status       Vlan       Duplex  Speed Type
Gi4/1              err-disabled 100          full   1000 1000BaseSX


  • There are various reasons for the interface to go into errdisable. The reason can be:
    • Duplex mismatch
    • Port channel misconfiguration
    • BPDU guard violation
    • UniDirectional Link Detection (UDLD) condition
    • Late-collision detection
    • Link-flap detection
    • Security violation
    • Port Aggregation Protocol (PAgP) flap
    • Layer 2 Tunneling Protocol (L2TP) guard
    • DHCP snooping rate-limit
    • Incorrect GBIC / Small Form-Factor Pluggable (SFP) module or cable
    • Address Resolution Protocol (ARP) inspection
    • Inline power
  • To disable error-disable detection, use the no errdisable detect cause command.
  • The show errdisable detect command displays the error-disable detection status.
  • When the switch puts a port in the error-disabled state, the switch sends a message to the console that describes why it disabled the port.

Example:

%SPANTREE-SP-2-BLOCK_BPDUGUARD:
   Received BPDU on port GigabitEthernet4/1 with BPDU Guard enabled. Disabling port.


  • To recover a port in the errdisabled state, manual intervention is required. The administrator must issue shutdown and no shutdown on the port.
  • If enabled, errdisable recovery can automatically re-enable a port after a timeout period.
  • The errdisable recovery feature is disabled by default.
  • The default timeout interval is 300 seconds and, by default, the timeout feature is disabled. If any one of the errdisable recovery conditions is enabled, the ports with this condition are reenabled after 300 seconds.
  • To simultaneously enable errdisable autorecovery for all supported features, use errdisable recovery cause all. The timeout period can be adjusted with the errdisable recovery interval <sec> command.

cat6knative#show errdisable recovery
ErrDisable Reason    Timer Status
-----------------    --------------
udld                 Enabled
bpduguard            Enabled
security-violatio    Enabled
channel-misconfig    Enabled
pagp-flap            Enabled
dtp-flap             Enabled
link-flap            Enabled
l2ptguard            Enabled
psecure-violation    Enabled
gbic-invalid         Enabled
dhcp-rate-limit      Enabled
mac-limit            Enabled
unicast-flood        Enabled
arp-inspection       Enabled

Timer interval: 300 seconds

Interfaces that will be enabled at the next timeout:

Interface      Errdisable reason      Time left(sec)
---------    ---------------------    --------------
  Fa2/4                bpduguard          273


  • If any ports have been errdisabled, they will be shown at the end of show errdisable recovery output with the timeout left.
  • To fully recover a port from the errdisabled state is to find and correct the root cause.
  • If you reenable the port before you fix the root problem, the port just becomes error disabled again.

L2 MTU

  • MTU is the maximum size of the packet that can still be sent on a data link, without fragmentation. 
  • Encapsulations and type of transmission medium affect the MTU. 
  • The MTU of a higher layer must fit within the payload of the lower layer. 
  • Classic IOS and IOS XR configure the MTU differently. 
  • For instance, on IOS XR the mtu command, which sets the L2 MTU, includes the 14-byte L2 header (6-byte src MAC, 6-byte dst MAC, and 2-byte EtherType) for an Ethernet interface (without dot1q), or 4 bytes for Point-to-Point Protocol (PPP) or high-level data link control (HDLC). 
  • If the interface MTU of a non-dot1q Ethernet interface is the default of 1514 bytes, then higher layer protocols such MPLS can have a maximum MTU of 1500 bytes on that interface. This means you can fit only a 1500 byte MPLS frame (including labels) inside the Ethernet frame. You cannot configure a 1508 byte MPLS MTU on that interface if you want to allow two MPLS tags on top of a 1500 byte IPv4 packet. In order to transmit a 1508 byte MPLS frame on an Ethernet interface, the interface MTU must be increased to 1522, or higher value, in order to ensure that the L2 interface payload is large enough to carry the MPLS frame. 
  • See: MTU Behavior on Cisco IOS XR and Cisco IOS Routers.

References

Errdisable Port State Recovery on the Cisco IOS Platforms.

Tuesday, January 5, 2016

CCIE 400-101: Network Principles - debug, conditional debug, ping, traceroute with extended options

debug

  • debug provides invaluable, real-time information for troubleshooting a problem.
  • Use debug commands with caution.
  • debug commands can be very noisy and spitting excessive output on the console can easily crash an IOS device. This is because IOS automatically prioritizes console output ahead of other router functions.
  • Before you start a debug command, always consider the output that this command will generate and the amount of time this can take.
  • Before debugging, look at your CPU load with the show processes cpu command. Verify that you have ample CPU available before you begin the debugs.
  • Routers can display debug outputs to various interfaces, including the console, aux, and vty ports.
  • Routers can also log messages to an internal buffer to an external syslog server.
  • By default, logging is enabled on the console port and the console port always processes debug output even if another method (such as vty or buffer) is used to capture the output.
  • Hence, Cisco recommends to have the no logging console enabled at all times.
  • To see debug output over a Telnet connection, use the terminal monitor command.
  • To log messages to an internal buffer instead of the console (default), use the logging buffered command.
  • The logging buffered command copies log messages to an internal buffer instead of writing them to the console.
  • The buffer is circular in nature, so newer messages overwrite older messages.
  • The show logging command displays the messages logged in the buffer from oldest to newest.
  • The buffer size and severity level can also be manually defined (logging buffered <size> and logging buffered <severity-level>, respectively).
  • Enable msec timestamps for accurate timing of debug outputs.
  • Stop debugging using no debug all or undebug all (u all).

 

Conditional debug

  • Conditional debug is a powerful tool to filter out unnecessary debug output. 
  • It allows IOS to display information that matches only specified parameters. 
  • For example, an access-list can be referenced in the debug ip packet command to narrow down the lengthy output traffic between specific hosts. 
  • See also: IOS Conditional Debugging.


ping, traceroute with extended options

  • The ping command is a very common and powerful tool for troubleshooting IP reachability.
  • Ping uses a series of ICMP messages to verify connectivity.
  • The ping is successful only if:
    • the echo request reaches the destination, and
    • the destination is able to send an echo reply back to the source before the timeout expires (two seconds by default).
  • Ping uses different codes to signify specific conditions:
    • ! = receipt of a reply
    • . = timeout while waiting for a reply
    • U = a destination unreachable error was received
    • Q = source quench (destination too busy)
    • M = could not fragment
    • ? = unknown packet type
    • & = packet lifetime exceeded

Example:

Router1#ping 12.0.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 12.0.0.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/6/8 ms

  • The extended ping is used to perform a more advanced check of host reachability and network connectivity.
  • There is a multitude of options that can be set with the extended ping.
  • Example: When a normal ping command is sent from a router, the source address of the ping is the IP address of the interface that the packet uses to exit the router. If an extended ping command is used, the source IP address can be changed to any IP address on the router.

Router A#ping
Protocol [ip]:
Target IP address: 192.168.40.1
Repeat count [5]:
Datagram size [100]:
Timeout in seconds [2]:
Extended commands [n]: y
Source address or interface: 172.16.23.2
Type of service [0]:
Set DF bit in IP header? [no]:
Validate reply data? [no]:
Data pattern [0xABCD]:
Loose, Strict, Record, Timestamp, Verbose[none]:
Sweep range of sizes [n]:
Type escape sequence to abort.

Sending 5, 100-byte ICMP Echos to 162.108.21.8, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 36/97/132 ms 



  • The traceroute command is used to discover the routes that packets actually take when traveling to their destination.
  • Three datagrams are sent, each with a TTL value of 1.
  • The TTL value of 1 causes the datagram to timeout at the first router, which then responds with an ICMP time exceeded message.
  • The next set of datagrams is sent with an initial TTL+1, causing the second router to return the ICMP time exceeded message.
  • This process continues until the packets actually reach the destination.
  • The traceroute also has a list of possible output codes:
    • xx msec = RTT for each node for the specified number of probes
    • * = the probe timed out
    • A = administratively prohibited (for example, an access-list)
    • Q = source quench (destination too busy)
    • I = user interrupted test
    • U = port unreachable
    • H = host unreachable
    • N = network unreachable
    • P = protocol unreachable
    • T = timeout
    • ? = unknown packet type

Example:

Router1#traceroute 34.0.0.4

Type escape sequence to abort.
Tracing the route to 34.0.0.4

  1 12.0.0.2 4 msec 4 msec 4 msec
  2 23.0.0.3 20 msec 16 msec 16 msec
  3 34.0.0.4 16 msec *  16 msec 

  • The extended traceroute is a variation of the regular traceroute command.
  • Much like the extended ping, the extended traceroute is more granular and allows the administrator to set various parameters.

Example:
Router A>enable
Router A#traceroute
Protocol [ip]:
Target IP address: 192.168.40.2   
Source address: 172.16.23.2
Numeric display [n]:
Timeout in seconds [3]:
Probe count [3]:
Minimum Time to Live [1]:
Maximum Time to Live [30]:
Port Number [33434]:
Loose, Strict, Record, Timestamp, Verbose[none]:
Type escape sequence to abort.
Tracing the route to 192.168.40.2

  1 172.31.20.2 16 msec 16 msec 16 msec
  2 172.20.10.2 28 msec 28 msec 32 msec
  3 192.168.40.2 32 msec 28 msec *   


See: Using the Extended ping and Extended traceroute Commands.

Friday, January 1, 2016

CCIE 400-101: Network Principles - UDP Operations

Starvation

  • When TCP flows are combined with UDP flows within a single traffic class and the class experiences congestion, TCP flows continually lower their transmission rates (global synchronization), potentially giving up their bandwidth to UDP flows that are oblivious to drops.
  • This effect is called TCP starvation/UDP dominance.
  • The best way to resolve this is to classify UDP and TCP streams separately as much as possible.

Latency

  • Latency is the end to end delay.  
  • Jitter is the variance in latency. 
  • UDP is connectionless, the real effect of the latency on the UDP stream is that there would be a great delay between the sender and the receiver.  
  • Jitter causes problems with UDP streams if packets arrive in the wrong order.  
  • Jiffer can be smoothed by buffering.


RTP/RTCP concepts

  • UDP is the prefferred protocol for real-time data transport like voice and video. 
  • However, UDP does not have any data flow management mechanisms like datagram sequencing or other "missing pieces". 
  • These functions are handled by RTP instead. 
  • For example, to help guarantee that datagrams are played back in the right order, RTP has a 16-bit sequence number field in its packet header, which allows the receiving device to ensure that the packets are correctly sequenced. 
  • Simply put, RTP carries the actual data, such as voice or video. RTCP carries information about that data – information such as jitter and delay statistics, or information about QoS mechanisms involved with these conversations.

References

UDP Operations

Wednesday, December 30, 2015

CCIE 400-101: Network Principles - Latency, Windowing, BDP, Global Synchronization

Latency

  • Latency is most often used to describe the delay between the time that data is requested and the time when it arrives (also known as round-trip time or RTT for short).
  • However, there are essentially four components that contribute to total latency:
    1. Transmission delay: the time to transmit data on the link.
    2. Propagation delay: the time for a packet to travel accross the link.
    3. Processing delay:  the time for a device to process the data.
    4. Queuing delay: the time the data spent waiting in queue (memory) inside a device.
  • The combination of transmission, propagation, processing, and queuing delays often produces a complex and variable network latency profile.


  • Different delays can be calculated as follows:
    • Transmission delay (t) = number of bits to send (b) / link data rate (r). (t = b/r)
    • Propagation delay (p) = link distance (d) / speed of signal (s). (p = d/s)
      • Unless otherwise stated, s = c = 3 x 10^8 (the speed of light)
    • Processing delay and queuing delay are variable and depend on many factors.

Example: The latency is indicated in the 'time' column of the ping output. The ping statistics also list the minimum, maximum, and average latencies.

Pinging www.google.com [74.125.224.82] with 1500 bytes of data:
Reply from 74.125.224.82: bytes=1500 time=68ms TTL=52
Reply from 74.125.224.82: bytes=1500 time=68ms TTL=52
Reply from 74.125.224.82: bytes=1500 time=65ms TTL=52
Reply from 74.125.224.82: bytes=1500 time=66ms TTL=52
Reply from 74.125.224.82: bytes=1500 time=70ms TTL=52

Ping statistics for 74.125.224.82:
    Packets: Sent = 5, Received = 5, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 65ms, Maximum = 70ms, Average = 67ms


  • A networking technology with very high throughput and bad (high) latency can be worse for some applications than one with relatively low throughput but good (low) latency. 
  • See: Ping vs. Latency: What's the Difference?

Windowing

  • Essentially, the TCP window size is the amount of ("in flight") data that a sender can send before it needs to wait for an acknowledgement from the recipient. 
  • If the sender has not received acknowledgement, it will stop and wait. If this wait time exceeds a certain limit, a retransmission will occur. This is how TCP achieves reliable data transmission. 
  • Even if there is no packet loss in the network, windowing can limit throughput. 
  • Because TCP transmits data up to the window size before waiting for the acknowledgement, the full bandwidth of the network may not always get used. 
  • The window size is communicated via a 16-bit field in the TCP header, giving it a maximum size of 64 KB. 
  • Window scale provides a scale factor which is required for TCP to support window sizes that are larger than 64 KB. 
  • Essentially, window scaling simply extends the 16-bit window field to 32 bits. 
  • This is done by specifying a 1-byte shift count in the header options field. 
  • The true receive window size is left shifted by the value in the shift count. 
  • The maximum value of the shift count is 14, resulting in the maximum window size of 1 GB!


  • The window scaling option may be sent only once during a connection by each host, in its SYN packet.
  • The window size can be dynamically adjusted by modifying the value of the window field in the TCP header, but the scale factor remains static for the duration of the TCP connection.
  • Scaling is only in effect if both ends include the option; if only one end of the connection supports window scaling, it will not be enabled in either direction.

 

Bandwidth Delay Product (BDP)

  • In short, bandwidth delay product (BDP) is the optimum window size to fully utilize link bandwidth. 
  • The best possible network performance is achieved when the link between the sender and the receiver forwards a constant stream of data. 
  • Analogy: a water hose. To achieve maximum water flow, the hose should be full. As the hose increases in diameter and length, the volume of water to keep it full also increases. 
  • In networks, diameter equates to bandwidth, length is measured as round-trip time, and the TCP window size is analogous to the volume of water necessary to keep the hose full. 
  • On fast networks with large round-trip times, the TCP window size must be increased to achieve maximum TCP bandwidth. 
  • Theoretically, the TCP window size should be set to the BDP, which is simply computed as bandwidth * delay. 
  • Example: For 1 Gbps link with 70 ms delay, the BDP (optimum TCP window size) is 1,000,000,000 * 0.07 = 70,000,000 (bits). Convert that to bytes, you get 70,000,000 / 8 = 8,750,000 bytes = 8,75 MB.  
  • In order to fully utilize 1 Gbps link with 70 ms, the TCP window size must be 8,75 MB.  
  • Based on these calculations, it is easy to see why the typical default buffer size of 64 KBytes would be completely inadequate for this connection. With 64 KBytes you would get only 0.1% of the available bandwidth.

Global Synchronization

  • During times of congestion, routers can use a technique called tail drop to simply discard packets when the interface queue is filled to its maximum.
  • When tail drop happens, TCP has an automatic recovery from the dropped packets, which causes the sender to reduce its sending rate.
  • Bandwidth utilization drops significantly and interface queue becomes less congested.
  • After a certain amount of time, the sender starts increasing the rate again.
  • The problem is that almost all the senders will use the same time delay. When these delays expire, at the same time, all senders will send additional packets, the interface queue will overflow again, more packets will be dropped, the senders will all back off for a fixed delay.. and the cycle repeats.
  • This pattern of each sender decreasing and increasing transmission rates at the same time as other senders is referred to as "global synchronization".
  • The result is a sawtooth-shaped bandwidth utilization, causing inefficient use of available bandwidth and inconsistent application throughput.


References

Transmission and Propagation Delay
Congestion Avoidance
Tuning TCP for High Bandwidth-Delay Networks
A User's Guide to TCP Windows
TCP Windows and Window Scaling

Wednesday, December 23, 2015

CCIE 400-101: Network Principles - IPv4 and IPv6 PMTU, MSS

MSS

  • The TCP Maximum Segment Size (MSS) defines the maximum amount of data that a host is willing to accept in a single TCP/IP datagram.
  • The MSS value is sent as a TCP header option only in TCP SYN segments.
  • Each side of a TCP connection reports its MSS value to the other side.
  • Contrary to popular belief, the MSS value is not negotiated between hosts. The sending host is required to limit the size of data in a single TCP segment to a value less than or equal to the MSS reported by the receiving host.
  • The way MSS now works is that each host will first compare its outgoing interface MTU with its own buffer and choose the lowest value as the MSS to send.
  • The hosts will then compare the MSS size received against their own interface MTU and again choose the lower of the two values.
  • Often the send MSS value will be the same on each end of a TCP connection.
  • By default, the MSS = 1460 bytes in an Ethernet network (1500 bytes - 20-byte IP header - 20-byte TCP header = 1460 bytes).
  • TCP MSS takes care of fragmentation at the two endpoints of a TCP connection, but it doesn't handle the case where there is a smaller MTU link in the middle between these two endpoints.

PMTU

  • Path MTU (PMTU) is the smallest MTU in the path from source to destination.
  • Path MTU Discovery (PMTUD) was developed to avoid fragmentation in the path between the endpoints. It is used to dynamically determine the lowest MTU along the path from a packet's source to its destination.  
  • PMTUD is only supported by TCP. UDP and other protocols do not support it. 
  • If PMTUD is enabled on a host, and it almost always is, all TCP/IP packets from the host will have the DF bit set. 
  • If a router tries to forward an IP datagram, with the DF bit set, onto a link that has a lower MTU than the size of the packet, the router will drop the packet and return an Internet Control Message Protocol (ICMP) "Destination Unreachable" message to the source of this IP datagram, with the code indicating "fragmentation needed and DF set" (type 3, code 4).
  • A router returning an ICMP message indicating "fragmentation needed and DF set" should include the MTU of that next-hop network in the low-order 16 bits of the ICMP additional header field that is labeled "unused" in the ICMP specification. The high-order 16 bits remain unused, and MUST be set to zero.  Thus, the message has the following format:

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |   Type = 3    |   Code = 4    |           Checksum            |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |           unused = 0          |         Next-Hop MTU          |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |      Internet Header + 64 bits of Original Datagram Data      |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+



  • When the source station receives the ICMP message, it will lower the packet size to reflect the smaller MTU. 
  • There are three things that can break PMTUD, two of which are uncommon and one of which is common.
    • A router can drop a packet and not send an ICMP message. (Uncommon)
    • A router can generate and send an ICMP message, but the sender ignores the message. (Uncommon)
    • A router can generate and send an ICMP message but the ICMP message gets blocked by a router or firewall between this router and the sender. (Common) 
  • The success or failure of PMTUD hinges upon ICMP unreachable messages getting through to the sender of a TCP/IP packet. 
  • The following commands will enable the necessary ICMP messages to pass through a firewall:

access-list 101 permit icmp any any unreachable
access-list 101 permit icmp any any time-exceeded
access-list 101 deny icmp any any
access-list 101 permit ip any any


 

References

Resolve IP Fragmentation, MTU, MSS, and PMTUD Issues with GRE and IPSEC