Detailed analysis of FCoE protocol

The first challenge encountered during the development of FCoE is to continue the flow control mechanism implemented by the Buffer-to-buffer CredITs feature of the local Fibre Channel. Although Ethernet switches have no corresponding buffer-to-buffer mechanism, the Ethernet standard can regulate the amount of incoming information by supporting MAC control frames. The IEEE 802.3x flow control standard is based on pause frame flow control technology. This technique will delay the transmission content of the sender before sending it for a certain period of time. If the receiving device clears the buffer before this period of time has passed, it will re-send the pause frame and reset the termination time to zero. This allows the sender to retransmit until another pause frame is received.

Because the FCoE mechanism must support reading and writing of stored data, all terminal devices and Ethernet switches under the network storage path must support bidirectional IEEE 802.3x flow control. Although this effect may not be as ideal as the Buffer-to-buffer CredITs mechanism, IEEE 802.3x pause frames can provide the corresponding functionality to regulate storage traffic and prevent frame loss due to blocking and buffer overflow.

The IEEE 802.3ar congestion management research group and IEEE 802.1au congestion notification research group in IEEE are responsible for research on Ethernet congestion. Especially for storage transactions, this helps to enhance the service level quality of the flow control mechanism, so that the data flow of the most critical tasks can get the highest priority in the case of possible blocking.

Redundant paths and failover

The characteristics of Fibre Channel high availability are mainly due to the Flat or CORE / EDGE topology network that provides redundant paths between the host and the target device. The host bus adapter card, link, switch port, switch, or storage port from the primary path to the secondary path, any one of which will cause a failure of the entire network. In some cases, both paths are dynamic and have both high performance and availability. The fiber shortest path priority protocol in the Fibre Channel architecture is used to determine the best path for transmission between fiber switches, and its judgment is based on the link bandwidth and traffic load of the switch.

The basic Ethernet system must provide FCoE with corresponding resistance to obstacles to ensure unimpeded storage access. When multiple Ethernet switches are connected through intra-switch links (for example, in a complete network topology), IEEE 802.1D rapid spanning tree protocol establishes the main path on the network to avoid the endless loop of frame transmission. The dynamic bridge ports between switches are in the push state, and the non-dynamic failover bridge ports are in the blocked state.

However, because blocked connections cannot be used for data transmission, all blocked connections in the network indicate unused and idle resources. Rapid spanning tree monitors the condition of all bridge ports through the bridge protocol data unit. If the connection, bridge port or exchange fails, the rapid spanning tree protocol starts the necessary failover bridge ports and establishes a selection path on the network.

In addition, IEEE 802.1s Multiple Spanning Tree Protocol (MulTIple Spanning Tree Protocol, MSTP) and IEEE 802.1Q-2003 virtual LAN (VLAN) technology define additional mechanisms for enhancing Ethernet path switching. Similar to Fibre Channel's hard zoning technology, VLAN tagging enables up to 4096 cluster node groups to coexist in a common Ethernet infrastructure.

The enhancement of spanning tree on the multi-service transmission platform can make each VLAN group have a separate spanning tree. Therefore, the bridge port in the blocking mode of one virtual local area network can be adjusted to the forwarding mode of another virtual local area network, and more fully utilize the interconnectivity of all networks.

Even with the enhancement of multi-service transmission equipment, the used network connection still inevitably leads to the fast spanning tree protocol's dependence on forwarding and blocking status. More and more complex layer 3 routing protocols, such as Open Shortest Path First (OSPF), select the best between end nodes based on hop count, bandwidth, delay time, and other measurement standards Path, and achieve load balancing on multiple paths. The Instant Streaming Protocol (RSTP) as a Layer 2 protocol cannot support such additional functionality while maintaining backward compatibility. You need to find ways to introduce load balancing, multi-point access (for example, a node has two dynamic links to the same Ethernet network segment), multicast technology, and broadcast technology into Layer 2 Ethernet.

Fibre Channel to Ethernet mapping

FCoE must also address the differences between the frames transmitted by Ethernet and Fibre Channel. Generally, the maximum Ethernet frame size is 1518 bytes. A typical Fibre Channel frame is approximately 2112 bytes. Therefore, when the optical fiber frame is packed on the Ethernet, it needs to be sent in segments, and then reassembled at the receiver. This will cause more processing overhead and hinder the smoothness of FCoE end-to-end transmission.

Therefore, a larger Ethernet frame is needed to balance the differences in Fibre Channel and Ethernet frame sizes. There is a substantive standard called "Jumbo Frame". Although it is not a formal IEEE standard, it allows Ethernet frames up to 9k bytes in length. When using "Jumbo Frames", it should be noted that all Ethernet switches and terminal devices must support a common "Jumbo Frame" format.

The largest jumbo frame (9K bytes) can encapsulate four Fibre Channel frames under one Ethernet frame. But this will make Fibre Channel connection layer recovery and buffer flow control using 802.3x pause commands more complicated. As shown in Figure 2, FCoE encapsulates a complete fiber frame into a giant Ethernet frame (without using cyclic redundancy check). Because Ethernet already provides a frame check sequence (FCS) to check the integrity of transmitted data, no cyclic redundancy check (CRC) for fiber frames is required. This further reduces the processing overhead required by the transport layer, while improving the performance of the channel. Because fiber frames may include extended, selectable letterhead or virtual fiber marking information, the size of Ethernet "jumbo frames" is not appropriate and will change with the need to encapsulate fiber frames.

FCoE frames are local Layer 2 Ethernet frames that use six-byte MAC hardware destination and source addresses. However, the MAC address is transparent to storage and can only be used for the exchange of frames from the source to the destination. The FCoE frame retains the Fibre Channel addressing required for storage transactions, so a method of mapping from FCID (Fibre Channel ID) to Ethernet MAC address is required. You can choose a protocol similar to the Address Resolution Protocol (ARP) to implement FCID to MAC address mapping.

For example, in the third layer IP environment, the address resolution protocol is used to map from the upper layer IP network address to the second layer hardware MAC address. In addition, Fibre Channel uses some well-known addresses to obtain storage services (for example, through the SNS discovery device mechanism). FCoE requires corresponding functionality to complete the mapping from well-known addresses to corresponding MAC addresses.

In traditional Fibre Channel, the HBA or storage port receives the FCID when connected to an Ethernet switch. FCoE devices cannot ensure that general Ethernet switches provide specialized storage services, so they must rely on domain controllers and storage service engines that can be used inside FCoE switches to provide Fibre Channel login, addressing, and other advanced services. Future data center directors will integrate Ethernet, Fibre Channel and FCoE storage services into one on a high-reliability, multi-protocol platform.


FCoE, iSCSI and FCIP are all storage protocols that can transmit block data over Ethernet. However, each one was developed with different goals and design standards. Because FCoE is developed from a special data center storage protocol, which includes FC and data center Ethernet protocol. iSCSI is designed to reliably transfer storage data on any IP-based system, including LAN and WAN. As shown in Figure 3, iSCSI uses the entire TCP / IP protocol stack at the third layer to implement routing and packet recovery, so iSCSI can be used for potential network bandwidth losses. In contrast, FCIP is designed as a tunneling protocol for remote connection to FC SAN. Like iSCSI, FCIP also bears the processing overhead of TCP / IP, so its design is not suitable for local high-performance data center applications.

The main function of iSCSI lies in its economy. It uses idle drives, Ethernet cards, Ethernet switches and IP routers to transfer SCSI data blocks between servers and storage. Although the cost of server access and network infrastructure is low, the target cost of iSCSI storage will vary depending on whether inexpensive disk drives are used and whether hard disk-based or floppy disk-based controllers are configured. Because there is no dedicated local iSCSI disk drive, the iSCSI target must rely on some form of protocol bridging (from iSCSI to SAS / SATA or from iSCSI to FC) controller to store and retrieve data blocks. So iSCSI is not the same as the JBOD that is sometimes used in departmental FC SANs.

In 1Gb Ethernet, iSCSI can be used to integrate low-performance secondary servers into the existing FC SAN of the data center through the gateway, or provide shared storage for departmental use. But in 10G Ethernet, iSCSI has gradually lost its widely advertised cost advantage. Using 10G Ethernet on the server means that the main program requires high performance and reliability. Although standard NIC cards can be used under 10G, 10G iSCSI servers enhance performance with auxiliary devices such as TCP offloadable adapters, and use iSER logic to avoid multiple storage copies of SCSI data from the interface to the application storage. Designing a sophisticated 10GB iSCSI adapter increases costs, but enables iSCSI to minimize CPU expenditures on the host to more fully utilize larger bandwidth.

to sum up

Due to the huge installation base, the already mature Fibre Channel technology already has many storage features and management tools, which greatly facilitates the configuration of shared storage systems in data centers. Converged Enhanced Ethernet (CEE) technology allows users to combine storage, information transfer, Internet telephony, video, and other data together in a public Ethernet infrastructure data center. FCoE is a component technology to realize high-efficiency block storage of Ethernet. FCoE is not a replacement for Fibre Channel but an extension of Fibre Channel, and will coexist with Fibre Channel SAN.

Because FCoE is a further enhancement to Ethernet, its development requires coordination between Fibre Channel and Ethernet technical experts and standards organizations. Although connection layer issues such as flow control and Ethernet spanning tree protocol are a major challenge, more solutions are needed to continue to retain the advanced Fibre Channel services that users are effectively configuring. Even under the 10G network transmission rate, it is still necessary to conduct in-depth research on today's Ethernet technology to suit the application of data center storage. As a pioneer in Fibre Channel architecture technology, Brocade has introduced expertise to FCoE to reduce the complexity of other solutions while retaining the advantages provided by data center performance, reliability, and advanced storage services.

In-ear Wired Earphones, they are small and comfortable and simple, fit in just about any pocket,and they provide great sound that literally goes straight into your ears,bring it on your next commute or run, or simply enjoy it in the comfort of your time.

In-ear Wired Earphones

Earbuds With Removable Cable,Earphones With Replaceable Cabl,Stereo In-Ear Earphones,In-Ear Wired Headphones

Dongguang Vowsound Electronics Co., Ltd. ,