This article discusses the various types of networks, the methods for connnecting networks, how network data is moved from network to network, and the protocols used on today's popular networks. It is excerpted from chapter one of the book The Definitive Guide to Linux Networking Programming, written by Keir Davis et. al. (Apress, 2004; ISBN: 1590593227).
At the Internet layer in our TCP/IP protocol stack, the only information available is the address of the remote node. No other information is available to the protocol, and none is needed. However, without additional information like a port number, your receiving node is limited to conducting a single network communication at any one time. Since modern operating systems allow multiple applications to run simultaneously, you must be able to address multiple applications on the receiving node simultaneously, instead of just one. If you consider that each networked application can “listen” on one or more ports, you can see that by using an IP address and a port, you can communicate with multiple applications simultaneously, up to any limits imposed by the operating system and protocol stack.
In the TCP/IP protocol stack, there are two protocols that provide a mechanism that allows applications to communicate with other applications using ports. One is the Transmission Control Protocol (TCP), which we will discuss in the next section, and the other is the User Datagram Protocol (UDP). UDP makes no guarantee of packet delivery. UDP datagrams can be lost, can arrive out of sequence, can be copied many times, and can be sent faster than the receiving node can process them. Thus, an application that uses UDP takes full responsibility for controlling message loss, reliability, sequencing, and loss of connection. This can be both an advantage and a disadvantage to developers, for while UDP is a lightweight protocol that can be used quickly, the additional application overhead needed to thoroughly manage packet transfer is often overlooked or poorly implemented.
TIP When using UDP for an application, make sure to thoroughly test your applications in real environments beyond a low-latency LAN. Many developers choose UDP and test in a LAN environment, only to find their applications are unusable when used over a larger TCP/IP network with higher latencies.
UDP datagrams have a simple format. Like other datagrams, UDP datagrams consist of a header and a data payload. The header is divided into four fields, each 16 bits in size. These fields specify the source port, the destination port, the length of the datagram, and a checksum. Following the header is the data area, as shown in Figure 1-10.
Figure 1-10.The UDP datagram format
The source port is optional. If used, it specifies the port to which replies should be sent. If unused, it should be set to zero. The length field is the total number of octets in the datagram itself, header and data. The minimum value for length is 8, which is the length of the header by itself.
The checksum value is also optional, and if unused should be set to zero. Even though it’s optional, however, it should be used, since IP doesn’t compute a checksum on the data portion of its own datagram. Thus, without a UDP checksum, there’s no other way to check for header integrity on the receiving node. To compute the checksum, UDP uses a pseudo-header, which is prepended to the datagram, followed by an octet of zeros, which is appended, to get an exact multiple of 16 bits. The entire object, pseudo-header and all, is then used to compute the checksum. The pseudo-header format is shown in Figure 1-11.
Figure 1-11.The UDP pseudo-header
The octet used for padding is not transmitted with the UDP datagram, nor is the pseudo-header, and neither is counted when computing the length of the datagram.
A number of network services use UDP and have reserved ports. A list of some of the more popular services is shown in Table 1-6.
Table 1-6. Popular UDP Services
Transmission Control Protocol
The main difference between UDP and TCP is that, like IP, UDP provides no guarantee of delivery and does not use any method to ensure datagrams are received in a certain order or are transmitted at a certain rate. TCP, on the other hand, provides a mechanism known as reliable stream delivery. Reliable stream delivery guarantees delivery of a stream of information from one network node to another without duplication or data loss. TCP has a number of features that describe the interface between it and the application programs that use it:
Virtual circuit: Using TCP is much like making a phone call. The sender requests a connection with the receiver. Both ends negotiate the parameters of the connection and agree on various details defining the connection. Once the connection is finalized, the applications are allowed to proceed. As far as the applications are concerned, a dedicated, reliable connection exists between the sender and the receiver, but this is an illusion. The underlying transfer mechanism is IP, which provides no delivery guarantee, but the applications are removed from dealing with IP by the TCP layer.
Buffered transfer: The TCP layer, independent of the application, determines the optimal way to package the data being sent, using whatever packet sizes are appropriate. To increase efficiency and decrease network traffic, TCP typically waits, if possible, until it has a relatively large amount of data to send before sending the packet, even if the application is generating data 1 byte at a time. The receiving TCP layer delivers data to the receiving application exactly the way it was sent, so a buffer may exist at each end, independent of the application.
Stream orientation: The receiving node delivers data to the receiving application in exactly the same sequence as it was sent.
Full duplex: Connections provided by TCP over IP are full duplex, which means that data can be transmitted in both directions simultaneously via two independent packet streams. The streams can be used to transfer data or to send control information or commands back to the sender, and either stream can be terminated without harming the other.
Unstructured stream: TCP does not guarantee the structure of the data stream, even though delivery is guaranteed. For example, TCP does not honor markers that might exist in a record set sent to or from a database. It is up to the applications to determine stream content and assemble or disassemble the stream accordingly on each end of the connection. Applications do this by buffering incoming packets when necessary and assembling them in an order that the applications recognize.
The method that TCP uses to guarantee reliable delivery can be described as confirmation and retransmission. The sender keeps track of every packet that is sent and waits for confirmation of successful delivery from the receiver before sending the next packet. The sender also sets an internal timer when each packet is sent and automatically resends the packet should the timer expire before getting confirmation from the receiver. TCP uses a sequence number to determine whether every packet has been received. This sequence number is sent on the confirmation message as well as the packet itself, allowing the sender to match confirmations to packets sent, in case network delays cause the transmission of unnecessary duplicates.
Even with full duplex, though, having to wait for a confirmation on every packet before sending the next can be horribly slow. TCP solves this by using a mechanism called a sliding window. The easiest way to imagine a TCP sliding window is to consider a number of packets that needs to be sent. TCP considers a certain number of packets to be a window and transmits all packets in that window without waiting for confirmation on each one. Once the confirmation is received for the first packet in the window, the window “slides” to contain the next packet to be sent, and it is sent. For example, if the window size was 8, then packets 1 through 8 would be sent. When the confirmation for packet 1 was received, the window would “slide” so that it covered packets 2 through 9, and the ninth packet would be sent. A packet that has been transmitted without confirmation is called an unacknowledged packet. Thus, the total number of unacknowledged packets allowed is equal to the window size. The advantage to a sliding window protocol is that it keeps the network saturated with packets as much as possible, minimizing the time spent waiting for confirmation. The window is matched on the receiving end, so that the receiver “slides” its own window according to which confirmation messages have been sent.
We noted earlier that TCP connections use what is known as a virtual circuit. Taking that abstraction further, TCP defines connections as a pair of endpoints, each endpoint consisting of an integer pair, consisting of the 32-bit integer IP address and the 16-bit integer port number. This is important, because by defining an endpoint as an integer pair, a TCP service on a given port number can be used by multiple connections at the same time. For example, even though a mail server has only one port 25 on which to receive mail, each sender making a connection offers a different integer pair because the source IP address and source port are different, allowing multiple concurrent connections on the same receiving port.
The TCP datagram is also known as a segment. Segments do all of the work: establishing and closing connections, advertising window sizes, transferring data, and sending acknowledgments. Figure 1-12 shows a diagram of a TCP segment.
Figure 1-12.The TCP segment format
As with other datagrams, a TCP segment consists of two parts: header and data. A description of each header field is listed in Table 1-7.
Table 1-7. TCP Header Fields
The port number of the sending
The port number of the receiving
The position of this segment’s payload in
the sender’s stream
The octet that the source expects to receive
The length of the header, in 32-bit
Reserved for future use
Defines the purpose and content of this
segment (see Table 1-8)
Specifies the maximum amount of data
that can be accepted
Verifies the integrity of the header
Flag notifying the receiver that the segment
should be handled immediately
Various options used to negotiate between
Needed to accommodate an OPTIONS
value of varying length to ensure header
and data end on a 32-bit boundary
Much like the UDP header, the TCP header has fields such as checksum, source port, destination port, and length. However, the TCP header goes much farther due primarily to the need to guarantee delivery. The is used to indicate the position of the data in a particular segment within the overall byte stream being sent. If a byte stream had 100 packets in it, the sequence number field would be where each segment was numbered, so the receiver would know how to reassemble the stream.
The acknowledgment number, on the other hand, is used by the sender to identify which acknowledgment the sender expects to receive next. Thus, the sequence number refers to the byte stream flowing in the same direction as the segment, while the acknowledgment number refers to the byte stream flowing in the opposite direction as the segment. This two-way synchronization system helps ensure that both connection endpoints are receiving the bytes they expect to receive and that no data is lost in transit.
The code bits field is a special header field used to define the purpose of this particular segment. These bits instruct the endpoint how to interpret other fields in the header. Each bit can be either 0 or 1, and they’re counted from left to right. A value of 111111, then, means that all options are “on.” Table 1-8 contains a list and description of the possible values for the code bits field.
Table 1-8. Possible Values for Code Bits Header Field
NAME (LEFT TO RIGHT)
Urgent pointer field is valid.
Acknowledgment field is valid.
Reset the connection.
Synchronize sequence numbers.
Sender has completed its byte stream.
Even though TCP is a stream-oriented protocol, situations arise when data must be transmitted out of band. Out-of-band data is sent so that the application at the other end of the connection processes it immediately instead of waiting to complete the entire stream. This is where the urgent header field comes into play. Consider a connection a user wishes to abort, such as a file transfer that occurs slower than expected. For the abort signal to be processed, the signal must be sent in a segment out of band. Otherwise, the abort signal would not be processed until the file transfer was complete. By sending the segment marked urgent, the receiver is instructed to handle the segment immediately.