A NETWORK SERVER APPLICATION THAT can handle only one client at a time isn’t very useful. For example, consider an IRC chat application wherein only one client could connect to an IRC chat server at a time. How much fun would it be to chat with yourself? A server is typically required to handle multiple clients simultaneously.
Handling multiple clients at the same time requires solving several problems. The first issue is allowing multiple clients to connect and stay connected simultaneously. In this chapter, we cover three different general strategies for handling this: multiplexing, forking, and threads. The second issue is one of resources and how to efficiently utilize the memory and processor(s) available. The final issue is keeping the server responsive to each of the clients—in other words, not allowing a client to monopolize the server at the expense of the other connected clients. This is especially important when large amounts of data are to be transferred between the client and server.
This chapter will explain the various strategies available to handle multiple clients. In addition, we’ll build servers of each type. We’ll start off with a client test program.
Client Test ProgramA server isn’t much good without a client program to connect to it. In this chapter we’ll look at and implement several types of servers. To see how they work we’ll use a client test program. This will help us see how each server type handles multiple clients.
To test a server you’ll need to open two xterm windows. In the first window, execute the server that you wish to test. In the second window, execute the client test program. You should see output in both the server and client windows.
Here’s our test client program, client.c. We’ll use it to test the various server examples throughout this chapter. First, we include the needed system header files:
/* client.c * /
#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <string.h>
We’ll use the fork() system call to generate a number of child processes to simulate multiple clients connecting to the server at the same time. This is the forward declaration of the process function:
void child_func(int childnum);
This is our main() function. We check the command line to see how many child processes to create.
int main(int argc, char *argv[])
{
int nchildren = 1;
int pid;
int x;
if (argc > 1) {
nchildren = atoi(argv[1]);
}
Next, we loop and create the specified number of children. We will look at this later, but if fork() returns 0, then it has returned in the child process, so we call our child function.
for (x = 0; x < nchildren; x++) {
if ((pid = fork()) == 0) {
child_func(x + 1);
exit(0);
}
}
Once we’ve created all of the children, the parent process waits for them to finish before returning.
wait(NULL);
return 0;
}
Next, we create our child function. This is where we connect to the server.
void child_func(int childnum)
{
int sock;
struct sockaddr_in sAddr;
char buffer[25];
We create our client socket and bind it to a local port.
memset((void *) &sAddr, 0, sizeof(struct sockaddr_in));
sAddr.sin_family = AF_INET;
sAddr.sin_addr.s_addr = INADDR_ANY;
sAddr.sin_port = 0;
sock = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
bind(sock, (const struct sockaddr *) &sAddr, sizeof(sAddr));
Then we attempt to connect to whichever server is running on the local machine.
sAddr.sin_addr.s_addr = inet_addr(“127.0.0.1”);
sAddr.sin_port = htons(1972);
if (connect(sock, (const struct sockaddr *) &sAddr, sizeof(sAddr)) != 0) {
perror(“client”);
return;
}
Once connected, we send some characters to the server and read what the server sends back. We also insert some pauses, using sleep() to keep the clients from connecting and disconnecting so quickly that we don’t have more than one connected to a server at the same time.
snprintf(buffer, 128, “data from client #%i.”, childnum);
sleep(1);
printf(“child #%i sent %i charsn”, childnum, send(sock, buffer,
strlen(buffer), 0));
sleep(1);
printf(“child #%i received %i charsn”, childnum,
recv(sock, buffer, 25, 0));
Finally, we close the connection and return.
sleep(1);
close(sock);
}
The test client can be compiled with the following command:
$>gcc -o client client.cThis runs the client with five child processes, each connecting to the server.
$>./client 5
{mospagebreak title=Multiplexing}
The first strategy for handling multiple connections that we’ll discuss is multiplexing. Multiplexing is a way of handling multiple clients in a single server process. The application allows clients to connect to the server and adds them to a watch list. This watch list is just an array of socket descriptors. Then the operating system tells the application which clients (if any) need to be serviced or if a new client has established a connection.
As an example, think of a restaurant with only one waiter. The waiter is responsible for attending to all the tables at the same time. As customers come in and are seated, the waiter adds them to a mental list of tables to check on. Then, when a table needs attention, he attends to it. Of course, only one table may be serviced at a time, and the possibility exists of a single table using up all the waiter’s time.
The select() Functionselect() is a system function that allows us to specify a set of descriptors (sockets, in this case) that we are interested in. It is worth noting that select() works with any descriptor, including files, pipes, FIFOs, etc. The system puts our program to sleep, polls the sockets for activity, and wakes the program when an event occurs at one of the sockets. This keeps us from writing a busy loop and wasting clock cycles. The select() function prototype looks like this:
#include <sys/select.h>
int select(int n, fd_set *readfds, fd_set *writefds,
fd_set *exceptfds, struct timeval *timeout);
The first parameter specifies the highest numbered descriptor (plus 1) to watch in the three sets. It is important to remember that you must add 1 to the highest numbered descriptor in the sets. The reason is that the watch lists are linear arrays of bit values, with 1 bit for every available descriptor in the system. What we are really passing to the function is the number of descriptors in the array that it needs to copy. Since descriptors start at 0, the number we pass is the largest descriptor number plus 1.
Next, we provide three descriptor sets. The first set contains descriptors to be watched for read events, the second for write events, and the third for exceptions or error events. Finally, we provide a timeval that specifies a timeout. If no event occurs in any of the sets before the timeout, then select() returns a 0. We can also specify a null pointer for the timeout parameter. In this case, the call will not return until an event happens on one of the watched descriptors. Otherwise, it returns the number of descriptors in the three sets.
It is important to note that select() does modify the descriptor sets that are passed to it. Upon return, the sets will contain only those descriptors that had some activity. To call select multiple times, we must retain a copy of the original sets. Other than a socket error, if any error occurs, then –1 is returned.
Four macros are provided to help deal with the descriptor sets. They are FD_CLR , FD_ISSET , FD_SET , and FD_ZERO . Each takes a pointer to a variable type fd_set . Except for FD_ZERO , each takes a descriptor as well. It is important to note that the behavior of these macros is undefined if you pass in a descriptor that is less than zero or greater than FD_SETSIZE. The macros are prototyped as follows:
- void FD_SET(int fd, fd_set *set) : FD_SET flags a descriptor to be watched.
- void FD_CLR(int fd, fd_set *set) : FD_CLR resets the flag set to a descriptor.
- int FD_ISSET(int fd, fd_set *set) : After select() returns, FD_ISSET determines whether a descriptor is flagged or not.
- void FD_ZERO(fd_set *set) : FD_ZERO clears the set so that no descriptors are watched.
A flagged descriptor indicates activity at the socket.
Here is a code fragment example of using select() :
int sd; /* our socket descriptor * /
fd_set sockreadset;
FD_ZERO(&sockreadset);
FD_SET(sd, &sockreadset);
select(FD_SETSIZE, sockreadset, NULL,
NULL, NULL);
if (FD_ISSET(sockreadset))
printf(“Socket ready for read.n”);
In this example, the program will wait indefinitely for a read event to occur on the descriptor whose value is specified in sd .
A Multiplexing ServerIn our example, the server uses select() to listen for new connections, check for client disconnects, and read events on existing connections. If a read event occurs on the server’s listening socket, then a new connection is initiated and the server calls accept() to get the new socket descriptor. The new descriptor is then added to the server’s watch set.
On the other hand, if a read event occurs on another socket, then the server calls recv to retrieve any data sent by the client. If no data is received, then the client has disconnected, and the server removes the respective descriptor from the watch set. Otherwise, the data is read and echoed back to the client. Figure 5-1 shows the basic architecture of a multiplexing server.
Figure 5-1. Basic architecture of a multiplexing server
Here is the program ( server1.c ) to implement the preceding example:
/* server1.c */
#include <stdio.h>
#include <sys/ioctl.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
int main(int argc, char *argv[])
Next, we set up the variables that we’ll need. As select() modifies the set passed to it, we use two variables: one to maintain our state and another to interact with the select() function. We need to keep the master set separately:
{
struct sockaddr_in sAddr;
fd_set readset, testset;
int listensock;
int newsock;
char buffer[25];
int result;
int nread;
int x;
int val;
Then we create the listening socket. This is the socket that will listen for incoming connections from the clients.
listensock = socket(AF_INET, SOCK_STREAM,
IPPROTO_TCP);
Afterward, we set the socket option SO_REUSEADDR . While debugging, you’ll be starting and stopping your server often. Linux tends to keep the address and port that was used by your program reserved. This option allows you to avoid the dreaded “address in use” error.
val = 1;
result = setsockopt(listensock,
SOL_SOCKET, SO_REUSEADDR, &val,
sizeof(val));
if (result < 0) {
perror(“server1”);
return 0;
}
Here, we bind the socket to the listening port. We use the special address INADDR_ANY to specify that we’ll listen on all IP addresses associated with the server:
sAddr.sin_family = AF_INET;
sAddr.sin_port = htons(1972);
sAddr.sin_addr.s_addr = INADDR_ANY;
result = bind(listensock, (struct sockaddr
*) &sAddr, sizeof(sAddr));
if (result < 0) {
perror(“server1”);
return 0;
}
We put the socket into “listen” mode so that we can accept incoming connections:
result = listen(listensock, 5);
if (result < 0) {
perror(“server1”);
return 0;
}
We initialize our descriptor set using FD_ZERO . Then we add the listening socket to the set so that the system will notify us when a client wishes to con nect. Connection requests are treated as read events on the listening socket:
FD_ZERO(&readset) ;
FD_SET(listensock, &readset);
Notice that we assign our descriptor set to an alternate variable to be passed to the select() function. As noted previously, this is because select() will alter the set we pass, so that upon return, only those sockets with activity are flagged in the set. Our call to select() signifies that we are interested only in read events. In a real-world application, we would need to be concerned with errors and pos sibly write events. We loop through the entire set of descriptors. FD_SETSIZE is a constant set in the kernel and is usually 1024. A more efficient server implementation would keep track of the highest numbered descriptor and not loop through the entire set. FD_ISSET is used to determine if the descriptor is flagged as having activity. It returns a nonzero value if the supplied descriptor is set as having had activity; otherwise, it returns 0.
while (1) {
testset = readset;
result = select(FD_SETSIZE, &testset,
NULL, NULL, NULL);
if (result < 1) {
perror(“server1”);
return 0;
}
for (x = 0; x < FD_SETSIZE; x++) {
if (FD_ISSET(x, &testset)) {
If the activity is on the listening socket, then we accept the new connection and add its socket to our watch set. Otherwise, we read characters from the client. If the number of characters read is less than or equal to zero, then the client is assumed to have closed the connection. We close the connection on our side and remove the descriptor from our watch list. Otherwise, we echo the charac ters to the screen and back to the client.
if (x == listensock) {
newsock = accept(listensock, NULL,NULL);
FD_SET(newsock, &readset);
} else {
nread = recv(x, buffer, 25, 0);
if (nread <= 0) {
close(x);
FD_CLR(x, &readset);
printf(“client on descriptor #%i disconnectedn”, x);
} else {
buffer[nread] = ‘