Asynchronous I/O - BrainDump

Performing asynchronous I/O requires kernel support at the very lowest layers. POSIX 1003.1-2003 defines the aio interfaces, which Linux fortunately implements. The aio library provides a family of functions for submitting asynchronous I/O and receiving notification upon its completion:

  #include <aio.h>

  /* asynchronous I/O control block */
  struct aiocb {
          int aio_filedes;        /* file descriptor */
          int aio_lio_opcode;     /* operation to perform */
          int aio_reqprio;        /* request priority offset */
          volatile void *aio_buf; /* pointer to buffer */
          size_t aio_nbytes;      /* length of operation */
          struct sigevent aio_sigevent;                     /*
signal number and value */

          /* internal, private members follow... */
  };

  int aio_read (struct aiocb *aiocbp);
  int aio_write (struct aiocb *aiocbp);
  int aio_error (const struct aiocb *aiocbp);
  int aio_return (struct aiocb *aiocbp);
  int aio_cancel (int fd, struct aiocb *aiocbp);
  int aio_fsync (int op, struct aiocb *aiocbp);
  int aio_suspend (const struct aiocb * const cblist[],
                   int n,
                   const struct timespec *timeout);

Thread-based asynchronous I/O

Linux only supports aio on files opened with the O_DIRECT flag. To perform asynchronous I/O on regular files opened without O_DIRECT, we have to look inward, toward a solution of our own. Without kernel support, we can only hope to approximate asynchronous I/O, giving results similar to the real thing.

First, let’s look at why an application developer would want asynchronous I/O:

1. To perform I/O without blocking
   
2. To separate the acts of queuing I/O, submitting I/O to the kernel, and receiving notification of operation completion

The first point is a matter of performance. If I/O operations never block, the overhead of I/O reaches zero, and a process need not be I/O-bound. The second point is a matter of procedure, simply a different method of handling I/O.

The most common way to reach these goals is with threads (scheduling matters are discussed thoroughly in Chapters 5 and 6). This approach involves the following programming tasks:

1. Create a pool of “worker threads” to handle all I/O.
   
2. Implement a set of interfaces for placing I/O operations onto a work queue.
   
3. Have each of these interfaces return an I/O descriptor uniquely identifying the associated I/O operation. In each worker thread, grab I/O requests from the head of the queue and submit them, waiting for their completion.
   
4. Upon completion, place the results of the operation (return values, error codes, any read data) onto a results queue.
   
5. Implement a set of interfaces for retrieving status information from the results queue, using the originally returned I/O descriptors to identify each operation.

This provides similar behavior to POSIX’s aio interfaces, albeit with the greater overhead of thread management.

Please check back next week for the continuation of this article.