Problem

The sendmail tarball is a source code distribution that must be compiled. Additionally, adding a special user ID and group ID may be necessary before the new sendmail software can be installed and used.

Compile sendmail using the Build utility provided by the sendmail developers. For most systems, only a few commands are needed to compile sendmail:

  #cd sendmail-8.12.9
  #./Build

The next installation step is to create the smmsp user and group for sendmail to use when it runs as a mail submission program. Do this by using the tools appropriate to your system. Here are the /etc/passwd and /etc/group entries added to the example Red Hat system used throughout this book:

  # grep smmsp /etc/passwd
  smmsp:x:25:25:Mail Submission:/var/spool/clientmqueue:/sbin/nologin
  # grep smmsp /etc/group
  smmsp:x:25:

Next, backup the current sendmail binary, the sendmail utilities, and the current sendmail configuration files. After the system is backed up, install the new sendmail and utilities as follows:

  #./Build install

Build detects the architecture of the system and builds a Makefile customized to that system. It then uses make to compile sendmail. Build does an excellent job of correctly detecting the system architecture.

A basic Build command should work for most situations. If it doesn’t work for you, either because of your unique hardware and software, or because you have unique requirements, create a custom configuration for Build to use. sendmail calls these custom configurations site configurations and looks for them in the devtools/Site directory, where it expects to find the local custom configuration stored under the name site.config.m4. By default, Build looks for files named site.OS.m4, where OS is the name of the computer’s operating system, site.config.m4, and site.post.m4. If you use another filename, use the -f argument on the Build command line to identify the file.*For example:

As the file extension .m4 file implies, the Build configuration is created with m4 commands. Three commands are used to set the variables used by Build. These commands are:

define

The define command is a built-in m4 command. When it is used, any current value stored in the variable is replaced by the new value.

APPENDDEF

The APPENDDEF macro is an m4 macro used to append a value to an existing list of values stored in a variable. It does not replace the current value in the variable; it adds values to the end of a list of values.

PREPENDDEF

The PREPENDDEF macro is an m4 macro used to prepend a value to an existing list of values stored in a variable. It does not replace the current value in the variable; it adds values to the beginning of a list of values.

Here is an example of when a site.config.m4 file might be needed. While the sendmail software compiles without error on our sample Red Hat system, we encounter a problem when installing the manpages. The devtools/OS/Linux file that comes with sendmail 8.12.9 contains the following command, which puts the manpages in /usr/man:

  define(`confMANROOT', `/usr/man/man')

Our sample Red Hat Linux system stores manpages in /usr/share/man. Therefore, we create the following devtools/Site/site.config.m4 file to set the manpage path to /usr/share/man:

  $ cat devtools/Site/site.config.m4
  define(`confMANROOT', `/usr/share/man/man')

m4 quoted strings are enclosed in unbalanced single quotes. The same unbalanced single quotes used with any m4 command are used in the site.config.m4 file.

Most custom Build configurations are no more complicated than this example. The next few recipes show additional examples of Build configurations. Those examples were chosen because the options they compile into sendmail are required by recipes later in this book. However, there are more than 100 variables that can be set for the Build configuration—far too many to cover with individual recipes. See the devtools/README file for a complete list.

It has always been necessary to compile sendmail before installing it. Starting with sendmail 8.12, a new step has been added to the installation procedure. sendmail 8.12 expects to find a user ID and a group ID named smmsp. If this user ID and group ID do not exist on your system, create them before installing sendmail because the sendmail binary is no longer installed as set-user-ID root. Traditionally, the sendmail binary was set-user-ID root so that any user could submit mail via the command line and have it written to the queue directory. However, this does not really require a set-user-ID root binary. With the proper directory permissions, a set-group-ID binary will work fine.

Before installing the freshly compiled sendmail, back up the current sendmail binary, the sendmail utilities, and your current sendmail configuration files. (You never know; you might need to drop back to the old sendmail configuration if the new one doesn’t work as anticipated.) Remember: if the previous version of sendmail was installed with a package manager, it should be removed using that package manager before the new sendmail software is installed.

Running Build install on the sample Red Hat system installs sendmail and the utilities, and it produces more than 100 lines of output. It should run without error. On our sample system, the install commands that place the manpages clearly show the path defined earlier in the devtools/Site/site.config.m4 file. Also notice that Build uses the smmsp user and group when it creates the clientmqueue directory and when it installs the sendmail binary. A quick check of the ownership and permissions for the queue directory and the sendmail binary shows this:

  drwxrwx---  2 smmsp   smmsp    4096 Aug 7 16:22 clientmqueue
  -r-xr-sr-x  1 root    smmsp  568701 Aug 7 16:51  /usr/sbin/sendmail

After sendmail is installed, it must be configured. Most of this book discusses how to configure sendmail to do what you want it to do.

The sendmail book covers compiling and installing sendmail in Chapter 2.