Using strace to see inside processes - Apache

The strace tool (truss on systems other than Linux) intercepts and records system calls that are made by a process. It gives much insight into how programs work, without access to the source code. Using chroot and ldd, you will be able to get programs to run inside jail, but you will need strace to figure out why they fail when they fail without an error message, or if the error message does not indicate the real cause of the problem. For that reason, you will often need strace inside the jail itself. (Remember to remove it afterwards.)

Using strace you will find that many innocent looking binaries do a lot of work before they start. If you want to experiment, I suggest you write a simple program such as this one:

  #include <stdarg.h>
  #include <stdarg.h>

  int main(void) {
      puts("Hello world!");
  }

Compile it once with a shared system support and once without it:

  # gcc helloworld.c -o helloworld.shared
  # gcc helloworld.c -o helloworld.static -static

Using strace on the static version gives the following output:

  # strace ./helloworld.static
  execve("./helloworld.static", ["./helloworld.static"], [/* 22 vars */]) = 0
  uname({sys="Linux", node="ben", ...})  = 0
  brk(0)                                 = 0x958b000
  brk(0x95ac000)                         = 0x95ac000
  fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
  old_mmap(NULL, 4096, PROT_READ|PROT_WRITE,
  MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xbf51a000
  write(1, "Hello world!\n", 13Hello world!
  )          = 13
  munmap(0xbf51a000, 4096)               = 0
  exit_group(13)

The strace output is ugly. Each line in the output represents a system call made from the process. It is not important at the moment what each line contains. Jailed binaries most often fail because they cannot open a file. If that happens, one of the lines near the end of the output will show the name of the file the binary attempted to access:

  open("/usr/share/locale/locale.alias", O_RDONLY) = -1 ENOENT
  (No such file or directory)

As an exercise, use strace on the dynamically compiled version of the program and compare the two outputs. You will see how many shared libraries are accessed even from a small program such as this one.

Please check back next week for the conclusion to this article.