Home arrow Apache arrow Page 3 - Logging in Apache

Reading the Log Files - Apache

Administrators need keep regular tabs on their Web servers to make they are running smoothly, so that their clients don't meet with any unpleasant surprises. Logging helps you to spot performance problems before they become an issue, and also assists in the detection of possible security concerns. This article will discuss configuring Apache for logging purposes, and will go into some detail about remote logging solutions. It is excerpted from Hardening Apache by Tony Mobily (Apress, 2004; ISBN: 1590593782).

  1. Logging in Apache
  2. Security Issues of Log Files
  3. Reading the Log Files
  4. Remote Logging
  5. Logging on a Remote Host
  6. Advantages and Disadvantages of Logging on a Remote Machine
  7. A Powerful, Hybrid Design
  8. Room for Improvement
By: Apress Publishing
Rating: starstarstarstarstar / 21
May 12, 2005

print this article



There is one issue that seems to be overlooked by many system administrators: it is important to read and analyze log files. As I mentioned earlier, Apache has two types of logs: the error log and the access log.

The Error Log

An ideal error log on a running server is an empty one (apart from information about the server starting and stopping) , when the error level is set to notice. For example, a “File not Found” error probably means that there is a broken link somewhere on the Internet pointing to your web site. In this case, you would see a log entry like this:

[Sat Oct 05 20:05:28 2003] [error] [client] File
does not exist: /home/merc/public_html/b.html, referer:    

The webmaster of the referrer site should be advised that there is a broken link on their site. If there is no answer, you might want to configure your Apache server so that the broken link is redirected to the right page (or, if in doubt, to your home page).

If crackers are looking for possible exploits, they will generate “File not Found” entries in the error log, so keeping the error log as clean as possible will help to locate malicious requests more easily. Some exploit attempts are logged in the error_log. For instance, you could find:

[merc@localhost httpd]$ grep -i formmail access_log
[Sun Sep 29 06:16:00 2003] [error] [client]
script not found or unable to stat: /extra/httpd/cgi-bin/formmail.pl
[merc@localhost httpd]$

The formmail script is widely used, but it generates a number of security issues.

A segmentation fault problem needs attention as well. Apache should never die, unless there is a problem in one of the modules or an attack has been performed against the server. Here is an example:

[Sun Sep 29 06:16:00 2002] [error] [notice] child pid 1772
exit signal Segmentation fault (11)

If you see such a line in the log file, you will have to see what was going on at the time in the server’s activity (possibly reading the access_log file as well) and consider upgrading Apache and its modules as soon as possible. Because of Apache’s extensive use and deployment, most such problems in the core Apache package have been eliminated. Therefore, a segmentation fault message usually implicates an after-market or third-party module failure, or a successful DOS attack.

The Access Log

The access log includes information about what the user requested. If the error log reports a segmentation fault, you can use the access log to find out what caused Apache to die. Remember that if the cause of death is really sudden, because of buffering issues, the latest log information might not be in the log file.

You can also use the access log to check whether someone is trying to break into your system. Some attacks are easy to identify by checking for the right string in the log. You can find the entries for many Windows-aimed attacks just by looking for the exe string in the access log. For example:

[root@localhost logs]# grep -i exe access_log - - [29/Sep/2003:06:25:22 +0200] "GET /_vti_bin/shtml.exe HTTP/1.0" 404 288 - - [29/Sep/2003:06:31:33 +0200] "GET /_vti_bin/shtml.exe HTTP/1.0" 404 288 - - [02/Oct/2003:02:17:53 +0200] "GET /scripts/..%c0%af../winnt/system32/cmd.exe?/c+dir+c:\ HTTP/1.1" 404 319 - -  [02/Oct/2002:02:34:46 +0200] "GET /scripts/..%255c%255c../winnt/system32/cmd.exe?/c+dir" 404 -
[root@localhost logs]#

The main problem with using grep to look for attacks: URLs can be URL-encoded (see Appendix B for more information). This means that the last entry you saw in the access_log shown above could be written as: - - [02/Oct/2003:02:34:46 +0200] "GET
/scripts/..%255c%255c../winnt/system32/cmd.%65x%65?/c+dir" 404 -

When encoded in this way, this URL would escape the grep filter. To perform an effective search, you will need to URL-decode all the URLs from your log files (you can use Perl for this), and then compare them with the suspicious ones. This simple script (Listing 3-1) should come in handy.

Listing 3-1. A Simple Script to Use As a Filter

use URI::Escape;
use strict;

# Declare some variables

# The cycle that reads
# the standard input

   # The URL is split, so that you have the
   # actual PATH and the query string in two
   # different variables. If you have
   # $path =
   # $qstring = "query=this"
   my ($path, $qstring) = split(/\?/, $_, 2);

   # If there is no query string, the result string
   # will be the path...
   $result = $path;

   # ...BUT! If the query string is not empty, it needs
   # some processing so that the "+" becomes "%20"!
   if($qstring ne ""){
           $qstring =~ s/\+/$space/ego;
           $result .= "?$qstring";

   # The string is finally unescaped...
   $str = uri_unescape($result);

   # ...and printed!

Note that the script is slightly complicated by the fact that a + (plus) in the query string (and only in the query string) must be converted into %20 ($qstring =~ s/\+/$space/ego;), which is then translated into a space once the string is URL-decoded ($str = uri_unescape($result);).

You should call this script urldecode, place it in /usr/local/bin, and give it executable permission (chmod 755 /usr/local/bin/urldecode). To test it, just run it:

[root@localhost logs]# urldecode
this is a test: .%65x%65
this is a test: .exe
[root@localhost logs]#

The script acts as a filter as it echoes information to the standard output. The command to test your logs should now be:

[root@merc root]# cat access_log | urldecode | grep exe

You can change exe into anything you want to look for in your log.

>>> More Apache Articles          >>> More By Apress Publishing

blog comments powered by Disqus
escort Bursa Bursa escort Antalya eskort


- Apache Unveils Cassandra 1.2
- Apache on ARM Chips? Dell and Calxeda Help M...
- The Down Side of Open Source Software
- VMware Unveils Serengeti for Apache Hadoop
- SAP Takes Steps to Improve Hadoop Integration
- Looking to Hone Apache Hadoop Skills?
- How to Install Joomla on WAMPP
- Working with XAMPP and Wordpress
- GUI Available for Apache Camel
- Reduce Server Load for Apache and PHP Websit...
- Creating a VAMP (Vista, Apache, MySQL, PHP) ...
- Putting Apache in Jail
- Containing Intrusions in Apache
- Server Limits for Apache Security
- Setting Permissions in Apache

Developer Shed Affiliates


Dev Shed Tutorial Topics: