Centralizing servers and applications logs is a good way to make search and monitoring easier. Coupled with a web-interface it simplifies access (think also about sharing logs…)

Moreover, as all my servers are based on arm mini-pc (cubieboard) with flash drive, Iogs are volatile (tmpfs) to maximize SdCards life. So centralizing logs on a harddrive is a must to keep history in case of failure.

Below is the recipe of how I did it on all my servers (cubieboards  + debian testing)

RSYSLOG Server installation

Centralizing implies having a specific server for that (or severals if you need failover). I choosed to store logs in mysql so its easier to query / view / use a web interface…

Packages installation

You will get prompted to change the root password of mysql (via dpkg-reconfigure)

The same for rsyslog-mysql plugin : give root mysql password, then enter a password for rsyslog user or let the system choose a random one…

Configure rsyslog

Edit /etc/rsyslog.conf to activate udp and tcp connections for (futur) rsyslog clients :

 

Then (re)start rsyslog

 

Install and configure the web interface : loganalyser

LAMP stack

loganalyser is written in PHP and officialy support apache. To keep things simple and straighforward, let use a classical LAMP architecture. For security reasons, it is advised to make the log server only available from the lan and/or for a restricted ip range (and for professional use or big network, I would also add through https + authentication).

 

loganalyser deployment

 

Note that we need to temporary allow write access to apache for the installation process…

Apache configuration

Now that loganalyser is deployed, we need to configure a virtualhost in apache.
Create the conf file in /etc/apache2/sites-available/loganalyser.conf

 

Then go to /etc/apache/site-enabled, remove the default host and activate loganalyser one

 

LogAnalyser configuration

Initial settings

Point your browser to the server ip adress and follow the instructions (which begin by a “fatal error screen”…)
Everything is explained in the installation doc : http://loganalyzer.adiscon.com/doc/install.html

Main choice I set (personal taste…) :

• Message character limit for the main view => 0 to view full message inline
• Show message details popup => no (because I want-it inline)
• Enable User Database => yes I use the same database as rsyslog for loganalyser own table

Rsysllog database informations are found in /etc/rsyslog.d/mysql.conf, it look-likes the following :

 

If all went fine, the installer will create tables prefixed with “logcon_” (if you did not modified this) and will ask to you create a user.

Configure a first source

LogAnaylser need to know from where it should read logs to display. For now, we just want to check that everything that will be logged will be displayed, so we create a simple source :

• Name = “All”
• Type = PDO
• View = Syslog
• Databse storage engine => MySQL
• table type = monitorWare
• Fill host, tablename, user and password according to /etc/rsyslog.d/mysql.conf (beware, all is case sensitive)

Ok, we have now a base installation that should already display the server’s logs. Don’t forget to remove write access to user www-data to /var/www/loganalyser/src.

MySQL tunning

As the logserver could receive an huge amount of data to write into tables, you should ajust mysql parameters after a fews days / weeks of usage.

Basicaly :

• use “mysqltunner” to check settings from time to time
• drefagment / optimize table at least once a week (mysqlcheck -o –all-databases -u root -pxxxxxxx)

Configures other servers to send their logs to the log server

Note that for each server and/or service, you can create a dedicated source in LogAnalyser to make browsing and searching easier…

Base logging : replace syslog by rsyslog

This part is the easiest one. Firstly, install rsyslog :

 

Then, put the following at the end of /etc/rsyslog.conf:

 

Restart rsyslog (or kill -HUP), and then every message for syslog will be sent to the log server.

Sending Apache log to rsylog

Apache use a proprietary log system that write on its own files. To send them to rsyslog we need to use the “imfile” module of rsyslog.

Edit /etc/rsyslog.conf to add file support :

 

Then, create a configuration file for apache (I personnaly create one file per virtualhost), in /etc/rsyslog.d/apache_<vhostname>.conf (You can find more information on http://www.rsyslog.com/using-the-text-file-input-module)

 

Set Exim4 to log in syslog

Edit  /etc/exim4/exim4.conf.template, and add the folowing after “main/02_exim4-config_options”

 

Then restart exim (/etc/init.d/exim4 restart) and check if its ok :

 

You should read “log_file_path = syslog”

If it does not work, try to put the “log_file_path” setting in “etc/exim4/conf.d/main/02_exim4-config_options”