Syslog

Configuring Syslog For Data Center Use.

Revision 1.0

Colin Bitterfield

August 3, 2001 

Contact Me


This paper is intended to assist a data center manager in setting up a centralized syslog server. There are a variety of commercial packages that deal with security and troubleshooting; however the use of the syslog facilities is common to all UNIX systems and most network equipment. The configurations defined here are tested in the Solaris 8 environment.

What is Syslog?

Is a well-defined program that allows you to take an action; (record to a file, send to a user, or forward to another machine) when an event takes place on your server. An event is a condition defined by a program that generates a message. Messages are defined by facility and level. Each level consists of all of its messages and all of the more severe ones. A facility is considered to be a service like the kernel, email, printer, or network. The level is the severity from (0-7), (Emergency to Debug.)

Why Centralize it?

In a complex environment like a data center, the manager of the data center may not have access to "root" on the boxes or even a login. A centralized server provides to security functions in addition to the normal monitoring. 1. It provides a separate server with different logins (limited access) to monitor all levels of security. 2. It prevents a "hacker" on a compromised server from deleting all traces of the compromise. The central server provides an easier method of finding problems. It provides one place to check. It provides a simple method for archival of logs.

What do I need?

*      A Server running Solaris 8. The processor of the machine is not as critical as the disk space.

*      128 MB RAM

*       At least 18GBs of free disk space (Preferably Mirrored/RAID5)

Notes on the Configuration Files

The configuration files for syslog are based on M4 macros, so spaces are different then tabs.

How do I configure Syslog?

There are two parts to this.

The local /etc/syslog.conf (What gets logged)

Client

#ident"@(#)syslog.conf1.598/12/14 SMI"/* SunOS 5.0 */
#
# Copyright (c) 1991-1998 by Sun Microsystems, Inc.
# All rights reserved.
#
# syslog configuration file.
#
# This file is processed by m4 so be careful to quote (`') names
# that match m4 reserved words.Also, within ifdef's, arguments
# containing commas must be quoted.
#
# Local Logging 
*.err;kern.notice;auth.notice/dev/sysmsg
*.err;kern.debug;daemon.notice;mail.crit/var/adm/messages
#
*.alert;kern.err;daemon.erroperator
*.alertroot
#
*.emerg*
#
auth.notice/var/log/authlog
mail.info/var/log/mail
#
#
#Local Messages
local0.debug/var/log/local0
local1.debug/var/log/local1
local2.debug/var/log/local2
local3.debug/var/log/local3
local4.debug/var/log/local4
local5.debug/var/log/local5
local6.debug/var/log/local6
local7.debug/var/log/local7
#
#
# Remote Logging
*.err@loghost
*.notice@loghost
*.alert@loghost
mail.info@loghost
#Local Messages
local0.debug@loghost
local1.debug@loghost
local2.debug@loghost
local3.debug@loghost
local4.debug@loghost
local5.debug@loghost
local6.debug@loghost
local7.debug@loghost

Client Rotation script for cron.

(Backup /usr/lib/newsyslog and replace with this)

#! /bin/sh
#
# Copyright(c) 1997, by Sun Microsystems, Inc.
# All rights reserved.
#
#ident @Z%newsyslog1.397/03/31 SMI
#
LOG=messages
cd /var/adm
test -f $LOG.2 && mv $LOG.2 $LOG.3
test -f $LOG.1 && mv $LOG.1 $LOG.2
test -f $LOG.0 && mv $LOG.0 $LOG.1
mv $LOG$LOG.0
cp /dev/null $LOG
chmod 644$LOG
#
# Deal with Mail
LOG=mail
cd /var/log
test -f $LOG.2 && mv $LOG.2 $LOG.3
test -f $LOG.1 && mv $LOG.1 $LOG.2
test -f $LOG.0 && mv $LOG.0 $LOG.1
mv $LOG$LOG.0
cp /dev/null $LOG
chmod 644$LOG
#
LOGDIR=/var/log
LOCALS="local0 local1 local2 local3 local4 local5 local6 local7"
if test -d $LOGDIR
then
cd $LOGDIR
for LOG in $LOCALS; do
if test -s $LOG
then
test -f $LOG.6 && mv $LOG.6$LOG.7
test -f $LOG.5 && mv $LOG.5$LOG.6
test -f $LOG.4 && mv $LOG.4$LOG.5
test -f $LOG.3 && mv $LOG.3$LOG.4
test -f $LOG.2 && mv $LOG.2$LOG.3
test -f $LOG.1 && mv $LOG.1$LOG.2
test -f $LOG.0 && mv $LOG.0$LOG.1
mv $LOG$LOG.0
cp /dev/null $LOG
chmod 644$LOG
fi
done 
sleep 40
fi
#
kill -HUP `cat /etc/syslog.pid`

Server Configuration(syslog.conf)

kern.debug/var/log/kernel
mail.info/var/log/mail
user.info/var/log/user
lpr.notice/var/log/lpr
cron.notice/var/log/cron
*.err;kern.debug;daemon.notice;mail.crit/var/adm/messages
# Config for other Local Syslogs
#Local Messages
local0.debug/var/log/local0
local1.debug/var/log/local1
local2.debug/var/log/local2
local3.debug/var/log/local3
local4.debug/var/log/local4
#Applications
local5.debug/var/log/local5
# CISCO Routers
local6.debug/var/log/local6
# Cisco Switchs
local7.debug/var/log/local7

Server Configuration Cron Rotation Script

#! /bin/sh
#
# Copyright(c) 1997, by Sun Microsystems, Inc.
# All rights reserved.
#
#ident @Z%newsyslog1.397/03/31 SMI
#
LOG=messages
cd /var/adm
test -f $LOG.2 && mv $LOG.2 $LOG.3
test -f $LOG.1 && mv $LOG.1 $LOG.2
test -f $LOG.0 && mv $LOG.0 $LOG.1
mv $LOG$LOG.0
cp /dev/null $LOG
chmod 644$LOG
#
#
LOGDIR=/var/log
LOCALS="kernel user lpr mail cron local0 local1 local2 local3 local4 local5 loca
l6 local7"
if test -d $LOGDIR
then
cd $LOGDIR
for LOG in $LOCALS; do
if test -s $LOG
then
test -f $LOG.6 && mv $LOG.6$LOG.7
test -f $LOG.5 && mv $LOG.5$LOG.6
test -f $LOG.4 && mv $LOG.4$LOG.5
test -f $LOG.3 && mv $LOG.3$LOG.4
test -f $LOG.2 && mv $LOG.2$LOG.3
test -f $LOG.1 && mv $LOG.1$LOG.2
test -f $LOG.0 && mv $LOG.0$LOG.1
mv $LOG$LOG.0
cp /dev/null $LOG
chmod 644$LOG
fi
done 
sleep 40
fi
#
kill -HUP `cat /etc/syslog.pid`

Notes on Configurations:

In the preceding examples the facility "uucp" and "news" were purposefully omitted because they were not in use in the test lab. If you use news or uucp, you would need to make appropriate changes to the configurations and scripts.

The facility local5 was setup to be the application log facility. You need to add the following to the various application start and stop scripts.

Start:

logger -p local5.info "Application STARTed normally"

Stop:

logger -p local5.info "Application STOPped normally"

Error

logger -p local5.err "Application failed to start"

Required Basic Machine Preparation:

1.    Setup NTP on everything. (Accurate time is required)

2.   Deativate all unnecessary services (i.e. telnet)

3.    Install openssh

4.   Limit access to machine. Users on client machines should not be users on this machine. Do not put on naming service like NIS, NIS+ or LDAP. Each operator should have his or her own account.

5.   Harden the machine as much as possible

6.   Make sure that /var/logs & /var/adm are on their own slices or change the control files.

Sample CISCO Configuration File:

logging facility local6

logging 192.168.1.10 (Ip of your syslog server)

! ntp configurable

ntp clock-period 17180345

ntp server 206.40.88.30

ntp server 192.5.5.250

Sample NTP Configuration [2]

File /etc/inet/ntp.conf

server 16.1.0.4 (This will work if you are on the Internet)

Future Considerations:

·        Restricting remote messages to server only starting syslogd with the –t option.

·        Setting up a standards document for Application installation and configuration.

Appendix:

Facility Definitions:[1]

Keyword

Description

Symbol in
sys/syslog.h

syslog

number

kern

Kernel

LOG_KERN

0

user

User Processes

LOG_USER

1

mail

Electronic Mail

LOG_MAIL

2

daemon

Background System Processes

LOG_DAEMON

3

auth

Authorization

LOG_AUTH

4

syslog

System Logging

LOG_SYSLOG

5

lpr

Printing

LOG_LPR

6

news

Usenet News

LOG_NEWS

7

uucp

Unix-to-Unix Copy Program (uucp)

LOG_UUCP

8

sys9 – sys14

Reserved for System (defined only on Cisco router)

Not Defined

9 - 14

cron

Daemon to Execute Scheduled Commands

LOG_CRON

15

local0 – local7

For Local Use

LOG_LOCALn

16 - 23

Table 2: Severity Definitions [1]

Keyword for

syslog.conf

Keyword for

Cisco Router

Symbol in
syslog.h

syslog

number

emerg

emergencies

LOG_EMERG

0

alert

alerts

LOG_ALERT

1

crit

critical

LOG_CRIT

2

err

errors

LOG_ERR

3

warning

warnings

LOG_WARNING

4

notice

notifications

LOG_NOTICE

5

info

informational

LOG_INFO

6

debug

debugging

LOG_DEBUG

7

References :

[1] "Log Consolidation with syslog" by Donald Pitts, December 23, 2000 

http://www.sans.org/infosecFAQ/unix/syslog.htm

[2] Understanding and using the Network Time Protocolby Ulrich Windl, et al. 

http://www.eecis.udel.edu/~ntp/ntpfaq/NTP-a-faq.htm

Analysis of Log Files:

"Automated Analysis of Cisco Log Files", Copyright © 1999, Networking Unlimited, Inc. All Rights Reserved

http://www.networkingunlimited.com/white007.html

Further Research:

(These URLs are reference by [1]). I have reviewed them that they are worthy of a read. 

1] "File Formats - syslog.conf(4)." 22 January 1997. URL:


http://www.bama.ua.edu/cgi-bin/man-cgi?syslog.conf+4 (23 December 2000). 

[2] Internet Engineering Task Force. "Security Issues in Network Event Logging (syslog)." 23 October 2000. URL: http://www.ietf.org/html.charters/syslog-charter.html   (23 December 2000).

[3] Carnegie Mellon University. "Manage logging and other data collection mechanisms." 18 October 2000. URL: http://www.cert.org/security-improvement/practices/p092.html   (23 December 2000).

[4] Orebaugh, Angela. "Securing Solaris." October 2, 2000. URL: http://www.sans.org/infosecFAQ/sec_solaris.htm(23 December 2000).

[5] Bezroukov, Nikolai. "Softpanorama University Pages: Solaris Hardening and Security." URL: http://www.softpanorama.org/Security/sos.shtml(23 December 2000).

[6] "Maintenance Commands - syslogd(1M)." 27 February 1997. URL: http://www.bama.ua.edu/cgi-bin/man-cgi?syslogd+1M (23 December 2000).

[7] "Tucows Linux Man Pages syslogd.8." 12 October 1998. URL: http://howto.tucows.com/man/man8/syslogd.8.html (23 December 2000).

[8] Carnegie Mellon University. "Configure firewall logging and alert mechanisms." CERT Security Improvement Modules. 2 August 1999. URL:
http://www.cert.org/security-improvement/practices/p059.html (23 December 2000).

[9] Adiscon. "EventReporter." URL:
http://www.eventreporter.com/en/Product/Integrate-NT-Event-Log-into-Unix-Syslogd.asp (23 December 2000).

[10] Kiwi Enterprises. "Kiwi’s Software." 23 December 2000. URL:
http://www.kiwi-enterprises.com/products.htm (23 December 2000).

[11] Cisco Systems. "Logging." Improving Security on Cisco Routers. URL:http://www.cisco.com/warp/public/707/21.html#logging (23 December 2000).

[12] Cisco Systems. "Troubleshooting Commands." 19 December 2000. URL: http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/12cgcr/fun_r/frprt3/frtroubl.htm (23 December 2000).

[13] Cisco Systems. "Managing the System." URL: http://www.cisco.com/univercd/cc/td/doc/product/software/ios111/mods/1mod/1cbook/1csysmgt.htm (23 December 2000).

[14] Cisco Systems. "Logging and Counter Caveats." Characterizing and Tracing Packet Floods Using Cisco Routers. URL: http://www.cisco.com/warp/public/707/22.html#3f (23 December 2000).

[15] Carnegie Mellon University. "Using newsyslog to rotate files containing logging messages on systems running Solaris 2.x." 2 March 2000. URL:
http://www.cert.org/security-improvement/implementations/i041.09.html (23 December 2000).

[16] Carnegie Mellon University. "Understanding system log files on a Solaris 2.x operating system." 2 March 2000. URL:
http://www.cert.org/security-improvement/implementations/i041.12.html (23 December 2000).

[17] Reed, Darren. "Nsyslogd." URL:http://coombs.anu.edu.au/~avalon/nsyslog.html (23 December 2000).

[18] Scheidler, Balázs. "syslog-ng." 6 November 2000. URL: http://www.balabit.hu/products/syslog-ng (23 December 2000).

[19] "Core FreeSoft." URL: http://www.core-sdi.com/english/freesoft.html(23 December 2000).

[20] Conover, Matt. "Index of /files/SRS." URL: http://www.w00w00.org/files/SRS/ (23 December 2000).

[21] Stokely, Celeste. "Celeste’s Tutorial On Solaris 2.x Modems & Terminals." 20 May 2000. URL: http://www.stokely.com/unix.serial.port.resources/modem.html (23 December 2000).

[22] Spitzner, Lance. "Watching Your Logs." 19 July 2000. URL: http://www.enteract.com/~lspitz/swatch.html   (23 December 2000).

[23] Rowland, Craig. "Psionic Logcheck Version 1.1.1." 10 May 2000. URL: http://www.psionic.com/abacus/logcheck/(23 December 2000).

last update on 28 Apr, 2005

[Colin's Home] [News] [Contact me] [Articles] [Solaris Resources] [Win32 Resources] [Mooniacs] [Aviation] [Photo Albums] [Family] [Private]