November 2006 - syslog-ng - lists.balabit.hu

AIX startup
by Brian Loe 27 Mar '08

27 Mar '08

Okay, my syslog-ng instance died at some point last night and because I don't have it setup properly for respawning... Can anyone running syslog-ng on AIX 5.* give me some tips for this? I'm feeding pipes so i have to start instances of SEC as soon as I start syslog-ng - but the SEC scripts appear to "close" so if they're configured to respawn I'll have 2000 instances of them in an hour.

7 21

incorrect firewall logs with syslogng-2.0.0 [security error?]
by Jacek Kalinski 12 Mar '08

12 Mar '08

Hello, After upgradeing from syslogng-1.6.11 to 2.0.0 I've got a strange messages in /var/log/messages file: Nov 29 08:14:05 denise WINDOW=16985 RES=0x00 ACK URGP=0 Nov 29 11:10:19 denise PT=34536 WINDOW=16985 RES=0x00 ACK URGP=0 Nov 29 16:42:16 denise W=0 RES=0x00 RST URGP=0 Nov 29 21:37:15 denise 116 ID=34901 DF PROTO=TCP SPT=3584 DPT=80 WINDOW=65535 RES=0x00 ACK FIN URGP=0 These are truncated iptables logs. Because in firewall.log file are simmilar entries to truncated one, I think full file should be: Nov 29 11:10:19 denise firewallp=INVALID:1 a=DROP IN=eth1 OUT= MAC=00:30:4f:36:2b:dc:00:04:9a:2c:7f:20:08:00 SRC=193.41.230.81 DST=xxx.xxx.xxx.x LEN=576 TOS=0x00 PREC=0x00 TTL=117 ID=3495 PROTO=TCP SPT=443 DPT=34536 WINDOW=16985 RES=0x00 ACK URGP=0 (xxx.xxx.xxx.x is a real server IP address - one 'x' is one digit) Config file with interesting lines: options { dir_perm(0700); perm(0600); chain_hostnames(no); stats(1200); keep_hostname(yes); dir_owner(root); dir_group(root); owner(root); group(root); }; source local { unix-stream("/dev/log" max_connections(300)); file("/proc/kmsg"); # udp(ip(0.0.0.0) port(514)); internal(); }; # ..... # authpriv.none;cron.none;mail.none;news.none -/var/log/messages filter f_2 { not facility(news); }; filter f_3 { not facility(local5); }; filter f_4 { not facility(local6); }; filter f_5 { not facility(authpriv); }; filter f_6 { not facility(cron); }; filter f_7 { not facility(mail); }; filter f_notlocal7 { not facility(local7); }; filter f_notfirewall { not match("firewallp="); }; destination d_2 { file("/var/log/messages" create_dirs(yes)); }; log { source(local); filter(f_2); filter(f_4); filter(f_5); filter(f_6); filter(f_7); filter(f_notimapd); filter(f_notpopa3d); filter(f_notnamed); filter(f_notfirewall); destination(d_2); }; #..... # firewallp.* -/var/log/firewall.log filter f_16 { level(warning) and match("firewallp="); }; destination d_12 { file("/var/log/firewall.log" create_dirs(yes)); }; log { source(local); filter(f_16); destination(d_12); }; #..... Do you know where could be an error? It is a production server so I can't debug syslog-ng on this machine (sorry). NOTE: the only truncated lines I can find are iptables (AFAIR they are logged by /proc/kmesg) entries. All other lines are complete. And it is probably a critical error in security. PS: simple statistics line from syslog-ng: Nov 30 00:00:06 denise syslog-ng[32682]: Log statistics; dropped='pipe(/dev/tty12)=0', processed='center(queued)=758289', processed='center(received)=369020', processed='destination(d_maillogimapd)=4245', processed='destination(d_5)=1', processed='destination(d_4)=0', processed='destination(d_3)=14259', processed='destination(d_2)=14259', processed='destination(d_13)=0', processed='destination(d_1)=6000', processed='destination(d_12)=6000', processed='destination(d_11)=369020', processed='destination(d_10)=36', processed='destination(d_8)=0', processed='destination(d_maillogpopa3d)=103224', processed='destination(d_7)=241245', processed='destination(d_6)=0', processed='source(local)=369020' I think you see that it could be a little hard to debug (with so many messages parsed by syslog-ng) ;) Jacek

3 6

Growing log files (i.e. access.log) and regex.
by Robert.Becker＠motoristsgroup.com 08 Mar '07

08 Mar '07

Here's the scenario. We have a JBoss server that outputs to *.log files. After doing all the research I could, I found out the only way to get these logs sent to syslog-ng was through a tail & logger command. The problem with that is logger appends the date and some other information to that logged message. When attempting to put that information in a mysql database I'm getting errors. So, what I would like to do is use some form of regex to alter the statement so it can be inserted into a database. There are basically two things that need to be changed. First, logger appends the date/time it receives the message and it also says it is coming from logger. I would like to remove that date and switch logger to jboss or something similar. Here is an example of what I am doing. This is a line from the jboss log file: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Then I send the log through logger to syslog-ng: #!/bin/sh tail -f /opt/jboss/server/default/log/boot.log | logger -p local7.info The log gets turned into this: Oct 11 13:14:15 src@linuxp17 logger: 2005-10-11 13:14:15,848 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 What I need is: Oct 11 13:14:15 src@linuxp17 jboss: DEBUG [org.jboss.resource.connectionmanager.IdleRemover] run: IdleRemover notifying pools, interval: 450000 Now, I've thought of using awk in the tail/logger script to remove the date, but that does not take care of the logger/jboss substitution. Anyone have any ideas? Can syslog-ng do this with the match() function? -Rob Becker Systems Engineer Motorists Insurance ********************************************************************** The information contained in this message is confidential and is intended for the addressee(s) only. If you have received this message in error or there are any problems please notify the originator immediately. The unauthorized use, disclosure, copying or alteration of this message is strictly forbidden. Motorists Insurance Group will not be liable for direct, special, indirect or consequential damages arising from the alteration of the contents of this message by a third party or as a result of any virus being passed on. **********************************************************************

6 5

Analysis of logs and alerting
by Muath Al Khalaf 11 Feb '07

11 Feb '07

Hi, I am implementing a syslog-ng centralized correlation server. The server receives the log entries from some servers, apps and devices. Is there any good product (commercial or open-source) with GUI or web interface to analyze these logs and give real time alerts with strong data mining tools? I need a tool which could be used by both help desk non-technical staff and technical engineers. It should give basic alerting on applications' or devices' failures first for help desk, and then engineers could dig deeper into these alerts to understand the problem. Kind regards

4 3

syslog-ng cvs / svn
by Brian A. Seklecki 24 Jan '07

24 Jan '07

Where is the repo? I want to submit a patch to a bad example in the docs. References to cvs.balabit.hu are clearly invalid. l8* -lava (Brian A. Seklecki - Pittsburgh, PA, USA) http://www.spiritual-machines.org/ "...from back in the heady days when "helpdesk" meant nothing, "diskquota" meant everything, and lives could be bought and sold for a couple of pages of laser printout - and frequently were."

3 4

syslog-ng 2.0.0 successful on Mac OS X Tiger
by Orth, Alan 14 Dec '06

14 Dec '06

Good morning, For some time now I've have syslog-ng 1.6 working on Mac OS X Panther servers, but I could never get 1.6 or 1.9 (and probably some early 2.0 betas) to work. I just completed an install of 2.0.0 on a clean installed Tiger server. I don't have a complicated logging setup (4 filters, 4 logging destinations), but I do like syslog-ng's ability to eliminate the need for log rotation by logging filters and directory structures... I know I had been trying every few months over the last year or so to get this installed, and a few other people posted with trouble getting it to run, so I'll post some stats about my setup: Machine is a 2 GHz G5 Xserve running Mac OS X Server 10.4.8 (8L127). $ uname -a Darwin hostname.school.edu 8.8.0 Darwin Kernel Version 8.8.0: Fri Sep 8 17:18:57 PDT 2006; root:xnu-792.12.6.obj~1/RELEASE_PPC Power Macintosh powerpc RackMac3,1 Darwin Good work, whoever fixed it! -- Alan Orth Upward Bound - Systems Administrator User Services - Field Technician California State University, Chico x6000

3 9

Advice on keeping hostnames/using dns
by Hari Sekhon 07 Dec '06

07 Dec '06

Hi, I'd like some advice on what I should do on my logserver regarding hostnames. I've currently got keep_hostnames(no) use_dns(yes) in order to get accurate and consistent hostnames but I'd like to consider just skipping the whole dns check rewriting thing and use keep_hostnames(yes) use_dns(no) The only issue I can see from this is that the hostname gets logged according to the packet. I'm reasonably confident that most machines will report the right name in their logs to the logserver but I also think that it makes it all too possible to screw up the logserver maliciously since any old junk that is sent to the port is put into the logs so you could hammer the integrity of the logs just by sending loads of bogus logs from a machine with the name set to that of any other machine on the network. Got any views on this? All opinions welcome. Thanks Hari -- Hari Sekhon

8 13

syslog-ng 2.0.0 compile problem
by Buozis, Martynas 02 Dec '06

02 Dec '06

Hello I tried to compile syslog-ng on Solaris 10 with Sun Studio 11 using following configure line : PKG_CONFIG_PATH=/opt/eventlog/lib/pkgconfig/ GLIB_LIBS=/opt/glib/lib/ CC=cc CXX=CC ./configure --prefix=/opt/syslog-ng --sysconfdir=/opt/syslog-ng/ I have compiled Eventlog and GLIB (statistically) with same compiler. Alas configure returns error: error: static GLib libraries not found (a file named libglib-2.0.a), either link GLib dynamically using the --enable-dynamic-linking or insta ll a static Glib What I found in config.log is a syntax problem in test code? Can this be fixed somehow by some patch or workaround? I also think, that compile line is also not correct : configure:5687: cc -o conftest -g conftest.c -Wl,-Bstatic /opt/glib/lib/ -Wl,-Bdynamic >&5 "conftest.c", line 52: syntax error before or at: { "conftest.c", line 55: undefined symbol: fn "conftest.c", line 59: warning: syntax error: empty declaration "conftest.c", line 60: syntax error before or at: return "conftest.c", line 60: warning: syntax error: empty declaration cc: acomp failed for conftest.c configure:5693: $? = 2 configure: failed program was: | /* confdefs.h. */ | | #define PACKAGE_NAME "" | #define PACKAGE_TARNAME "" | #define PACKAGE_VERSION "" | #define PACKAGE_STRING "" | #define PACKAGE_BUGREPORT "" | #define PACKAGE "syslog-ng" | #define VERSION "2.0.0" | #define STDC_HEADERS 1 | #define HAVE_SYS_TYPES_H 1 | #define HAVE_SYS_STAT_H 1 | #define HAVE_STDLIB_H 1 | #define HAVE_STRING_H 1 | #define HAVE_MEMORY_H 1 | #define HAVE_STRINGS_H 1 | #define HAVE_INTTYPES_H 1 | #define HAVE_STDINT_H 1 | #define HAVE_UNISTD_H 1 | #define HAVE_STRINGS_H 1 | #define HAVE_GETOPT_H 1 | #define HAVE_STROPTS_H 1 | #define HAVE_SYS_STRLOG_H 1 | #define HAVE_DOOR_H 1 | #define HAVE_O_LARGEFILE 1 | #define HAVE_LIBDOOR 1 | #define HAVE_LIBSOCKET 1 | #define HAVE_LIBRT 1 | #define HAVE_LIBNSL 1 | #define HAVE_STRDUP 1 | #define HAVE_STRTOL 1 | #define HAVE_INET_NTOA 1 | #define HAVE_GETOPT_LONG 1 | #define HAVE_GETADDRINFO 1 | #define HAVE_PTHREAD_H 1 | #define HAVE_LIBPTHREAD 1 | #define ENABLE_SUN_STREAMS 1 | #define ENABLE_SUN_DOOR 1 | #define ENABLE_DEBUG 0 | #define ENABLE_MEM_TRACE 0 | #define ENABLE_SSL 0 | #define ENABLE_IPV6 1 | /* end confdefs.h. */ | | int | main () | { | | extern void g_hash_table_new(void); | | int main() | { | void (*fn)(void) = g_hash_table_new; | | return (int) fn; | } | | | ; | return 0; | } configure:5717: result: no configure:5728: error: static GLib libraries not found (a file named libglib-2.0.a), either link GLib dynamically using the --enable-dynamic-linking or insta ll a static Glib I can fix above code with look like : /* end confdefs.h. */ extern void g_hash_table_new(void); int main () { void (*fn)(void) = g_hash_table_new; return (int) fn; } And compile it with command : cc -o test -g test.c -Wl,-Bstatic /opt/glib/lib/libglib-2.0.a -Wl,-Bdynamic With best regards Martynas

4 3

unix_stream socket
by pat.hartemann＠bluewin.ch 30 Nov '06

30 Nov '06

Hello all, I'm configuring syslog-ng 2.0 on a RHEL ES 4 U4 system. One of the destination is an unix stream socket. After the first log writen on this socket, I get the error message : ------------------------------------------------------------------------------------------------------- Nov 27 11:32:30 ch-syslog-1 syslog-ng[10153]: I/O error occurred while writing; fd='8', error='Connection refused (111)' Nov 27 11:32:30 ch-syslog-1 syslog-ng[10153]: Connection broken; time_reopen='10' ------------------------------------------------------------------------------------------------------- I checked the socket with a thin client (perl script) and that's working (whitout restart socket server of course), so I assume this issue comes from syslog-ng, but syslog-ng works as root ... I'm little bit confusing ... Hereby the the dest, template and options configurations. -------------------------------------------------------------------------------------------------------- options { sync (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); flush_lines(1); flush_timeout(5000); }; template t_test { template ("$R_STAMP;$S_STAMP;$HOST;$FACILITY;$LEVEL;$PROGRAM;$MSGONLY\n"); }; destination d_test { file("/var/log/test.log" template(t_test)); unix-stream('/var/tmp/syslog_generic.soc' template(t_test));}; --------------------------------------------------------------------------------------------------------- Many thx for your help Ciao patrick

3 3

UI for editing syslog-ng configuration files
by John Svazic 29 Nov '06

29 Nov '06

Hi, Not sure if this is the best forum to ask this question, but I'm not having any luck trolling through Google, Sourceforge and Freshmeat. Does anyone know of any front-end UI (web or GUI) for editing a syslog-ng configuration file? I'm looking for something simple for users who are not familiar with the config file format to take on. Thanks!

5 5