- syslog-ng - lists.balabit.hu

patterndb context-timeout problem
by Zoltán Pallagi 04 Mar '11

04 Mar '11

Hi, I've found a problem in handling context-timeout in a special case. If context-timeout=10 (10 sec) and I am using a trigger, where trigger="timeout", it can happen that the timeout won't expire: I sent the following two messages to syslog-ng before expiring timeout (but the difference between the two timestamp is bigger than the timeout value): "<38>1990-01-01T14:45:25 customhostname program6[1234]: program6 testmessage" "<38>1990-01-01T14:55:25 customhostname program6[1234]: program6 testmessage" I receive the timeout message immediately, when the second message arrives (it's ok), but there is no timeout for the second message.

2 1

Message correlation, everything needs an "@"? And a grep problem.
by Valentijn Sessink 04 Mar '11

04 Mar '11

Hello list, After playing with correlation for a while, here are a few remarks on it's workings. I have played with syslog-ng 3.2.2, built from source on an Ubuntu 10.04 machine. For some reason, the "@1", "@2"... etcetera specifiers needed to be all over the place to be able to see any variable at all. For example, the rule below describes the last message of a Postfix log trail. Note that even the context-id is specified as "${queueid}@1". All other references also have an "@" sign. Without the @, the variables seem just empty. <rule id='X' context-id="${queueid}@1" context-scope="host" provider="openoffice" class="maildelivery"> <patterns> <pattern>@ESTRING:queueid: @removed</pattern> </patterns> <actions> <action> <message> <values> <value name="MESSAGE">${queueid}@1 mail to=<${toaddress}@2> from=<${pffrom}@3> connection from=${pfremotehost}@5[${pfremoteip}@5] msgid=<${msgid}@4> orig_to=<${origto}@2> delay=${deliverystatus}@2</value> </values> </message> </action> </actions> </rule> Is this how it's meant to be? Then, when trying to use the "grep" template function in this message, I can't seem to find out how it works. Using the exact example from the documentation, $(grep ("${username}" == "root") ${auth_method}), I only get an error: Error parsing filter expression, syntax error, unexpected $undefined in <string> at line 1, column 3: (${ ^ Trying to alter the message, using "$something" == "value", doesn't seem to have an effect, i.e. I can't get "grep" to find any values. In fact, I'm not even sure if I can use the names from the (E/Q)STRING parsers right away, or if I need to have <value name="something"> tags before I can grep for them. In fact, most of the time, the grep I would like to use is something like: $(grep ( "$variable" != "" ) "$variable"). But whatever I try, no results from grep. Could anyone please clarify? Best regards, Valentijn

2 15

Permission Denied (13)
by Hidayath Basha 04 Mar '11

04 Mar '11

Hi, we are using SLES 11 We are trying to read a tailing file and send it to the server syslog-ng.conf (on client 192.168.18.10 some-host) #syslog-ng.conf options { long_hostnames(off); sync(1); perm(0640); stats(3600); }; source s_mysrc{ file("/some/file.log" ); }; destination d_mydest { tcp("192.168.18.7" port(5140)); }; log { source(s_mysrc); destination(d_mydest); }; #syslog-ng.conf ends syslog-ng.conf (on Server 192.168.18.7) #syslog-ng.conf options { long_hostnames(off); sync(0); perm(0640); stats(3600); }; source s_mysrc{ tcp(ip("192.168.18.7") port(5140) keep-alive(yes)); }; destination d_mydest { file ("/var/log/hosts/$YEAR-$MONTH/$HOST/$FACILITY-$YEAR-$MONTH-$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes) ); }; log { source(s_mysrc); destination(d_mydest); }; #syslog-ng.conf ends When I start the service at client, I'm getting the following error Starting syslog servicesError opening file for reading; filename='/some/file.log', error='Permission denied (13)' Error initializing source driver; source='s_mysrc' startproc: exit status of parent of /sbin/syslog-ng: 2 The file /some/file.log has -rw-rw-rw- permissions When I modify the source driver to ... file("/some/file.log" follow_freq(1)); I'm able to start the syslog service, but I don't see anything on the server (192.162.18.7) at /var/log/hosts/2011-03/some-host/ Please help Thanks and regards, Hidayath Saven Technologies: A leader in technology driven business solutions ---------------------------------------------------------------------------- ----------------------------------- #302, 3rd Floor, MyHome Sarovar Plaza, Hyderabad- 500 063, India Off: +91 040 23237303/7304 ext 115 email: hidayath.basha(a)saventech.com | www.saventech.com follow me: http://twitter.com/hidayath26 ---------------------------------------------------------------------------- -----------------------------------

2 3

[Bug 117] New: syslog-ng.persist should not reside in /var/lib
by bugzilla＠bugzilla.balabit.com 03 Mar '11

03 Mar '11

https://bugzilla.balabit.com/show_bug.cgi?id=117 Summary: syslog-ng.persist should not reside in /var/lib Product: syslog-ng Version: unspecified Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: wlt(a)obsidian-studios.com Type of the Report: --- Estimated Hours: 0.0 syslog-ng uses a syslog-ng.persist file that is created in /var/lib. Per FHS 2.3 that should exist in a subdir, if not one specific to syslog-ng, then /var/lib/misc. http://www.pathname.com/fhs/pub/fhs-2.3.html#VARLIBVARIABLESTATEINFORMATION An application (or a group of inter-related applications) must use a subdirectory of /var/lib for its data. There is one required subdirectory, /var/lib/misc, which is intended for state files that don't need a subdirectory; the other subdirectories should only be present if the application in question is included in the distribution. [38] [38] An important difference between this version of this standard and previous ones is that applications are now required to use a subdirectory of /var/lib. This causes problems on diskless systems with a shared /var/lib. Subdirs that contain unique per system data are not shared, and instead mounted on top of the shared /var/lib. I could do the same thing even for a single file, syslog-ng.persist in this case. However given FHS, it should be written in a subdir. Which I can then make that unique per system, like other subdirs in /var/lib that are not shared amongst all diskless systems. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

internal-sftp and logs files
by Naira Kaieski 03 Mar '11

03 Mar '11

Good afternoon, Staff set up openssh to direct users to a certain group members to a chroot environment and these users will have access only to the server using sftp protocol. Put in the sshd_config file: Match Group customers ChrootDirectory% h ForceCommand internal-sftp-l VERBOSE f-AUTH Thus each user is directed to the chroot environment indicated in the variable% h (home directory defined in / etc / passwd) An example of directory is: User: naira Home directory: /var/www/naira.com.br The problem is that I am not able to capture logs of the user group "clients" that are targeted to the chroot environment. Access via internal-sftp from other users who do not belong to the "client" I get the logs in auth.log files. I just create dir and file /var/www/naira.com.br/dev/log but i not had log files... I'm using syslog-ng. Has anyone ever made this kind of setup? Thanks, -- Naira Kaieski Nucleo de Internet/Redes - Faccat Linux Professional Institute - LPI000223834

2 2

linking changes
by Balazs Scheidler 03 Mar '11

03 Mar '11

Hi, I've tried to put a closure on how syslog-ng 3.2 and above is being linked. There were numerous problems as it started to get into Linux distributions. This patch documents the intent with --enable-mixed-linking and how that behaves wrt the executables & shared libraries that syslog-ng installs. commit f2c7f3ea16d81e82b79582d3f2fe3882add4eb23 Author: Balazs Scheidler <bazsi(a)balabit.hu> Date: Wed Feb 23 13:01:02 2011 +0100 configure.in: reorganize LIBS variables so that unit tests/programs can be linked in mixed mode too Previously if --enable-mixed-linking was specified, unit tests and other tools (like pdbtool) couldn't be linked properly. The situation with --enable-mixed-linking is: * the primary aim here is to have a syslog-ng executable that can be started early in the boot process, and not a set of syslog-ng executables that can be distributed to different hosts that lack the required dependencies. * syslog-ng contains all static libs required by libsyslog-ng.so * therefore libsyslog-ng.so cannot be used outside the syslog-ng binary (except if the executable is linked against the required libs) * tools like pdbtool will be linked dynamically * unit tests will be linked dynamically too This patch cleans up and documents the purpose of various _DEPS_LIBS variables and updates Makefile.am files to use the updated names. A static GLib detection bug was also fixed. Reported-By: Arkadiusz Miśkiewicz <arekm(a)maven.pl> Signed-off-by: Balazs Scheidler <bazsi(a)balabit.hu> A related change is that the shared library libsyslog-ng-patterndb.so that was introduced in 3.2.2 will be dropped. Also, the main syslog-ng executable is not linked against libssl/libcrypto on purpose, it is only used by the afsocket module (which implements the tcp/syslog destinations). Could anyone who is in charge of maintaining syslog-ng binaries in various distributions give this a test so 3.2.3 will compile everywhere out-of-the-box? Thanks in advance, -- Bazsi

4 29

patternize and IPv6 hosts
by Valentijn Sessink 02 Mar '11

02 Mar '11

Hi, Just a small remark. If you run pdbtool with "patternize" on a log file with logs from an IPv6 host, pdbtool thinks that everything after the first ":" is the log message. For example, the log message: Feb 24 11:39:26 2a01:4f8:8a0:5141::3c2 named[31090]: lame server resolving .... ... will be patternized as if the logging host were "2a01" and if the pattern should be: <pattern>4f8:8a0:5141::3c2 named[31090]: lame server resolving ....</pattern> I searched around in the patternize code, but could not find the particular code where the host got cut off. However, for someone a bit at home in this code, I think the fix should be trivial. Best regards, Valentijn

3 6

afsql dbi initialization errors checks
by Hendrik Visage 02 Mar '11

02 Mar '11

Hi there, Busy compiling (more like porting) syslog-ng 3.2 to HP-UX 11.31, and I've been hitting a few snags that I since over come, but now I hit one where I see the source code is not checking whether the afsql got a proper DBI initialization, nor whether the actual driver is available before starting the db thread/etc. I've modified the lines 988-990 below, where they were just a simple "dbi_initialize();". On my setup, it "initialized", but with 0 drivers available :( (No, I DO have installed the Postgrsql dbi-driver, so yes I'll be debugging the DBI side now :() 986 if (!dbi_initialized) 987 { 988+ msg_error("HVT: dbi_initialization", 989= evt_tag_int("status",dbi_initialize("/opt/hvsyslog-ng3/lib/dbd") ), 990+ NULL); 991 dbi_initialized = TRUE; 992 } I also would like syslog-ng to a check at startup time and to rather exit with an error if the requested DBI driver is not available, than to wait till the messages starts to stream in and then to throw screens full of errors :( Hendrik

2 3

[Bug 116] New: Gentoo QA warning: app-admin/syslog-ng-3.1.4: logpatterns.c: 220: warning: implicit declaration of function ‘atoi’
by bugzilla＠bugzilla.balabit.com 02 Mar '11

02 Mar '11

https://bugzilla.balabit.com/show_bug.cgi?id=116 Summary: Gentoo QA warning: app-admin/syslog-ng-3.1.4: logpatterns.c:220: warning: implicit declaration of function ‘atoi’ Product: syslog-ng Version: 3.1.x Platform: PC OS/Version: Linux Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: mrdanwallis(a)gmail.com Type of the Report: bug Estimated Hours: 0.0 When installing the app-admin/syslog-ng-3.1.4 package on my Gentoo system, I received the following notification: * QA Notice: Package has poor programming practices which may compile * fine but exhibit random runtime failures. * logpatterns.c:220: warning: implicit declaration of function ‘atoi’ * Please do not file a Gentoo bug and instead report the above QA * issues directly to the upstream developers of this software. * Homepage: http://www.balabit.com/products/syslog_ng/ * ERROR: app-admin/syslog-ng-3.1.4 failed: * install aborted due to poor programming practices shown above Looking through the build output, here is the warning in context: i686-pc-linux-gnu-gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I.. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/eventlog -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -march=i686 -pipe -Wall -c apphook.c i686-pc-linux-gnu-gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I.. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/eventlog -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -march=i686 -pipe -Wall -c radix.c i686-pc-linux-gnu-gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I.. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/eventlog -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -march=i686 -pipe -Wall -c logpatterns.c logpatterns.c: In function ‘log_classifier_xml_start_element’: logpatterns.c:220: warning: implicit declaration of function ‘atoi’ i686-pc-linux-gnu-gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I.. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/eventlog -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -march=i686 -pipe -Wall -c cfg-lex.c cfg-lex.c:1562: warning: ‘input’ defined but not used i686-pc-linux-gnu-gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I.. -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/eventlog -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -march=i686 -pipe -Wall -c cfg-grammar.c -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

consecutive pattern parsers, and some other pattern matching questions
by Valentijn Sessink 01 Mar '11

01 Mar '11

Hello list, I'm trying to setup a pattern DB for Postfix, and I'm running into the a couple of problems. 1) One of the log messages I would like to catch is: connect from smtp.example.com[xxxx:xxxx:300:40c1::23] Now I'd like to catch both hostname and IP-address, and maybe I'd like to feed the IP address into some sort of program later. So I thought I'd better put these in individual variables. However, the matching rule <pattern>connect from @ESTRING:postfix.remotehost:[@@IPvANY:postfix.remoteip@]</pattern> doesn't work, because of the double "@@" - which is handled as an escaped "@", instead of two consecutive pattern parsers. So my first question is: how can I have two consecutive pattern parsers in a pattern? 2) Other messages say things like: 64A7F3001E7: from=<something(a)example.com> ..... The syslog-ng OSE admin guide tells me to use @QSTRING:<>@ to match the mail address; but this shows an error. Is @QSTRING:<&gt@ the correct way to proceed? (Or is this impossible with the current implementation?) 3) My third question boils down to: is it possible to correlate *one* single message into *two* separate trails? Would that just work by adding the same pattern to two different contexts? The question comes from Postfix using a bunch of small, interconnected programs, so a log trail of a single mail message will change characteristics during it's journey. For example, when a message comes in, smtpd will log: connect from smtp.example.com[xxxx:xxxx:300:40c1::23] ... it could then hand over the message with: 599903001E7: client=smtp.example.com[xxxx:xxxx:300:40c1::23] ... and then other programs continue the log trail useing this queue-ID 599903001E7 as a marker. So here are basically two events intertwined: a connection (that just says "connect..." and "... disconnect" and has a context-scope of "program); and a longer event that starts with the "connect..." and ends with delivery - or even forwarding - of the mail message - and probably has a "host" context-scope. 4) and finally: is there a good way to immediately end a certain context-scope? For example, after "disconnect from smtp.example.com[xxxx:xxxx:300:40c1::23]", the smtp phase is over, so there's no use keeping this context in memory anymore. Would adding something like "context-timeout=0" to the "disconnect" pattern work? Best regards, Valentijn

3 8