February 2011 - syslog-ng - lists.balabit.hu

[Bug 104] New: Add libtool option -module
by bugzilla＠bugzilla.balabit.com 23 Oct '11

23 Oct '11

https://bugzilla.balabit.com/show_bug.cgi?id=104 Summary: Add libtool option -module Product: syslog-ng Version: 3.2.x Platform: All OS/Version: All Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: vinschen(a)redhat.com Type of the Report: bug Estimated Hours: 0.0 Created an attachment (id=30) --> (https://bugzilla.balabit.com/attachment.cgi?id=30) Patch to add libtool option -module Libtool 2.2.6 and later provides an option -module which should be used when creating shared libraries which are supposed to be used as modules which get loaded via dlopen. See the "Building modules to dlopen" chapter in `info libtool'. For Cygwin the additional benefit is that the modules are not copied into a parallel "bin" directory at installation time, which is the default otherwise to accommodate the DLL search algorithm on Windows. The attached patch just adds the -module option to each of the Makefile.am files in the modules subdir. Corinna -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

2 4

[Bug 14] New: Maintain symlinks to current and previous log output files...
by bugzilla＠bugzilla.balabit.com 28 Sep '11

28 Sep '11

https://bugzilla.balabit.com/show_bug.cgi?id=14 Summary: Maintain symlinks to current and previous log output files... Product: syslog-ng Version: 2.0.x Platform: PC URL: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=218688 OS/Version: Linux Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: sasa(a)debian.org Type of the Report: enhancement Estimated Hours: 0.0 >From bugs.debian.org: I'm hoping that upstream might consider adding functionality to syslog-ng to allow it to track current and previous log files and maintain symlinks to them, as a user option. This replicates one of the options that I (and others) find useful in cronolog. This has been previously discussed on the syslog-ng mailing list, though apparently nothing fully came of it. The archive of that discussion is at: https://lists.balabit.hu/pipermail/syslog-ng/2002-June/003452.html Obviously, it's probably somewhat of a low priority, but it would be nice. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

escapes in SQL statements (was Re: dbtype in SQL logging)
by Hendrik Visage 21 Mar '11

21 Mar '11

Hi there, Okay, looks like I was looking/thinking about something else, so lets explain my problem. I want the pid inseted to be a "proper" number/integer, not a string/character, and the fun starts with several of my sources not having a pid entry, thus that field I would like to be filled with NULL values. However, it still gets escaped, and when it gets escaped, the NULL is not null anymore, but the string 'NULL' :( I was hoping that in some why I could coerce syslog-ng to understand that that column is a integer column, thus should not be escaped as it is currently done ;( I have the following destination: destination d_pgsql { sql(type(pgsql) host("127.0.0.1") username("logwriter") password("logwriter") port("5432") database("syslog") table("logs_${HOST}_${R_YEAR}${R_MONTH}${R_DAY}") #or whatever you want, example ${HOST}" for hosts, ${LEVEL}" for levels.. etc columns("datetime varchar(16)", "host varchar(32)", "program varchar(16)", "pid varchar(8) default null", "message varchar") values("$R_DATE", "$HOST", "$PROGRAM", "${PID:-NULL}", "$MSG") indexes("datetime", "host", "program", "pid", "message")); }; and the following log lines: Incoming log entry; line='<166>Feb 22 08:38:59 Vpxa: [2011-02-22 08:38:59.862 1B7A7B90 verbose \'App\'] [VpxaHalServices] HostChanged Event Fired, properties changed []' Running SQL query; query='INSERT INTO logs_somedevice_20110222 (datetime, host, program, pid, message) VALUES (\'Feb 22 11:39:27\', \'somedevice\', \'Vpxa\', \'NULL\', \'[2011-02-22 08:38:59.862 1B7A7B90 verbose \'\'App\'\'] [VpxaHalServices] HostChanged Event Fired, properties changed []\')'

5 8

[PATCH]: Experimental SMTP destination
by Gergely Nagy 11 Mar '11

11 Mar '11

Hi! I've had a little free time, and started toying with the idea of implementing an SMTP destination for syslog-ng. So far, it's progressing nicely, but there are a few shortcomings, as I will explain later. For anyone interested, it can be found on the 'work/modules/afsmtp' branch of my syslog-ng-3.3 tree: git://git.balabit.hu/algernon/syslog-ng-3.3.git Or, one can browse the branch on the web: http://git.balabit.hu/?p=algernon/syslog-ng-3.3.git;a=shortlog;h=work/modul… One will need the libesmtp library installed for the module to work. It should be available in most distributions, but if one doesn't have it, it can be downloaded from http://www.stafford.uklinux.net/libesmtp/ (although, as of this writing, the site seems to be down). The usage is something like this: destination d_smtp { smtp( host("localhost") port(25) mail_from("syslog-ng alert service <noreply(a)example.com>") rcpt_to("Admin #1 <admin(a)example.com>") rcpt_to("Admin #2 <admin2(a)example.com>") subject("[ALERT] $LEVEL condition in $PROGRAM!") header("X-Program" "$PROGRAM") header("X-Facility" "$FACILITY") body("Hi!\nThe syslog-ng alerting service thinks you might be interested in the following message:\n $MSG\n-- \nsyslog-ng\n") ); }; One can specify any number of recipients, and the mail will be sent to all of them. Custom headers can also be specified, but at this time, a selected few headers do not work as one would expect, due to libesmtp handling them specially. These are: Date, From, To, Cc, Bcc, Reply-To - trying to set either of these will result in unspecified behaviour (ranging from mangled headers to crashes, in case of Date). I will fix these at a later point. They will probably be need to set via their own statements (header_date, header_from, etc, or something along those lines). Specifying mail_from or subject multiple times will override any previous values. Adding the same custom header multiple times will only add the first (though, plans include changing that, so that one can include the same header multiple times, with different values - if I can find a use-case for that). There are also a few other minor issues here and there, but nothing major, as far as I could check. Further plans include authentication & starttls support. libesmtp supports both, I just didn't have time to write the neccessary glue code yet. -- |8]

8 26

[PATCH]: support setting a default suppress() globally
by Gergely Nagy 04 Mar '11

04 Mar '11

Hi! Attached is a patch that makes it possible to set a default value for suppress() in the global options block, which will then be inherited by all drivers that support this option. Of course, it can still be overridden on a per-destination basis. What this means is, that if one has a larger number of destinations that all need the same suppress settings, we'd write that like this before this patch: destination d_1 { file("/var/log/messages-1" suppress(10)); }; destination d_2 { file("/var/log/messages-2" suppress(10)); }; ... destination d_N { file("/var/log/messages-N" suppress(10)); }; destination d_XX { file("/var/log/messages-XX" suppress(5)); }; After the patch, this can be shortened to: options { suppress(10); }; destination d_1 { file("/var/log/messages-1"); }; ... destination d_N { file("/var/log/messages-N"); }; destination d_XX { file("/var/log/messages-XX" suppress(5)); }; Both result in exactly the same behaviour, but the second is a lot shorter, and easier to maintain, as long as the majority of destinations need the same suppress() setting. Hopefully people will find this useful. The patch is also available from the work/global-suppress branch of my tree at git://git.balabit.hu/algernon/syslog-ng-3.3.git -- |8]

2 4

Message correlation, everything needs an "@"? And a grep problem.
by Valentijn Sessink 04 Mar '11

04 Mar '11

Hello list, After playing with correlation for a while, here are a few remarks on it's workings. I have played with syslog-ng 3.2.2, built from source on an Ubuntu 10.04 machine. For some reason, the "@1", "@2"... etcetera specifiers needed to be all over the place to be able to see any variable at all. For example, the rule below describes the last message of a Postfix log trail. Note that even the context-id is specified as "${queueid}@1". All other references also have an "@" sign. Without the @, the variables seem just empty. <rule id='X' context-id="${queueid}@1" context-scope="host" provider="openoffice" class="maildelivery"> <patterns> <pattern>@ESTRING:queueid: @removed</pattern> </patterns> <actions> <action> <message> <values> <value name="MESSAGE">${queueid}@1 mail to=<${toaddress}@2> from=<${pffrom}@3> connection from=${pfremotehost}@5[${pfremoteip}@5] msgid=<${msgid}@4> orig_to=<${origto}@2> delay=${deliverystatus}@2</value> </values> </message> </action> </actions> </rule> Is this how it's meant to be? Then, when trying to use the "grep" template function in this message, I can't seem to find out how it works. Using the exact example from the documentation, $(grep ("${username}" == "root") ${auth_method}), I only get an error: Error parsing filter expression, syntax error, unexpected $undefined in <string> at line 1, column 3: (${ ^ Trying to alter the message, using "$something" == "value", doesn't seem to have an effect, i.e. I can't get "grep" to find any values. In fact, I'm not even sure if I can use the names from the (E/Q)STRING parsers right away, or if I need to have <value name="something"> tags before I can grep for them. In fact, most of the time, the grep I would like to use is something like: $(grep ( "$variable" != "" ) "$variable"). But whatever I try, no results from grep. Could anyone please clarify? Best regards, Valentijn

2 15

linking changes
by Balazs Scheidler 03 Mar '11

03 Mar '11

Hi, I've tried to put a closure on how syslog-ng 3.2 and above is being linked. There were numerous problems as it started to get into Linux distributions. This patch documents the intent with --enable-mixed-linking and how that behaves wrt the executables & shared libraries that syslog-ng installs. commit f2c7f3ea16d81e82b79582d3f2fe3882add4eb23 Author: Balazs Scheidler <bazsi(a)balabit.hu> Date: Wed Feb 23 13:01:02 2011 +0100 configure.in: reorganize LIBS variables so that unit tests/programs can be linked in mixed mode too Previously if --enable-mixed-linking was specified, unit tests and other tools (like pdbtool) couldn't be linked properly. The situation with --enable-mixed-linking is: * the primary aim here is to have a syslog-ng executable that can be started early in the boot process, and not a set of syslog-ng executables that can be distributed to different hosts that lack the required dependencies. * syslog-ng contains all static libs required by libsyslog-ng.so * therefore libsyslog-ng.so cannot be used outside the syslog-ng binary (except if the executable is linked against the required libs) * tools like pdbtool will be linked dynamically * unit tests will be linked dynamically too This patch cleans up and documents the purpose of various _DEPS_LIBS variables and updates Makefile.am files to use the updated names. A static GLib detection bug was also fixed. Reported-By: Arkadiusz Miśkiewicz <arekm(a)maven.pl> Signed-off-by: Balazs Scheidler <bazsi(a)balabit.hu> A related change is that the shared library libsyslog-ng-patterndb.so that was introduced in 3.2.2 will be dropped. Also, the main syslog-ng executable is not linked against libssl/libcrypto on purpose, it is only used by the afsocket module (which implements the tcp/syslog destinations). Could anyone who is in charge of maintaining syslog-ng binaries in various distributions give this a test so 3.2.3 will compile everywhere out-of-the-box? Thanks in advance, -- Bazsi

4 29

patternize and IPv6 hosts
by Valentijn Sessink 02 Mar '11

02 Mar '11

Hi, Just a small remark. If you run pdbtool with "patternize" on a log file with logs from an IPv6 host, pdbtool thinks that everything after the first ":" is the log message. For example, the log message: Feb 24 11:39:26 2a01:4f8:8a0:5141::3c2 named[31090]: lame server resolving .... ... will be patternized as if the logging host were "2a01" and if the pattern should be: <pattern>4f8:8a0:5141::3c2 named[31090]: lame server resolving ....</pattern> I searched around in the patternize code, but could not find the particular code where the host got cut off. However, for someone a bit at home in this code, I think the fix should be trivial. Best regards, Valentijn

3 6

afsql dbi initialization errors checks
by Hendrik Visage 02 Mar '11

02 Mar '11

Hi there, Busy compiling (more like porting) syslog-ng 3.2 to HP-UX 11.31, and I've been hitting a few snags that I since over come, but now I hit one where I see the source code is not checking whether the afsql got a proper DBI initialization, nor whether the actual driver is available before starting the db thread/etc. I've modified the lines 988-990 below, where they were just a simple "dbi_initialize();". On my setup, it "initialized", but with 0 drivers available :( (No, I DO have installed the Postgrsql dbi-driver, so yes I'll be debugging the DBI side now :() 986 if (!dbi_initialized) 987 { 988+ msg_error("HVT: dbi_initialization", 989= evt_tag_int("status",dbi_initialize("/opt/hvsyslog-ng3/lib/dbd") ), 990+ NULL); 991 dbi_initialized = TRUE; 992 } I also would like syslog-ng to a check at startup time and to rather exit with an error if the requested DBI driver is not available, than to wait till the messages starts to stream in and then to throw screens full of errors :( Hendrik

2 3

consecutive pattern parsers, and some other pattern matching questions
by Valentijn Sessink 01 Mar '11

01 Mar '11

Hello list, I'm trying to setup a pattern DB for Postfix, and I'm running into the a couple of problems. 1) One of the log messages I would like to catch is: connect from smtp.example.com[xxxx:xxxx:300:40c1::23] Now I'd like to catch both hostname and IP-address, and maybe I'd like to feed the IP address into some sort of program later. So I thought I'd better put these in individual variables. However, the matching rule <pattern>connect from @ESTRING:postfix.remotehost:[@@IPvANY:postfix.remoteip@]</pattern> doesn't work, because of the double "@@" - which is handled as an escaped "@", instead of two consecutive pattern parsers. So my first question is: how can I have two consecutive pattern parsers in a pattern? 2) Other messages say things like: 64A7F3001E7: from=<something(a)example.com> ..... The syslog-ng OSE admin guide tells me to use @QSTRING:<>@ to match the mail address; but this shows an error. Is @QSTRING:<&gt@ the correct way to proceed? (Or is this impossible with the current implementation?) 3) My third question boils down to: is it possible to correlate *one* single message into *two* separate trails? Would that just work by adding the same pattern to two different contexts? The question comes from Postfix using a bunch of small, interconnected programs, so a log trail of a single mail message will change characteristics during it's journey. For example, when a message comes in, smtpd will log: connect from smtp.example.com[xxxx:xxxx:300:40c1::23] ... it could then hand over the message with: 599903001E7: client=smtp.example.com[xxxx:xxxx:300:40c1::23] ... and then other programs continue the log trail useing this queue-ID 599903001E7 as a marker. So here are basically two events intertwined: a connection (that just says "connect..." and "... disconnect" and has a context-scope of "program); and a longer event that starts with the "connect..." and ends with delivery - or even forwarding - of the mail message - and probably has a "host" context-scope. 4) and finally: is there a good way to immediately end a certain context-scope? For example, after "disconnect from smtp.example.com[xxxx:xxxx:300:40c1::23]", the smtp phase is over, so there's no use keeping this context in memory anymore. Would adding something like "context-timeout=0" to the "disconnect" pattern work? Best regards, Valentijn

3 8