September 2011 - syslog-ng - lists.balabit.hu

Re: [syslog-ng] How to send tracebacks through syslog-ng? (Jakub Jankowski)
by Luis Pugoy 28 Sep '11

28 Sep '11

> > Message: 4 > Date: Sat, 24 Sep 2011 09:31:34 +0200 (CEST) > From: Jakub Jankowski <shasta(a)toxcorp.com> > Subject: Re: [syslog-ng] How to send tracebacks through syslog-ng? > To: Syslog-ng users' and developers' mailing list > <syslog-ng(a)lists.balabit.hu> > Message-ID: <alpine.LNX.2.00.1109240928240.15891(a)bluzg.slackware.pl> > Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed > > On 2011-09-24, Luis Pugoy wrote: > > > rewrite r_rewrite_subst { subst("a*", "?", value("MESSAGE") type("pcre") > > flags("utf8" "global")); }; > > > > I thought that it would replace 0 or more occurrences of 'a' with a > question > > mark. However, when the input is 'a', the output becomes 'a?'. It just > > appends a question mark at the end. 'aa' also becomes 'aa?'. What is > weirder > > still is that even if the input does not contain 'a', the question mark > is > > still appended. So '1' becomes '1?'. > > Well, does "1" match /a*/ ? :) > It doesn't make any difference if that's /a*/, /b*/ or /.*/ - this kind of > regexp matches everything, even an empty string. > > HTH > > -- > Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ > GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D Hello Jakub. Thanks for the reply. You're right in that "1" matches /a*/, but wouldn't the resulting replacement be "?" instead of "1?" ? Also, if I just use /a/ and use "a" as input, it does not seem to match. The text is still outputted as "a". >

2 2

[Bug 134] New: AIX syslog message not parsed correctly for hostname
by bugzilla＠bugzilla.balabit.com 28 Sep '11

28 Sep '11

https://bugzilla.balabit.com/show_bug.cgi?id=134 Summary: AIX syslog message not parsed correctly for hostname Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: ericduda(a)yahoo.com Type of the Report: bug Estimated Hours: 0.0 U 1.1.1.1:32768 -> 2.2.2.2:514 <38>Sep 22 10:1 1:56 Message fo rwarded from cd aix66: sshd[67 9960]: Accepted publickey for nagios from 1. 9.1.1 port 4 2096 ssh2 This is AIX forwarded. The hostname (in the example, cdaix66) is not parsed correctly, and put into macro $HOST What you end up with is "1.1.1.1", which is not right, it should parse out "cdaix66". I'm running syslog-ng on rhel5. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

[PATCH] afmongodb: Print the collection name upon init.
by Gergely Nagy 28 Sep '11

28 Sep '11

When initializing a mongodb destination, include the collection name in the logged message too. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afmongodb/afmongodb.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/modules/afmongodb/afmongodb.c b/modules/afmongodb/afmongodb.c index 0f01f05..4bf17a9 100644 --- a/modules/afmongodb/afmongodb.c +++ b/modules/afmongodb/afmongodb.c @@ -435,6 +435,7 @@ afmongodb_dd_init(LogPipe *s) evt_tag_str("host", self->host), evt_tag_int("port", self->port), evt_tag_str("database", self->db), + evt_tag_str("collection", self->coll), NULL); self->queue = log_dest_driver_acquire_queue(&self->super, afmongodb_dd_format_persist_name(self)); -- 1.7.5.4

2 1

[PATCH] configure.in: Fix --without-compile-date.
by Gergely Nagy 28 Sep '11

28 Sep '11

The option forgot to properly set the supplied value, and thus, always resulted in compile date being enabled. This patch corrects that. Reported-By: Peter Czanik <czanik(a)balabit.hu> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- configure.in | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/configure.in b/configure.in index 36da2c0..2efc397 100644 --- a/configure.in +++ b/configure.in @@ -187,7 +187,8 @@ AC_ARG_ENABLE(systemd, AC_ARG_WITH(compile-date, [ --without-compile-date Do not include the compile date in the binary] - ,, wcmp_date="1") + ,wcmp_date="${withval}", wcmp_date="yes") + if test "x$wcmp_date" != "xno"; then wcmp_date="1" else -- 1.7.5.4

2 1

[Bug 14] New: Maintain symlinks to current and previous log output files...
by bugzilla＠bugzilla.balabit.com 28 Sep '11

28 Sep '11

https://bugzilla.balabit.com/show_bug.cgi?id=14 Summary: Maintain symlinks to current and previous log output files... Product: syslog-ng Version: 2.0.x Platform: PC URL: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=218688 OS/Version: Linux Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: sasa(a)debian.org Type of the Report: enhancement Estimated Hours: 0.0 >From bugs.debian.org: I'm hoping that upstream might consider adding functionality to syslog-ng to allow it to track current and previous log files and maintain symlinks to them, as a user option. This replicates one of the options that I (and others) find useful in cronolog. This has been previously discussed on the syslog-ng mailing list, though apparently nothing fully came of it. The archive of that discussion is at: https://lists.balabit.hu/pipermail/syslog-ng/2002-June/003452.html Obviously, it's probably somewhat of a low priority, but it would be nice. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

[PATCH] gprocess: Implement stricter CAP_SYSLOG checking.
by Gergely Nagy 27 Sep '11

27 Sep '11

Some distributions ship libcap with headers generated from the kernel sources, instead of the headers of libcap itself, resulting in the headers and the library diagreeing about what capabilities are supported. For this reason, before deciding whether CAP_SYSLOG is supported, verify that libcap can actually parse it, and fall back to CAP_SYS_ADMIN if it can't. The benefit of this is that even if ran on such a system, syslog-ng will still continue to work (albeit triggering a kernel warning) instead of terminating. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/gprocess.c | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/lib/gprocess.c b/lib/gprocess.c index e9315d9..b0925fb 100644 --- a/lib/gprocess.c +++ b/lib/gprocess.c @@ -328,6 +328,14 @@ g_process_check_cap_syslog(void) if (ret == -1) return FALSE; + ret = cap_from_name("cap_syslog", NULL); + if (ret == -1) + { + fprintf (stderr, "CAP_SYSLOG seems to be supported by the system, but " + "libcap can't parse it. Falling back to CAP_SYS_ADMIN!\n"); + return FALSE; + } + have_capsyslog = TRUE; return TRUE; } -- 1.7.6.3

2 1

[PATCH] afmongodb, afsql: Fix a race condition in the queue handling.
by Gergely Nagy 27 Sep '11

27 Sep '11

On occassions where we received a message, woke up the worker thread, then received two two others before the worker finished, we had a high chance of triggering an assertion, because we ended up calling the insert before the parallel_push stuff managed to wake up the thread and clear the callbacks. Now we explicitly clear parallel_push before popping the queue head, for the simple reason that we do not need it at that point anymore. Many thanks to Balazs Scheidler <bazsi(a)balabit.hu> for pointing out what happens. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afmongodb/afmongodb.c | 1 + modules/afsql/afsql.c | 1 + 2 files changed, 2 insertions(+), 0 deletions(-) diff --git a/modules/afmongodb/afmongodb.c b/modules/afmongodb/afmongodb.c index 0f01f05..b2bd615 100644 --- a/modules/afmongodb/afmongodb.c +++ b/modules/afmongodb/afmongodb.c @@ -263,6 +263,7 @@ afmongodb_worker_insert (MongoDBDestDriver *self) afmongodb_dd_connect(self, TRUE); g_mutex_lock(self->queue_mutex); + log_queue_reset_parallel_push(self->queue); success = log_queue_pop_head(self->queue, &msg, &path_options, FALSE, FALSE); g_mutex_unlock(self->queue_mutex); if (!success) diff --git a/modules/afsql/afsql.c b/modules/afsql/afsql.c index 20db131..6fbd656 100644 --- a/modules/afsql/afsql.c +++ b/modules/afsql/afsql.c @@ -682,6 +682,7 @@ afsql_dd_insert_db(AFSqlDestDriver *self) else { g_mutex_lock(self->db_thread_mutex); + log_queue_reset_parallel_push(self->queue); success = log_queue_pop_head(self->queue, &msg, &path_options, (self->flags & AFSQL_DDF_EXPLICIT_COMMITS), FALSE); g_mutex_unlock(self->db_thread_mutex); if (!success) -- 1.7.6.3

2 1

How to send tracebacks through syslog-ng?
by Luis Pugoy 27 Sep '11

27 Sep '11

Hello, We have a Python process that logs into a file. Included in the file are tracebacks. We would like to send these log entries through syslog-ng. To solve the problem of newlines in tracebacks, our plan was to modify the Python log formatter to replace all newlines with the Unicode character for line separator, U+2028. Syslog-ng then tails the log file and sends the entries to the central logging server through tcp, which is setup to replace the Unicode character with a newline and record it in a file. The problem with this approach is that the Unicode character does not seem to be persisted in the file. A lot of rewrite rules were tried to replace the Unicode character. These are: rewrite r_newlines { subst("\p{Zl}", " test ", value("MESSAGE") type("pcre") flags("utf8" "global")); }; rewrite r_newlines { subst("\x{2028}", " test ", value("MESSAGE") type("pcre") flags("utf8" "global")); }; rewrite r_newlines { subst("\u2028", " test ", value("MESSAGE") flags("global")); }; but none of them worked. The file source was also set to have UTF-8 encoding, as well as the tcp source (from the view of the central logging server) but these didn't work either. The problem seems to be either the rewrite rule or the Unicode character is not persisted in the log file. We also tried logging to syslog directly from Python (through /dev/log) but that also failed. Replacing newlines with a string such as " [NEWLINE] " and using syslog-ng to rewrite that to newlines worked, so it seems to be a problem with the Unicode character. This solution is a last resort though since it could be problematic. Thanks in advance.

3 3

Load balancing in syslog-ng(verison 3.0)
by Jain, Vaibhav (GE Healthcare) 27 Sep '11

27 Sep '11

Hi, In my syslog-ng server configuration file I want to divide the messages in 2 different pipes to share the load. Please let me know how to do this ? Thanks Vaibhav

7 9

[Bug 133] New: syslog-ng-3.3. 0beta2 stops to receive messages from systemd activated /dev/log
by bugzilla＠bugzilla.balabit.com 27 Sep '11

27 Sep '11

https://bugzilla.balabit.com/show_bug.cgi?id=133 Summary: syslog-ng-3.3.0beta2 stops to receive messages from systemd activated /dev/log Product: syslog-ng Version: 3.2.x Platform: PC URL: https://bugzilla.novell.com/show_bug.cgi?id=719102 OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: mt(a)suse.com Type of the Report: --- Estimated Hours: 0.0 Even all looks fine and syslog-ng-3.3.0beta2 starts to work correctly: > systemctl status syslog.service syslog.service - System Logging Service Loaded: loaded (/lib/systemd/system/syslog.service; enabled) Active: active (running) since Tue, 20 Sep 2011 14:08:31 +0200; 19h ago Process: 2317 ExecReload=/sbin/killproc -p /var/run/syslog-ng.pid -HUP /sbin/syslog-ng (code=exited, status=0/SUCCESS) Process: 740 ExecStart=/sbin/syslog-ng (code=exited, status=0/SUCCESS) Process: 738 ExecStartPre=/var/run/syslog-ng/addsockets (code=exited, status=0/SUCCESS) Process: 735 ExecStartPre=/bin/systemctl stop systemd-kmsg-syslogd.service (code=exited, status=0/SUCCESS) Main PID: 763 (syslog-ng) CGroup: name=systemd:/system/syslog.service ├ 762 supervising syslog-ng └ 763 /sbin/syslog-ng > lsof | grep /dev/log systemd 1 root 25u unix 0xffff8802211690c0 0t0 4225 /dev/log syslog-ng 762 root 3u unix 0xffff8802211690c0 0t0 4225 /dev/log syslog-ng 763 root 3u unix 0xffff8802211690c0 0t0 4225 /dev/log > ps ax | grep syslog 762 ? S 0:00 supervising syslog-ng 763 ? Ss 0:00 /sbin/syslog-ng > ls -l /proc/763/fd/ total 0 lr-x------ 1 root root 64 Sep 20 14:10 0 -> /dev/null l-wx------ 1 root root 64 Sep 20 14:10 1 -> /dev/null lrwx------ 1 root root 64 Sep 20 14:10 10 -> anon_inode:[eventfd] lrwx------ 1 root root 64 Sep 20 14:10 11 -> anon_inode:[eventfd] lrwx------ 1 root root 64 Sep 20 14:10 12 -> anon_inode:[eventfd] lrwx------ 1 root root 64 Sep 20 14:10 13 -> anon_inode:[eventfd] lrwx------ 1 root root 64 Sep 20 14:10 19 -> /dev/xconsole l-wx------ 1 root root 64 Sep 20 14:10 2 -> /dev/null lrwx------ 1 root root 64 Sep 20 14:10 3 -> socket:[4225] lrwx------ 1 root root 64 Sep 20 14:10 4 -> anon_inode:[eventpoll] l-wx------ 1 root root 64 Sep 20 14:10 5 -> pipe:[8583] lrwx------ 1 root root 64 Sep 20 14:10 6 -> socket:[8602] lrwx------ 1 root root 64 Sep 20 14:10 7 -> anon_inode:[eventfd] lrwx------ 1 root root 64 Sep 20 14:10 8 -> /var/lib/syslog-ng/syslog-ng.persist lr-x------ 1 root root 64 Sep 20 14:10 9 -> /proc/kmsg it stops receiving at the socket after some time (hours) and tries to send something to /dev/log gets blocked: > strace logger =========================================== ... socket(PF_FILE, SOCK_DGRAM|SOCK_CLOEXEC, 0) = 1 connect(1, {sa_family=AF_FILE, path="/dev/log"}, 110) = 0 sendto(1, "<13>Sep 21 09:26:30 logger: ===="..., 71, MSG_NOSIGNAL, NULL, 0 --- blocks --- Syslog-ng shows no reaction on the data send to the systemd /dev/log socket, it just waits in epoll: > strace -f -s128 -p 763 Process 763 attached - interrupt to quit clock_gettime(CLOCK_MONOTONIC_RAW, {69938, 341467128}) = 0 epoll_wait(4, {}, 9, 21975) = 0 clock_gettime(CLOCK_MONOTONIC_RAW, {69960, 335242759}) = 0 epoll_wait(4, ^C and logs its statistics and -- MARK -- messages from time to time to /var/log/messages. After a "systemctl restart syslog.service", it starts reading messages from /dev/log again -- still the using the same socket node 4225 from systemd. I don't think some of our patches are responsible for this problem as all are from git and IMO unrelated. See also our bug reference in Url. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5