- syslog-ng - lists.balabit.hu

Question about PE edition disk-based buffering
by Clayton Dukes 04 Oct '11

04 Oct '11

Hi Folks, We have a team doing testing on the PE edition and they are finding that the timeout for the failover trigger takes 60 seconds: It was observed that it takes anytime between 10 to 60 seconds to detect connection is broken. We found out it by running syslog-ng in debug mode. I guess it relies on connection timeout. This 60 second is large interval compared to the default interval that we have (15 sec). My question: Is there a way to tune this parameter down to 15 seconds in the config? ______________________________________________________________ Clayton Dukes ______________________________________________________________

2 2

RFC: An option to not warn when a module's not found.
by Gergely Nagy 04 Oct '11

04 Oct '11

Hi! Recently I stumbled upon an issue while packaging syslog-ng 3.3 modules for Debian: for convenience's sake, whenever a module package gets installed, it places a config file in /etc/syslog-ng/conf.d/, and all files from that directory are included into the main configuration. This makes it possible that one does not have to add explicit @module options to his config when he wants to use a module he installed: it's already there. This works well, and there's no problem here. Except that when one removes a module package, Debian leaves the config file on the system. They get removed only when the package is purged. This is understandable, and there's good reasons behind the distinction between remove and purge. However, this results in syslog-ng trying to load a removed module whenever it starts up or is reloaded, which emits a warning, which will result in a mail from cron after logrotates, among other things. So I'd like to introduce a command-line option that would result in these warnings NOT getting printed. So if a module is not found, even if it was explicitly loaded, with the command-line option set, syslog-ng wouldn't print a warning. This would mean I can use this flag in the init script and in the logrotate config aswell: if the module is loaded, but not used, there's no harm in ignoring its inexistence. If it would be used, syslog-ng will throw an error later anyway. I'll prepare a patch that adds this sometime this week, but I wanted to give a heads up, and ask for opinions too. If there's anyone who sees a problem with this approach, or has a better idea, please let me know! -- |8]

1 0

Syslog-ng under cygwin
by pedro castro 04 Oct '11

04 Oct '11

Does anyone know the step by step to compile syslog-ng under cygwin? I know that inside cygwin there is a syslog-ng package but it does not have spoof-source option enable so as documentation indicate I need to compile. any ideas? Thanks a lot

2 1

Error in documentation on syslog() source port
by Patrick H. 03 Oct '11

03 Oct '11

The syslog-ng admin guide for 3.2 says the default port for syslog() sources is 514. This isnt the observed behavior. When a syslog() source is configured without specifying the port, it listens on 601

3 4

syslog-ng Digest, Vol 73, Issue 37
by Kristen Eisenberg 02 Oct '11

02 Oct '11

Someone must have used a so-called "smart" editor to edit this file, since it seems to have used fancy quotes on the console line. #destination console_all { file("/dev/tty12"); }; destination console_all { file(“/dev/console”); }; #destination loghost { udp("loghost" port(999)); }; Notice how they are different from those on the line above? Word processors often replace quotes with matching pairs like that automatically; only plain text editors should be used to edit config files like this. Check for this kind of thing with a hex dumper or 'cat -vet': $ cat frank #destination console_all { file("/dev/tty12"); }; destination console_all { file("/dev/console"); }; #destination loghost { udp("loghost" port(999)); }; Kristen Eisenberg Billige Flüge Marketing GmbH Emanuelstr. 3, 10317 Berlin Deutschland Telefon: +49 (33) 5310967 Email: utebachmeier at gmail.com Site: http://flug.airego.de- Billige Flüge vergleichen

1 0

[PATCH] afmongodb: Support system-installed libmongo-client at configure time.
by Gergely Nagy 30 Sep '11

30 Sep '11

Modify configure.in to recognise a suitable version (>= 0.1.0) of the libmongo-client library, and use it instead of the internal version if requested so. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- configure.in | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-) diff --git a/configure.in b/configure.in index e5bf174..e7038fb 100644 --- a/configure.in +++ b/configure.in @@ -28,6 +28,7 @@ IVYKIS_MIN_VERSION="0.18" JSON_C_MIN_VERSION="0.7" JSON_GLIB_MIN_VERSION="0.12" PCRE_MIN_VERSION="6.1" +LMC_MIN_VERSION="0.1.0" dnl *************************************************************************** dnl Initial setup @@ -743,12 +744,13 @@ if test "x$with_libmongo_client" = "xinternal"; then AC_MSG_WARN([Internal libmongo-client sources not found in modules/afmongodb/libmongo-client]) with_libmongo_client="no" fi -else - AC_MSG_WARN([Don't know how to locate system installed libmongo-client.]) - with_libmongo_client="no" +elif test "x$with_libmongo_client" = "xsystem"; then + PKG_CHECK_MODULES(LIBMONGO, libmongo-client >= $LMC_MIN_VERSION,,with_libmongo_client="no") fi - +if test "x$with_libmongo_client" = "xno"; then + enable_mongodb="no" +fi dnl *************************************************************************** dnl misc features to be enabled -- 1.7.2.5

4 14

syslog-ng 3.2.4 : How to handle syslogs with multiple lines
by Jay 30 Sep '11

30 Sep '11

Hello I am using syslog-ng on RHEL 5 Some syslogs from devices are wrapped. i.e. some syslogs contain multiple lines. In such cases, syslog-ng is treating each line as a separate syslog. I need all lines beloging to that syslog to be considered as single line. The \n\r (s) should be ignored. Adding flags (no-multi-line) option didn't help. Could someone help to address this issue, please ? Thanks

1 0

[Bug 136] New: duplicated logpaths gives no error
by bugzilla＠bugzilla.balabit.com 29 Sep '11

29 Sep '11

https://bugzilla.balabit.com/show_bug.cgi?id=136 Summary: duplicated logpaths gives no error Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: balazs.nemeth(a)balabit.com Type of the Report: --- Estimated Hours: 0.0 During my daily work I started to experience to receive every log message twice. It turned out that I made a silly copy&paste mistake but syslog-ng showed no warning or something like that at all. Steps to reproduce: 1) create a config like this # snippet start parser pattern_db { db_parser( file("/var/lib/misc/patterndb.xml") ); }; source s_netcat { tcp(ip(127.0.0.1) port(5514)); }; destination d_patterndb { file("/var/log/hosts/$HOST/$R_YEAR.$R_MONTH.$R_DAY/messages-${.classifier.class}"); }; log{ source(s_netcat); parser(pattern_db); destination(d_netcat); }; log{ source(s_netcat); parser(pattern_db); destination(d_netcat); }; # snippet end 2) send logs with netcat 3) check that every log message arrives twice to the given destination syslog-ng 3.2.4 Installer-Version: 3.2.4 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.2#master#ef7b91e4a1b1f9628c66138b4ae83de7e4c697c6 Compile-Date: Aug 16 2011 09:13:16 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-SSL: on Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: on Enable-Pacct: off -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

current 3.3 git
by Peter Czanik 29 Sep '11

29 Sep '11

Hello, There were many updates to 3.3 git this afternoon. I did a quick compile, packaging and functional test on my platforms (FreeBSD and openSUSE). I did not do many tests, but what I tried seems to work fine. >From the previous git version I found only one change: README was removed, so package lists had to be changed. You can find my current openSUSE syslog-ng package together with syslog-ng sources at https://build.opensuse.org/package/show?package=syslog-ng&project=home%3Acz… The file name says beta2, but it's actually a git snapshot from this afternoon. Have a nice weekend! Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

2 3

"Errors" in /var/log/messages
by Aldrich, Jamie S 28 Sep '11

28 Sep '11

I have setup syslog-ng to report application logs to a LogLogic device over UDP. Below is a excerpt of the syslog.conf file: source s_file_fs3 { file("/tmp/APPSRV_fs3_current.LOG" flags(no-parse) follow-freq(1)); }; destination d_messages{ udp("10.13.33.11" port(514)); }; log { source(s_file_fs3); destination(d_messages); }; I am seeing the following messages from our OSSEC server and /var/log/messages: OSSEC HIDS Notification. 2011 Sep 28 09:50:11 Received From: lxfwossecp3->/var/log/messages Rule: 1003 fired (level 13) -> "Non standard syslog message (size too large)." Portion of the log(s): Sep 28 09:50:10 lxfwossecp3 syslog-ng[3992]: Log statistics; processed='destination(d_mail)=2', processed='destination(d_spol)=0', processed='source(s_file_fs3)=1690', processed='global(payload_reallocs)=218', processed='source(s_sys)=325', processed='destination(d_mesg)=114', processed='global(msg_clones)=0', processed='src.internal(s_sys#2)=114', stamp='src.internal(s_sys#2)=1317220810', processed='destination(d_kern)=0', processed='destination(d_mlal)=0', processed='destination(d_cron)=200', dropped='dst.udp(d_messages#0,10.13.33.11:514)=0', processed='dst.udp(d_messages#0,10.13.33.11:514)=20670', stored='dst.udp(d_messages#0,10.13.33.11:514)=0', processed='global(sdata_updates)=0', processed='destination(d_auth)=9', processed='destination(d_boot)=0', processed='source(s_file_hr1)=2025', processed='center(received)=0', processed='source(s_file_fs1)=3896', processed='destination(d_messages)=20670', processed='source(s_file_hr2)=2743', processed='center(queued)=0', processed='source(s_file_fs2)=2662', processed='source(s_file_hr3)=7654' Do we have an issue? It looks like the log size is too large to process correctly, and it is kicking it out. What would we need to do to remedy the situation? Jamie

2 1

[Bug 135] New: pipe() source missing follow_freq() option
by bugzilla＠bugzilla.balabit.com 28 Sep '11

28 Sep '11

https://bugzilla.balabit.com/show_bug.cgi?id=135 Summary: pipe() source missing follow_freq() option Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.syslogng(a)feystorm.net Type of the Report: --- Estimated Hours: 0.0 According to the 3.2 documentation, the pipe() source is supposed to support the follow_freq() option. However attempting to use it results in an error: Error parsing affile, syntax error, unexpected KW_FOLLOW_FREQ, expecting ')' Looking at the current source tree in git, the option is not allowed in the configuration syntax. http://git.balabit.hu/?p=bazsi/syslog-ng-3.2.git;a=blob;f=modules/affile/af… -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

3 4

syslog-ng in books
by Peter Czanik 28 Sep '11

28 Sep '11

Hello, I just updated the list of book mentioning syslog-ng. For a list of recently found books check my blog at https://czanik.blogs.balabit.com/2011/09/syslog-ng-in-books/ If you learn about any other books not listed by us, and writing about syslog-ng in detail, please let us know, so we can update our list. Also, if you write an interesting blog, article, etc. about syslog-ng, please don't hesitate publishing at least a link about it here in the mailing list. For example there was a great blog series posted by Monitis at http://blog.monitis.com/index.php/tag/syslog-ng/ Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

Memory Growth in syslog-ng
by Pramod Pillai 28 Sep '11

28 Sep '11

Hi I am using syslog-ng 3.1.2 . I am observing that memory is growing approximately 64 MB, every 6 or 7 hours. The growth is consistent. Is this a known issue . Any patch available to fix it. sh-3.00# ./syslog-ng -V syslog-ng 3.1.2 Installer-Version: 3.1.2 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.1#master#8bf13c304b6ab5fc1a372b49d55c78370efe14ca Compile-Date: Oct 25 2010 23:56:18 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: on Enable-Sun-Door: on Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: off Enable-SSL: on Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: on

3 3

Reading from standard input
by Remy Zandwijk 28 Sep '11

28 Sep '11

Hi list, I use syslog-ng to create log files from within a script. The script sends the message through a pipe to syslog-ng, which I use as command line tool. In the syslog-ng.conf I created a source which reads the standard input. very simplified: Config: source src { pipe("/dev/fd/0"); }; destination dst { file("/tmp/test.log"); }; log { source(src); destination(dst); }; Called as: echo "test" | /usr/sbin/syslog-ng -f /tmp/syslog-ng.conf This works perfectly in 2.0.9. Now I am in the process of upgrading the system and syslog-ng 3.2.4. With this version of syslog-ng the config file above results in an error: Error opening file for reading; filename='/dev/fd/0', error='Text file busy (26)' Error initializing source driver; source='src', id='src#0' Error initializing message pipeline; I tried several things to get rid of this error, like replacing 'pipe' in 'file', but nothing seems to work. Any ideas? Regards, Remy

2 3

Re: [syslog-ng] log file size limit
by Sergei Zhirikov 28 Sep '11

28 Sep '11

On 2011-07-25 17:50, William Tambellini wrote: > Dear Syslog-ng team, > > We are currently evaluating syslog-ng in order to satisfy some requests from our customers regarding logs from our software. > One of their requests is to limit log file size using some kind of rotation algo : if size > 6Mo then rotate to another file. > We have found some options in file destination like fifo buffer control but nothing about files rotation. > Is there anything for that kind of feature ? > I have a patch that does something of that kind. When the log file grows above certain size syslog-ng switches to a new one and renames the old one. Further processing (such as compression) should be done externally. I posted that patch in this list quite some time ago, but as far as I can tell, it went unnoticed... -- Sergei.

4 7

Re: [syslog-ng] How to send tracebacks through syslog-ng? (Jakub Jankowski)
by Luis Pugoy 28 Sep '11

28 Sep '11

> > Message: 4 > Date: Sat, 24 Sep 2011 09:31:34 +0200 (CEST) > From: Jakub Jankowski <shasta(a)toxcorp.com> > Subject: Re: [syslog-ng] How to send tracebacks through syslog-ng? > To: Syslog-ng users' and developers' mailing list > <syslog-ng(a)lists.balabit.hu> > Message-ID: <alpine.LNX.2.00.1109240928240.15891(a)bluzg.slackware.pl> > Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed > > On 2011-09-24, Luis Pugoy wrote: > > > rewrite r_rewrite_subst { subst("a*", "?", value("MESSAGE") type("pcre") > > flags("utf8" "global")); }; > > > > I thought that it would replace 0 or more occurrences of 'a' with a > question > > mark. However, when the input is 'a', the output becomes 'a?'. It just > > appends a question mark at the end. 'aa' also becomes 'aa?'. What is > weirder > > still is that even if the input does not contain 'a', the question mark > is > > still appended. So '1' becomes '1?'. > > Well, does "1" match /a*/ ? :) > It doesn't make any difference if that's /a*/, /b*/ or /.*/ - this kind of > regexp matches everything, even an empty string. > > HTH > > -- > Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ > GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D Hello Jakub. Thanks for the reply. You're right in that "1" matches /a*/, but wouldn't the resulting replacement be "?" instead of "1?" ? Also, if I just use /a/ and use "a" as input, it does not seem to match. The text is still outputted as "a". >

2 2

[Bug 134] New: AIX syslog message not parsed correctly for hostname
by bugzilla＠bugzilla.balabit.com 28 Sep '11

28 Sep '11

https://bugzilla.balabit.com/show_bug.cgi?id=134 Summary: AIX syslog message not parsed correctly for hostname Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: ericduda(a)yahoo.com Type of the Report: bug Estimated Hours: 0.0 U 1.1.1.1:32768 -> 2.2.2.2:514 <38>Sep 22 10:1 1:56 Message fo rwarded from cd aix66: sshd[67 9960]: Accepted publickey for nagios from 1. 9.1.1 port 4 2096 ssh2 This is AIX forwarded. The hostname (in the example, cdaix66) is not parsed correctly, and put into macro $HOST What you end up with is "1.1.1.1", which is not right, it should parse out "cdaix66". I'm running syslog-ng on rhel5. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

[PATCH] afmongodb: Print the collection name upon init.
by Gergely Nagy 28 Sep '11

28 Sep '11

When initializing a mongodb destination, include the collection name in the logged message too. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afmongodb/afmongodb.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/modules/afmongodb/afmongodb.c b/modules/afmongodb/afmongodb.c index 0f01f05..4bf17a9 100644 --- a/modules/afmongodb/afmongodb.c +++ b/modules/afmongodb/afmongodb.c @@ -435,6 +435,7 @@ afmongodb_dd_init(LogPipe *s) evt_tag_str("host", self->host), evt_tag_int("port", self->port), evt_tag_str("database", self->db), + evt_tag_str("collection", self->coll), NULL); self->queue = log_dest_driver_acquire_queue(&self->super, afmongodb_dd_format_persist_name(self)); -- 1.7.5.4

2 1

[PATCH] configure.in: Fix --without-compile-date.
by Gergely Nagy 28 Sep '11

28 Sep '11

The option forgot to properly set the supplied value, and thus, always resulted in compile date being enabled. This patch corrects that. Reported-By: Peter Czanik <czanik(a)balabit.hu> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- configure.in | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/configure.in b/configure.in index 36da2c0..2efc397 100644 --- a/configure.in +++ b/configure.in @@ -187,7 +187,8 @@ AC_ARG_ENABLE(systemd, AC_ARG_WITH(compile-date, [ --without-compile-date Do not include the compile date in the binary] - ,, wcmp_date="1") + ,wcmp_date="${withval}", wcmp_date="yes") + if test "x$wcmp_date" != "xno"; then wcmp_date="1" else -- 1.7.5.4

2 1

[Bug 14] New: Maintain symlinks to current and previous log output files...
by bugzilla＠bugzilla.balabit.com 28 Sep '11

28 Sep '11

https://bugzilla.balabit.com/show_bug.cgi?id=14 Summary: Maintain symlinks to current and previous log output files... Product: syslog-ng Version: 2.0.x Platform: PC URL: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=218688 OS/Version: Linux Status: NEW Severity: minor Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: sasa(a)debian.org Type of the Report: enhancement Estimated Hours: 0.0 >From bugs.debian.org: I'm hoping that upstream might consider adding functionality to syslog-ng to allow it to track current and previous log files and maintain symlinks to them, as a user option. This replicates one of the options that I (and others) find useful in cronolog. This has been previously discussed on the syslog-ng mailing list, though apparently nothing fully came of it. The archive of that discussion is at: https://lists.balabit.hu/pipermail/syslog-ng/2002-June/003452.html Obviously, it's probably somewhat of a low priority, but it would be nice. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

[PATCH] gprocess: Implement stricter CAP_SYSLOG checking.
by Gergely Nagy 27 Sep '11

27 Sep '11

Some distributions ship libcap with headers generated from the kernel sources, instead of the headers of libcap itself, resulting in the headers and the library diagreeing about what capabilities are supported. For this reason, before deciding whether CAP_SYSLOG is supported, verify that libcap can actually parse it, and fall back to CAP_SYS_ADMIN if it can't. The benefit of this is that even if ran on such a system, syslog-ng will still continue to work (albeit triggering a kernel warning) instead of terminating. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/gprocess.c | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diff --git a/lib/gprocess.c b/lib/gprocess.c index e9315d9..b0925fb 100644 --- a/lib/gprocess.c +++ b/lib/gprocess.c @@ -328,6 +328,14 @@ g_process_check_cap_syslog(void) if (ret == -1) return FALSE; + ret = cap_from_name("cap_syslog", NULL); + if (ret == -1) + { + fprintf (stderr, "CAP_SYSLOG seems to be supported by the system, but " + "libcap can't parse it. Falling back to CAP_SYS_ADMIN!\n"); + return FALSE; + } + have_capsyslog = TRUE; return TRUE; } -- 1.7.6.3

2 1

[PATCH] afmongodb, afsql: Fix a race condition in the queue handling.
by Gergely Nagy 27 Sep '11

27 Sep '11

On occassions where we received a message, woke up the worker thread, then received two two others before the worker finished, we had a high chance of triggering an assertion, because we ended up calling the insert before the parallel_push stuff managed to wake up the thread and clear the callbacks. Now we explicitly clear parallel_push before popping the queue head, for the simple reason that we do not need it at that point anymore. Many thanks to Balazs Scheidler <bazsi(a)balabit.hu> for pointing out what happens. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afmongodb/afmongodb.c | 1 + modules/afsql/afsql.c | 1 + 2 files changed, 2 insertions(+), 0 deletions(-) diff --git a/modules/afmongodb/afmongodb.c b/modules/afmongodb/afmongodb.c index 0f01f05..b2bd615 100644 --- a/modules/afmongodb/afmongodb.c +++ b/modules/afmongodb/afmongodb.c @@ -263,6 +263,7 @@ afmongodb_worker_insert (MongoDBDestDriver *self) afmongodb_dd_connect(self, TRUE); g_mutex_lock(self->queue_mutex); + log_queue_reset_parallel_push(self->queue); success = log_queue_pop_head(self->queue, &msg, &path_options, FALSE, FALSE); g_mutex_unlock(self->queue_mutex); if (!success) diff --git a/modules/afsql/afsql.c b/modules/afsql/afsql.c index 20db131..6fbd656 100644 --- a/modules/afsql/afsql.c +++ b/modules/afsql/afsql.c @@ -682,6 +682,7 @@ afsql_dd_insert_db(AFSqlDestDriver *self) else { g_mutex_lock(self->db_thread_mutex); + log_queue_reset_parallel_push(self->queue); success = log_queue_pop_head(self->queue, &msg, &path_options, (self->flags & AFSQL_DDF_EXPLICIT_COMMITS), FALSE); g_mutex_unlock(self->db_thread_mutex); if (!success) -- 1.7.6.3

2 1

How to send tracebacks through syslog-ng?
by Luis Pugoy 27 Sep '11

27 Sep '11

Hello, We have a Python process that logs into a file. Included in the file are tracebacks. We would like to send these log entries through syslog-ng. To solve the problem of newlines in tracebacks, our plan was to modify the Python log formatter to replace all newlines with the Unicode character for line separator, U+2028. Syslog-ng then tails the log file and sends the entries to the central logging server through tcp, which is setup to replace the Unicode character with a newline and record it in a file. The problem with this approach is that the Unicode character does not seem to be persisted in the file. A lot of rewrite rules were tried to replace the Unicode character. These are: rewrite r_newlines { subst("\p{Zl}", " test ", value("MESSAGE") type("pcre") flags("utf8" "global")); }; rewrite r_newlines { subst("\x{2028}", " test ", value("MESSAGE") type("pcre") flags("utf8" "global")); }; rewrite r_newlines { subst("\u2028", " test ", value("MESSAGE") flags("global")); }; but none of them worked. The file source was also set to have UTF-8 encoding, as well as the tcp source (from the view of the central logging server) but these didn't work either. The problem seems to be either the rewrite rule or the Unicode character is not persisted in the log file. We also tried logging to syslog directly from Python (through /dev/log) but that also failed. Replacing newlines with a string such as " [NEWLINE] " and using syslog-ng to rewrite that to newlines worked, so it seems to be a problem with the Unicode character. This solution is a last resort though since it could be problematic. Thanks in advance.

3 3

Load balancing in syslog-ng(verison 3.0)
by Jain, Vaibhav (GE Healthcare) 27 Sep '11

27 Sep '11

Hi, In my syslog-ng server configuration file I want to divide the messages in 2 different pipes to share the load. Please let me know how to do this ? Thanks Vaibhav

7 9

[Bug 133] New: syslog-ng-3.3. 0beta2 stops to receive messages from systemd activated /dev/log
by bugzilla＠bugzilla.balabit.com 27 Sep '11

27 Sep '11

https://bugzilla.balabit.com/show_bug.cgi?id=133 Summary: syslog-ng-3.3.0beta2 stops to receive messages from systemd activated /dev/log Product: syslog-ng Version: 3.2.x Platform: PC URL: https://bugzilla.novell.com/show_bug.cgi?id=719102 OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: mt(a)suse.com Type of the Report: --- Estimated Hours: 0.0 Even all looks fine and syslog-ng-3.3.0beta2 starts to work correctly: > systemctl status syslog.service syslog.service - System Logging Service Loaded: loaded (/lib/systemd/system/syslog.service; enabled) Active: active (running) since Tue, 20 Sep 2011 14:08:31 +0200; 19h ago Process: 2317 ExecReload=/sbin/killproc -p /var/run/syslog-ng.pid -HUP /sbin/syslog-ng (code=exited, status=0/SUCCESS) Process: 740 ExecStart=/sbin/syslog-ng (code=exited, status=0/SUCCESS) Process: 738 ExecStartPre=/var/run/syslog-ng/addsockets (code=exited, status=0/SUCCESS) Process: 735 ExecStartPre=/bin/systemctl stop systemd-kmsg-syslogd.service (code=exited, status=0/SUCCESS) Main PID: 763 (syslog-ng) CGroup: name=systemd:/system/syslog.service ├ 762 supervising syslog-ng └ 763 /sbin/syslog-ng > lsof | grep /dev/log systemd 1 root 25u unix 0xffff8802211690c0 0t0 4225 /dev/log syslog-ng 762 root 3u unix 0xffff8802211690c0 0t0 4225 /dev/log syslog-ng 763 root 3u unix 0xffff8802211690c0 0t0 4225 /dev/log > ps ax | grep syslog 762 ? S 0:00 supervising syslog-ng 763 ? Ss 0:00 /sbin/syslog-ng > ls -l /proc/763/fd/ total 0 lr-x------ 1 root root 64 Sep 20 14:10 0 -> /dev/null l-wx------ 1 root root 64 Sep 20 14:10 1 -> /dev/null lrwx------ 1 root root 64 Sep 20 14:10 10 -> anon_inode:[eventfd] lrwx------ 1 root root 64 Sep 20 14:10 11 -> anon_inode:[eventfd] lrwx------ 1 root root 64 Sep 20 14:10 12 -> anon_inode:[eventfd] lrwx------ 1 root root 64 Sep 20 14:10 13 -> anon_inode:[eventfd] lrwx------ 1 root root 64 Sep 20 14:10 19 -> /dev/xconsole l-wx------ 1 root root 64 Sep 20 14:10 2 -> /dev/null lrwx------ 1 root root 64 Sep 20 14:10 3 -> socket:[4225] lrwx------ 1 root root 64 Sep 20 14:10 4 -> anon_inode:[eventpoll] l-wx------ 1 root root 64 Sep 20 14:10 5 -> pipe:[8583] lrwx------ 1 root root 64 Sep 20 14:10 6 -> socket:[8602] lrwx------ 1 root root 64 Sep 20 14:10 7 -> anon_inode:[eventfd] lrwx------ 1 root root 64 Sep 20 14:10 8 -> /var/lib/syslog-ng/syslog-ng.persist lr-x------ 1 root root 64 Sep 20 14:10 9 -> /proc/kmsg it stops receiving at the socket after some time (hours) and tries to send something to /dev/log gets blocked: > strace logger =========================================== ... socket(PF_FILE, SOCK_DGRAM|SOCK_CLOEXEC, 0) = 1 connect(1, {sa_family=AF_FILE, path="/dev/log"}, 110) = 0 sendto(1, "<13>Sep 21 09:26:30 logger: ===="..., 71, MSG_NOSIGNAL, NULL, 0 --- blocks --- Syslog-ng shows no reaction on the data send to the systemd /dev/log socket, it just waits in epoll: > strace -f -s128 -p 763 Process 763 attached - interrupt to quit clock_gettime(CLOCK_MONOTONIC_RAW, {69938, 341467128}) = 0 epoll_wait(4, {}, 9, 21975) = 0 clock_gettime(CLOCK_MONOTONIC_RAW, {69960, 335242759}) = 0 epoll_wait(4, ^C and logs its statistics and -- MARK -- messages from time to time to /var/log/messages. After a "systemctl restart syslog.service", it starts reading messages from /dev/log again -- still the using the same socket node 4225 from systemd. I don't think some of our patches are responsible for this problem as all are from git and IMO unrelated. See also our bug reference in Url. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5