- syslog-ng - lists.balabit.hu

Help getting my rewrite to work
by Dave Vernon 19 Jul '13

19 Jul '13

Good morning all, I'm having difficulties getting my rewrite statement to work properly. I've been spending more hours than I care to admit pouring through docs, examples, etc and just can't get it. (I really thought I was on the right track from a post in the May archives) In short, I am using syslog-ng 3.4.2 to receive my Windows event logs. There are several common Windows events that includes a detailed description of the event EVERY TIME that I am trying to drop before writing to my syslog server, leaving just the meat. I originally wanted to start my testing by replacing a single word (i.e. the word "interactive" so that I could make sure the re-write rule was working before I tweaked my regex.) However, whenever I add my rule it's not logging any event that matches at all, and I'm not sure why. I'm doing a tail on the log file and it just doesn't write (literally) anything at all. My examples... source s_syslog { udp(ip(0.0.0.0) port(514)); }; destination d_allwindows { file("/syslog/log/windows/hosts/$HOST/$R_YEAR/$R_MONTH/$R_DAY/$HOST$YEAR$MONTH$DAY" create_dirs(yes)); }; rewrite r_rewrite_4624_2 { subst("interactive", "QuackApple", value("MSGONLY"));}; ("interactive" is contained in the message, so it was an easy one to pickout for testing, and I used the string "QuackApple" as the replacement as I know it won't naturally appear, so I can search on it easily) log { source(s_syslog); filter(f_allwindows); rewrite(r_rewrite_4624_2); destination(d_allwindows); }; With these rules in place, the log is actually dropped, not re-written. If I change my rewrite rule to be something not found (i.e. replace "interactive" with "ZZowkr" then the events start showing up in the tail again) Thanks! Dave Vernon

2 2

[Bug 239] New: syslog-ng refuses to allow 'Common Name' CN wildcards
by bugzilla＠bugzilla.balabit.com 11 Jul '13

11 Jul '13

https://bugzilla.balabit.com/show_bug.cgi?id=239 Summary: syslog-ng refuses to allow 'Common Name' CN wildcards Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Windows Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: shawn.starr(a)statpro.com Type of the Report: bug Estimated Hours: 0.0 Distribution package version: 3.3.4.dfsg-2ubuntu1 (3.3.4) When attempting to use the following configuration: source system_stuff { system(); }; destination dest_kern { tcp("192.168.70.4" port(514) tls( #peer-verify(optional-untrusted) peer-verify(required-trusted) cipher_suite("AES256-SHA") trusted_dn("*, O=MyCompany Name, L=Toronto, ST=Ontario, C=CA") cert_file("/etc/syslog-ng/certs/genericServer.crt") ca_dir("/etc/syslog-ng/ca")) ); }; log { source(system_stuff); destination(dest_kern); }; Jul 9 12:20:01 testad syslog-ng[12607]: Certificate subject does not match configured hostname; hostname='192.168.70.4', certificate='*.dev.company.com' When trying to use trusted_tn("CN=*.dev.company.com, O=MyCompany Name, L=Toronto, ST=Ontario, C=CA) it then shows: Jul 9 11:21:20 testad syslog-ng[12473]: Certificate valid, but DN constraints were not met, rejecting; If I read this right, CN is provided (as per default CA policy) but we should be able to match hosts to the wildcard. Similar to what rsyslog has: http://www.rsyslog.com/doc/tls_cert_server.html I don't know if this also happens in 3.5.x but I can test this on my Fedora systems at home -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

Re: [syslog-ng] syslog-ng Digest, Vol 99, Issue 9
by Yarick Tsagoyko 11 Jul '13

11 Jul '13

Dear Satish, multiline is supported in open source version of syslog-ng, as bazsi wrote. To glue the lines together, into one line you can do one of two things. First, configure syslog-ng not to break messages into separate lines, by disabling native multiline support. Second, define a pattern that identifies the beginning of a new line. I encourage you to read the manual prior to engaging entire list to solve your issue, which is clearly not an issue with the way code is written. Thanks. Y. This is my source declaration and i have put flags which you have mentioned. source s_tomcat { syslog( transport("udp") flags(indent-multi-line)); }; I got following error when i am trying to put flags Error parsing afsocket, Unknown flag indent-multi-line in /usr/local/syslog-ng-3.4.2/etc/syslog-ng.conf at line 54, column 33: syslog( transport("udp") flags(indent-multi-line) ); On Thu, Jul 11, 2013 at 10:22 AM, <syslog-ng-request(a)lists.balabit.hu>wrote: > Send syslog-ng mailing list submissions to > syslog-ng(a)lists.balabit.hu > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.balabit.hu/mailman/listinfo/syslog-ng > or, via email, send a message with subject or body 'help' to > syslog-ng-request(a)lists.balabit.hu > > You can reach the person managing the list at > syslog-ng-owner(a)lists.balabit.hu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of syslog-ng digest..." > > > Today's Topics: > > 1. Re: Multi-line support issue (Balazs Scheidler) > 2. Re: Multi-line support issue (Satish Patel) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 11 Jul 2013 13:53:56 +0200 > From: Balazs Scheidler <bazsi(a)balabit.hu> > Subject: Re: [syslog-ng] Multi-line support issue > To: Syslog-ng users' and developers' mailing list > <syslog-ng(a)lists.balabit.hu> > Message-ID: <1373543636.3171.17.camel@bzorp> > Content-Type: text/plain; charset="UTF-8" > > > I can't see the source declaration, it must be something along the lines > of: > > source s_tomcat { > file("/var/log/tomcat/xxx.log" flags(indent-multi-line)); > }; > > On Wed, 2013-07-10 at 12:54 -0400, Satish Patel wrote: > > Hi Balazs, > > > > > > what is your thought about my config? did you see? > > > > > > > > On Mon, Jul 8, 2013 at 12:30 PM, Satish Patel <satish.txt(a)gmail.com> > > wrote: > > This is what i have configured and no luck with it.. can you > > suggest what i am missing? > > > > destination d02_tc74_log > > { file("/logs/server1/tomcat7.4/catalina_$YEAR$MONTH$DAY.log" > > template("$(indent-multi-line ${MESSAGE})\n") > > template(t_tomcatlog) owner("root") group("root") perm(0644) > > dir_perm(0755) create_dirs(yes)); }; > > filter server1 { host("server1.example.com") }; > > log { > > source (s_tomcat); > > filter (server1); > > filter (tomcat7_4); > > destination (d02_tc74_log); > > }; > > > > > > > > > > On Mon, Jul 8, 2013 at 12:08 PM, Satish Patel > > <satish.txt(a)gmail.com> wrote: > > How do i use indented-multi-line ? I meant where do i > > configure it? I tried but my syslog-ng doesn't > > recognizing this option i have syslog-ng 3.3.7 could > > you give me example where and how do i check whether > > it is supported or not > > > > > > > > On Sat, Jul 6, 2013 at 2:12 AM, Balazs Scheidler > > <bazsi77(a)gmail.com> wrote: > > This looks.like the format that should be > > supported by indented-multi-line > > > > On Jul 5, 2013 9:33 PM, "Satish Patel" > > <satish.txt(a)gmail.com> wrote: > > Here is my tomcat catalina.out log > > file sample. See there is a tab space > > in logs > > > > 2013-06-27 05:30:00,065 > > [EDISN-Scheduler_Worker-2] ERROR > > com.example.edisn.sftp.SftpSession - > > Exception attempting to work with an > > SFTP Session: connection is closed by > > foreign host > > 2013-06-27 05:30:00,066 > > [EDISN-Scheduler_Worker-2] ERROR > > org.quartz.core.JobRunShell - Job > > EDISN.CTMS_Upload threw an unhandled > > Exception: > > com.example.edisn.EdisnRuntimeException: > Exception attempting to work with an SFTP Session: connection is closed by > foreign host > > at > > > com.example.edisn.sftp.SftpSession.doSession(SftpSession.java:64) > > at > > > com.example.edisn.EdisnSession.exec(EdisnSession.java:13) > > at > > > com.example.ctms.CtmsScheduledJob.executeInternal(CtmsScheduledJob.java:27) > > at > > > org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86) > > at > > > org.quartz.core.JobRunShell.run(JobRunShell.java:202) > > at > > org.quartz.simpl.SimpleThreadPool > > > $WorkerThread.run(SimpleThreadPool.java:525) > > Caused by: > > com.jcraft.jsch.JSchException: > > connection is closed by foreign host > > at > > com.jcraft.jsch.Session.connect(Unknown > Source) > > at > > com.jcraft.jsch.Session.connect(Unknown > Source) > > at > > > com.example.edisn.sftp.SftpSession.doSession(SftpSession.java:45) > > ... 5 more > > > > > > > > > > On Fri, Jul 5, 2013 at 3:27 PM, Balazs > > Scheidler <bazsi77(a)gmail.com> wrote: > > No, I implemented a different > > multiline style support first > > (that is not in pe), where > > continuation lines are > > indicated by indentation, like > > mime. > > > > Iirc tomcat has this kind of > > log file. Can you show a > > sample log entry? > > > > The infrastructure for > > multiline-prefix is also there > > but not added yet. > > > > Let me see the sample, I'll > > tell if the current solution > > works or not. > > > > On Jul 5, 2013 8:24 PM, > > "Satish Patel" > > <satish.txt(a)gmail.com> wrote: > > Thanks for reply > > Balazs, > > > > > > You mean say this > > feature is available > > in Open Source Edition > > (OSE) 3.4? Once after > > specifying flag > > "indented-multi-line" > > i can use > > multi-line-prefix? > > > > > > > > On Fri, Jul 5, 2013 at > > 1:26 PM, Balazs > > Scheidler > > <bazsi77(a)gmail.com> > > wrote: > > You have found > > the PE > > documentation > > but I have > > already ported > > this to the > > OSE tree and > > has been > > released as > > part of 3.4. > > > > You have to > > specify > > > indented-multi-line as a flag to the file source. > > > > On Jul 5, 2013 > > 6:28 PM, > > "Satish Patel" > > < > satish.txt(a)gmail.com> wrote: > > > > We > > have > > tomcat > > shop > > and at > > everyone > know tomcat has a java call trace in logs with tab space but syslog-ng > doesn't know about it and printing lines as a new line. I have read here > syslog-ng 3.x does support multi-line logs > http://www.balabit.com/sites/default/files/documents/syslog-ng-pe-4.0-guide… > > > > > > But > > does > > this > > feature > available in Open Source syslog-ng? If yes then why its not working for me? > > > > > > > > > ______________________________________________________________________________ > > Member > > info: > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > > FAQ: > > > http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > > > ______________________________________________________________________________ > > Member info: > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Documentation: > > > http://www.balabit.com/support/documentation/?product=syslog-ng > > FAQ: > > > http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > > > > > > > ______________________________________________________________________________ > > Member info: > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Documentation: > > > http://www.balabit.com/support/documentation/?product=syslog-ng > > FAQ: > > > http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > > > ______________________________________________________________________________ > > Member info: > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Documentation: > > > http://www.balabit.com/support/documentation/?product=syslog-ng > > FAQ: > > > http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > > > > > > > ______________________________________________________________________________ > > Member info: > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Documentation: > > > http://www.balabit.com/support/documentation/?product=syslog-ng > > FAQ: > > > http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > > > ______________________________________________________________________________ > > Member info: > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Documentation: > > > http://www.balabit.com/support/documentation/?product=syslog-ng > > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > > > > > > > > > > > > > ______________________________________________________________________________ > > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > > ------------------------------ > > Message: 2 > Date: Thu, 11 Jul 2013 10:22:45 -0400 > From: Satish Patel <satish.txt(a)gmail.com> > Subject: Re: [syslog-ng] Multi-line support issue > To: "Syslog-ng users' and developers' mailing list" > <syslog-ng(a)lists.balabit.hu> > Message-ID: > <CAPgF-fpPdRswwj2HNUXAxfgRekHVhwwF6O= > 7N39q9mfggN-nUQ(a)mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > This is my source declaration and i have put flags which you have > mentioned. > > source s_tomcat { > syslog( transport("udp") flags(indent-multi-line)); > }; > > I got following error when i am trying to put flags > > Error parsing afsocket, Unknown flag indent-multi-line in > /usr/local/syslog-ng-3.4.2/etc/syslog-ng.conf at line 54, column 33: > > syslog( transport("udp") flags(indent-multi-line) ); > ^^^^^^^^^^^^^^^^^ > > > > > On Thu, Jul 11, 2013 at 7:53 AM, Balazs Scheidler <bazsi(a)balabit.hu> > wrote: > > > > > I can't see the source declaration, it must be something along the lines > > of: > > > > source s_tomcat { > > file("/var/log/tomcat/xxx.log" flags(indent-multi-line)); > > }; > > > > On Wed, 2013-07-10 at 12:54 -0400, Satish Patel wrote: > > > Hi Balazs, > > > > > > > > > what is your thought about my config? did you see? > > > > > > > > > > > > On Mon, Jul 8, 2013 at 12:30 PM, Satish Patel <satish.txt(a)gmail.com> > > > wrote: > > > This is what i have configured and no luck with it.. can you > > > suggest what i am missing? > > > > > > destination d02_tc74_log > > > { file("/logs/server1/tomcat7.4/catalina_$YEAR$MONTH$DAY.log" > > > template("$(indent-multi-line ${MESSAGE})\n") > > > template(t_tomcatlog) owner("root") group("root") perm(0644) > > > dir_perm(0755) create_dirs(yes)); }; > > > filter server1 { host("server1.example.com") }; > > > log { > > > source (s_tomcat); > > > filter (server1); > > > filter (tomcat7_4); > > > destination (d02_tc74_log); > > > }; > > > > > > > > > > > > > > > On Mon, Jul 8, 2013 at 12:08 PM, Satish Patel > > > <satish.txt(a)gmail.com> wrote: > > > How do i use indented-multi-line ? I meant where do i > > > configure it? I tried but my syslog-ng doesn't > > > recognizing this option i have syslog-ng 3.3.7 could > > > you give me example where and how do i check whether > > > it is supported or not > > > > > > > > > > > > On Sat, Jul 6, 2013 at 2:12 AM, Balazs Scheidler > > > <bazsi77(a)gmail.com> wrote: > > > This looks.like the format that should be > > > supported by indented-multi-line > > > > > > On Jul 5, 2013 9:33 PM, "Satish Patel" > > > <satish.txt(a)gmail.com> wrote: > > > Here is my tomcat catalina.out log > > > file sample. See there is a tab space > > > in logs > > > > > > 2013-06-27 05:30:00,065 > > > [EDISN-Scheduler_Worker-2] ERROR > > > com.example.edisn.sftp.SftpSession - > > > Exception attempting to work with an > > > SFTP Session: connection is closed by > > > foreign host > > > 2013-06-27 05:30:00,066 > > > [EDISN-Scheduler_Worker-2] ERROR > > > org.quartz.core.JobRunShell - Job > > > EDISN.CTMS_Upload threw an unhandled > > > Exception: > > > > com.example.edisn.EdisnRuntimeException: > > Exception attempting to work with an SFTP Session: connection is closed > by > > foreign host > > > at > > > > > com.example.edisn.sftp.SftpSession.doSession(SftpSession.java:64) > > > at > > > > > com.example.edisn.EdisnSession.exec(EdisnSession.java:13) > > > at > > > > > > com.example.ctms.CtmsScheduledJob.executeInternal(CtmsScheduledJob.java:27) > > > at > > > > > > org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86) > > > at > > > > > org.quartz.core.JobRunShell.run(JobRunShell.java:202) > > > at > > > org.quartz.simpl.SimpleThreadPool > > > > > $WorkerThread.run(SimpleThreadPool.java:525) > > > Caused by: > > > com.jcraft.jsch.JSchException: > > > connection is closed by foreign host > > > at > > > com.jcraft.jsch.Session.connect(Unknown > > Source) > > > at > > > com.jcraft.jsch.Session.connect(Unknown > > Source) > > > at > > > > > com.example.edisn.sftp.SftpSession.doSession(SftpSession.java:45) > > > ... 5 more > > > > > > > > > > > > > > > On Fri, Jul 5, 2013 at 3:27 PM, Balazs > > > Scheidler <bazsi77(a)gmail.com> wrote: > > > No, I implemented a different > > > multiline style support first > > > (that is not in pe), where > > > continuation lines are > > > indicated by indentation, like > > > mime. > > > > > > Iirc tomcat has this kind of > > > log file. Can you show a > > > sample log entry? > > > > > > The infrastructure for > > > multiline-prefix is also there > > > but not added yet. > > > > > > Let me see the sample, I'll > > > tell if the current solution > > > works or not. > > > > > > On Jul 5, 2013 8:24 PM, > > > "Satish Patel" > > > <satish.txt(a)gmail.com> wrote: > > > Thanks for reply > > > Balazs, > > > > > > > > > You mean say this > > > feature is available > > > in Open Source Edition > > > (OSE) 3.4? Once after > > > specifying flag > > > "indented-multi-line" > > > i can use > > > multi-line-prefix? > > > > > > > > > > > > On Fri, Jul 5, 2013 at > > > 1:26 PM, Balazs > > > Scheidler > > > <bazsi77(a)gmail.com> > > > wrote: > > > You have found > > > the PE > > > documentation > > > but I have > > > already ported > > > this to the > > > OSE tree and > > > has been > > > released as > > > part of 3.4. > > > > > > You have to > > > specify > > > > > indented-multi-line as a flag to the file source. > > > > > > On Jul 5, 2013 > > > 6:28 PM, > > > "Satish Patel" > > > < > > satish.txt(a)gmail.com> wrote: > > > > > > We > > > have > > > tomcat > > > shop > > > and at > > > > everyone > > know tomcat has a java call trace in logs with tab space but syslog-ng > > doesn't know about it and printing lines as a new line. I have read here > > syslog-ng 3.x does support multi-line logs > > > http://www.balabit.com/sites/default/files/documents/syslog-ng-pe-4.0-guide… > > > > > > > > > But > > > does > > > this > > > feature > > available in Open Source syslog-ng? If yes then why its not working for > me? > > > > > > > > > > > > > > > ______________________________________________________________________________ > > > Member > > > info: > > > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > > > Documentation: > > http://www.balabit.com/support/documentation/?product=syslog-ng > > > FAQ: > > > > > http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > > > > > > > > > ______________________________________________________________________________ > > > Member info: > > > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > Documentation: > > > > > http://www.balabit.com/support/documentation/?product=syslog-ng > > > FAQ: > > > > > http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > > > > > > > > > > > > > > > ______________________________________________________________________________ > > > Member info: > > > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > Documentation: > > > > > http://www.balabit.com/support/documentation/?product=syslog-ng > > > FAQ: > > > > > http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > > > > > > > > > ______________________________________________________________________________ > > > Member info: > > > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > Documentation: > > > > > http://www.balabit.com/support/documentation/?product=syslog-ng > > > FAQ: > > > > > http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > > > > > > > > > > > > > > > ______________________________________________________________________________ > > > Member info: > > > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > Documentation: > > > > > http://www.balabit.com/support/documentation/?product=syslog-ng > > > FAQ: > > > > > http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > > > > > > > > > ______________________________________________________________________________ > > > Member info: > > > > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > Documentation: > > > > > http://www.balabit.com/support/documentation/?product=syslog-ng > > > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ______________________________________________________________________________ > > > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > Documentation: > > http://www.balabit.com/support/documentation/?product=syslog-ng > > > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > > > > > > > > ______________________________________________________________________________ > > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Documentation: > > http://www.balabit.com/support/documentation/?product=syslog-ng > > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > > > >

1 0

iv_port_poll socket operation on non-socket -- syslog-ng 3.3.7 Solaris 10 opterron
by Rakesh Rajasekharan 09 Jul '13

09 Jul '13

Hello everyone, I am using syslog-ng 3.3.7 compiled on solaris 10 opteron. When I try to run the daemon I get this error.. "iv_port_poll socket operation on non-socket". But this same binary works fine on Soalris 11 opteron. Any ideas on this please, why the error comes...? Thanks, Rakesh.

1 0

IPv6 TLS source question
by Jeff Lange 05 Jul '13

05 Jul '13

I'm attempting to setup a TCP IPv6 source declaration that uses TLS but syslog-ng is not happy with the configuration file. The following is what I use for IPv4 and it works fine: source s_tls { syslog( ip(0.0.0.0) port(6514) transport( "tls" ) tls ( key_file("/srv/syslog-ng/certs/priv-key.pem") cert_file("/srv/syslog-ng/certs/id-cert.pem") ca_dir("/srv/syslog-ng/certs") cipher_suite("SHA256:!aNULL:!eNULL:!ECDH:!DSS") ) ); }; However the following is not valid: source s_tls6 { syslog( ip("::") port(6515) transport( "tls" ) tls ( key_file("/srv/syslog-ng/certs/priv-key.pem") cert_file("/srv/syslog-ng/certs/id-cert.pem") ca_dir("/srv/syslog-ng/certs") cipher_suite("SHA256:!aNULL:!eNULL:!ECDH:!DSS") ) ); }; I've tried the ip with and without quotes with no luck. What am I doing wrong? Thanks -Jeff

3 4

[GSoC2013][syslog-ng:MySQL destination] Status update
by Gyula Petrovics 04 Jul '13

04 Jul '13

Hello, I published a new status update on my blog http://petrovicsgyula.blogspot.hu/ , and some source code on github https://github.com/gyula/mysqlclient/. Best regards, Gyula

1 0

Upcoming changes regarding my Debian/Ubuntu packages
by Gergely Nagy 04 Jul '13

04 Jul '13

Hi! For a good while now, I've been providing Debian and Ubuntu packages[1] for syslog-ng, and rest assured, I will continue to do so. However, the way they are prepared and how the repository is structured, will change. At the moment, there are two different components one can choose from: syslog-ng and syslog-ng-devel, which have packages for 3.3 and 3.4, respectively. However, -devel is not appropriate for 3.4 anymore, but I don't want to move it to syslog-ng either, partly because up to this point, there were no Ubuntu Lucid packages of 3.4. So, I will be making the following changes: * I'll create syslog-ng-3.3, syslog-ng-3.4 and syslog-ng-3.5 components * syslog-ng will point to the latest stable branch (3.4) * syslog-ng will point to the current development branch (3.5) * I'll be preparing Ubuntu Lucid packages for 3.4 too, so all branches will be available for every debian or ubuntu system that are still supported by their respective vendors. To be able to provide Ubuntu Lucid (10.04) packages sanely, I have to do the packaging a little bit differently: the source package will not be shared between debian and ubuntu anymore, and systemd support will be explicitly disabled on Ubuntu. But since Ubuntu uses upstart by default, ubuntu systems with systemd are fairly unlikely anyway. I plan to make these changes once 3.4.2 comes out. Until then, I'll introduce the syslog-ng-3.3 and -3.4 components, but syslog-ng will point to -3.3 still, and -devel to -3.4. Once 3.4.2 is out, though, I'll change the symlinks. [1]: http://asylum.madhouse-project.org/projects/debian/ -- |8]

1 2

[PATCH (3.5)] affile: Error out when there's a syntax error in the filename template
by Gergely Nagy 04 Jul '13

04 Jul '13

When there is a syntax error in the filename template (eg, "/var/log/foo-$(+ 0"), error out at config load time. This is done by adding an error output variable to affile_dd_new_instance() and both affile_dd_new() and afpipe_dd_new(), then using that to store the result of log_template_compile(). If that function fails, we bail out early, and use CHECK_ERROR() in the grammar to notify the user of the issue. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/affile/affile-dest.c | 23 +++++++++++++++-------- modules/affile/affile-dest.h | 6 +++--- modules/affile/affile-grammar.ym | 12 +++++++++--- 3 files changed, 27 insertions(+), 14 deletions(-) diff --git a/modules/affile/affile-dest.c b/modules/affile/affile-dest.c index c8fa9c1..c9447ce 100644 --- a/modules/affile/affile-dest.c +++ b/modules/affile/affile-dest.c @@ -1,5 +1,5 @@ /* - * Copyright (c) 2002-2012 BalaBit IT Ltd, Budapest, Hungary + * Copyright (c) 2002-2013 BalaBit IT Ltd, Budapest, Hungary * Copyright (c) 1998-2012 Balázs Scheidler * * This program is free software; you can redistribute it and/or modify it @@ -703,17 +703,24 @@ affile_dd_free(LogPipe *s) } static AFFileDestDriver * -affile_dd_new_instance(gchar *filename) +affile_dd_new_instance(gchar *filename, GError **error) { AFFileDestDriver *self = g_new0(AFFileDestDriver, 1); + LogTemplate *filename_template; + + filename_template = log_template_new(configuration, NULL); + if (!log_template_compile(filename_template, filename, error)) + { + log_template_unref(filename_template); + return NULL; + } log_dest_driver_init_instance(&self->super); self->super.super.super.init = affile_dd_init; self->super.super.super.deinit = affile_dd_deinit; self->super.super.super.queue = affile_dd_queue; self->super.super.super.free_fn = affile_dd_free; - self->filename_template = log_template_new(configuration, NULL); - log_template_compile(self->filename_template, filename, NULL); + self->filename_template = filename_template; log_writer_options_defaults(&self->writer_options); file_perm_options_defaults(&self->file_perm_options); self->writer_options.mark_mode = MM_NONE; @@ -728,15 +735,15 @@ affile_dd_new_instance(gchar *filename) } LogDriver * -affile_dd_new(gchar *filename) +affile_dd_new(gchar *filename, GError **error) { - return &affile_dd_new_instance(filename)->super.super; + return &affile_dd_new_instance(filename, error)->super.super; } LogDriver * -afpipe_dd_new(gchar *filename) +afpipe_dd_new(gchar *filename, GError **error) { - AFFileDestDriver *self = affile_dd_new_instance(filename); + AFFileDestDriver *self = affile_dd_new_instance(filename, error); self->is_pipe = TRUE; return &self->super.super; } diff --git a/modules/affile/affile-dest.h b/modules/affile/affile-dest.h index 3f4f21b..dcee654 100644 --- a/modules/affile/affile-dest.h +++ b/modules/affile/affile-dest.h @@ -1,5 +1,5 @@ /* - * Copyright (c) 2002-2012 BalaBit IT Ltd, Budapest, Hungary + * Copyright (c) 2002-2013 BalaBit IT Ltd, Budapest, Hungary * Copyright (c) 1998-2012 Balázs Scheidler * * This program is free software; you can redistribute it and/or modify it @@ -53,8 +53,8 @@ typedef struct _AFFileDestDriver gint time_reap; } AFFileDestDriver; -LogDriver *affile_dd_new(gchar *filename); -LogDriver *afpipe_dd_new(gchar *filename); +LogDriver *affile_dd_new(gchar *filename, GError **error); +LogDriver *afpipe_dd_new(gchar *filename, GError **error); void affile_dd_set_create_dirs(LogDriver *s, gboolean create_dirs); void affile_dd_set_fsync(LogDriver *s, gboolean enable); diff --git a/modules/affile/affile-grammar.ym b/modules/affile/affile-grammar.ym index abecee4..0d2f099 100644 --- a/modules/affile/affile-grammar.ym +++ b/modules/affile/affile-grammar.ym @@ -1,5 +1,5 @@ /* - * Copyright (c) 2002-2012 BalaBit IT Ltd, Budapest, Hungary + * Copyright (c) 2002-2013 BalaBit IT Ltd, Budapest, Hungary * Copyright (c) 1998-2012 Balázs Scheidler * * This program is free software; you can redistribute it and/or modify it @@ -144,7 +144,10 @@ dest_affile dest_affile_params : string { - last_driver = *instance = affile_dd_new($1); + GError *error; + + last_driver = *instance = affile_dd_new($1, &error); + CHECK_ERROR(last_driver, @1, "Error compiling template; %s", error->message); free($1); last_writer_options = &((AFFileDestDriver *) last_driver)->writer_options; last_file_perm_options = &((AFFileDestDriver *) last_driver)->file_perm_options; @@ -171,7 +174,10 @@ dest_affile_option dest_afpipe_params : string { - last_driver = *instance = afpipe_dd_new($1); + GError *error; + + last_driver = *instance = afpipe_dd_new($1, &error); + CHECK_ERROR(last_driver, @1, "Error compiling template; %s", error->message); free($1); last_writer_options = &((AFFileDestDriver *) last_driver)->writer_options; last_file_perm_options = &((AFFileDestDriver *) last_driver)->file_perm_options; -- 1.7.10.4

2 2

Problems with recovering from 'disk full' I/O error
by Johnson, Chris (HP TippingPoint Roseville) 26 Jun '13

26 Jun '13

Hi all, I've come across a situation where syslog-ng (3.3.3 and 3.3.9) aborts after trying to write to a disk that has no space and then is cleaned up. I've been able to reproduce the 'error' condition with the following Set up: In the config file: ########################################################### # test log destination # filter f_test{program("LOGID_99-*" type("glob"));}; destination d_test { file("/var/testpartition/test.log" perm(0644) flags(no-multi-line)); }; log { source(s_local); filter(f_test); destination(d_test); flags(final); }; 1) Fill up the '/var/testpartition' disk: root@device:/var/testpartition# dd if=/dev/zero of=/var/testpartition/foo bs=1M dd: writing '/var/testpartition/foo': No space left on device 3531+0 records in 3529+1 records out 2) Send messages ('TEST: pre I/O error')to 'test.log' until syslog-ng complains: 2013-06-14 22:41:09.623 [device] [syslog-ng-ERROR:] "I/O error occurred while writing; fd='22', error='No space left on device (28)'" 2013-06-14 22:41:09.623 [device] [syslog-ng-NOTICE:] "Suspending write operation because of an I/O error; fd='22', time_reopen='60'" 3) Restart syslog-ng with a SIGHUP root@device:/var/testpartition# kill -s HUP $(cat /var/run/syslog-ng.pid) Program received signal SIGHUP, Hangup. 0x00007f38ee8dad63 in __epoll_wait_nocancel () from /xxx/xxx/xxx/xxx/xxx/xxx/lib/libc.so.6 (gdb) continue Continuing. 4) Send another message ('TEST: post I/O error; post SIGHUP')to syslog-ng (it complains): 2013-06-14 22:41:37.147 [device] [syslog-ng-NOTICE:] "Configuration reload request received, reloading configuration;" 2013-06-14 22:41:59.610 [device] [syslog-ng-ERROR:] "I/O error occurred while writing; fd='23', error='No space left on device (28)'" 2013-06-14 22:41:59.610 [device] [syslog-ng-NOTICE:] "Suspending write operation because of an I/O error; fd='23', time_reopen='60'" 5) Clean up '/var/testpartition' disk: root@device:/var/testpartition# rm foo 6) Send another message ('TEST: post I/O error; post SIGHUP 2'); syslog-ng aborts: Program received signal SIGABRT, Aborted. 0x00007f38ee83eda5 in raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. in ../nptl/sysdeps/unix/sysv/linux/raise.c (gdb) bt #0 0x00007f38ee83eda5 in raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x00007f38ee8402c3 in abort () at abort.c:88 #2 0x00007f38ee87af18 in __libc_message (do_abort=2, fmt=0x7f38ee927cc8 "*** glibc detected *** %s: %s: 0x%s ***\n") at ../sysdeps/unix/sysv/linux/libc_fatal.c:170 #3 0x00007f38ee8804b8 in malloc_printerr (action=2, str=0x7f38ee927dd0 "double free or corruption (!prev)", ptr=<value optimized out>) at malloc.c:5891 #4 0x00007f38ee8825c6 in __libc_free (mem=0x7f38ee91e820) at malloc.c:3626 #5 0x00007f38ef7d989a in ?? () #6 0x0000000000401800 in ?? () at elf-init.c:99 #7 0x00007f38efcf0257 in log_proto_file_writer_flush (s=0x674de0) at logproto.c:306 #8 0x00007f38efcf04a8 in log_proto_file_writer_post (s=0x674de0, msg=0x6558a0 "2013-06-14T22:42:30.476+00:00 device LOGID_99-INFO: TEST: post I/O error; post SIGHUP 2\n", msg_len=89, consumed=0x7fffa53e570c) at logproto.c:380 #9 0x00007f38efcfc274 in log_proto_post (s=0x674de0, msg=0x6558a0 "2013-06-14T22:42:30.476+00:00 device LOGID_99-INFO: TEST: post I/O error; post SIGHUP 2\n", msg_len=89, consumed=0x7fffa53e570c) at logproto.h:95 #10 0x00007f38efcfc095 in log_writer_flush (self=0x655a30, flush_mode=LW_FLUSH_NORMAL) at logwriter.c:983 #11 0x00007f38efcf9be3 in log_writer_work_perform (s=0x655a30) at logwriter.c:129 #12 0x00007f38efcf9efe in log_writer_io_flush_output (s=0x655a30) at logwriter.c:198 #13 0x00007f38efd238b5 in iv_run_tasks (st=0x6036a0) at iv_task.c:46 #14 0x00007f38efd22a86 in iv_main () at iv_main.c:266 #15 0x00007f38efcfedf9 in main_loop_run () at mainloop.c:736 #16 0x00000000004017c9 in main (argc=1, argv=0x7fffa53e5968) at main.c:263 (gdb) The last two test messages are being written (run together with the last partial message that generated the 'disk full' I/O error): root@device:/var/testpartition# tail test.log 2013-06-14T22:40:58.140+00:00 device LOGID_99-INFO: TEST: pre I/O error 2013-06-14T22:40:59.150+00:00 device LOGID_99-INFO: TEST: pre I/O error 2013-06-14T22:41:00.180+00:00 device LOGID_99-INFO: TEST: pre I/O error 2013-06-14T22:41:01.176+00:00 device LOGID_99-INFO: TEST: pre I/O error 2013-06-14T22:41:02.170+00:00 device LOGID_99-INFO: TEST: pre I/O error 2013-06-14T22:41:03.010+00:00 device LOGID_99-INFO: TEST: pre I/O error 2013-06-14T22:41:03.930+00:00 device LOGID_99-INFO: TEST: pre I/O error 2013-06-14T22:41:05.040+00:00 device LOGID_99-INFO: TEST: pre I/O error 2013-06-14T22:41:06.180+00:00 device LOGID_99-INFO: TEST: pre I/O error 2013-06-14T22:41:07.140+00:00 device LOGID_99-INFO: T2013-06-14T22:42:30.476+00:00 device LOGID_99-INFO: TEST: post I/O error; post SIGHUP 2013-06-14T22:42:30.476+00:00 device LOGID_99-INFO: TEST: post I/O error; post SIGHUP 2 root@device:/var/testpartition# Thanks, Chris ---------------------------------------- Christopher Johnson chris.johnson3(a)hp.com<mailto:chris.johnson3@hp.com> HP Software - Security Product Group (916) 785-2817 ----------------------------------------

3 3

Syslog-ng unable to configure the client for sending messages to central server
by Rakesh Rajasekharan 26 Jun '13

26 Jun '13

I am getting some errors when i try to set up my client machine and try to send the messages to a remote central server. This is the error that I get, syslog-ng service starting. Plugin module not found in 'module-path'; module-path='/opt/soe/local/syslog-ng-3.3.7/lib/syslog-ng', module='afsocket' Plugin module not found in 'module-path'; module-path='/opt/soe/local/syslog-ng-3.3.7/lib/syslog-ng', module='afsocket' Error parsing destination, destination plugin tcp not found in /opt/soe/local/syslog_ng/etc/syslog-ng.conf at line 24, column 9: tcp("10.5.68.164" port(514)); ^^^ syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng. Any help on this please...

3 2