January 2013 - syslog-ng - lists.balabit.hu

syslog-ng 3.4.0 parserdb on other than MESSAGE
by Evan Rempel 17 Jan '13

17 Jan '13

OK, from what Balazs writes below I guess it is not possible to use the parserdb functionality on the "$HOST $PROGRAM" contents. Can someone confirm this is the case? If it can be done, a quick pointer please. If it can not be done, but I want to, how can I do it? rewrite { set("$MSG" value("orig.message")); set("$HOST($PROGRAM)" value("MESSAGE")); } parser(pattern_host_program); rewrite { set("$orig.message" value("MESSAGE")); set("" value("orig.message")); } parser(pattern_message); ... all of my filters, log statements etc. Evan. ________________________________________ From: Balazs Scheidler [bazsi77(a)gmail.com] Sent: Saturday, December 22, 2012 6:50 AM To: Syslog-ng users' and developers' mailing list; Evan Rempel Subject: Re: [syslog-ng] syslog-ng 3.4.0beta1 has been released ----- Original message ----- > > I thought that 3.4 was going to include > > - junctions so that results of multiple parsers could be combined it is in. > > - the ability to run a parser against a user defined macro rather than > always parsing the complete MESSAGE marcro. csvparser has a template() option, but dbparser doesn't (though that's trivial to add). what I had in mind (fully n dimensional patterndb) wasn't finished, and I didn't want to wait any longer. > > Did these features fail to make it into 3.4 or is the documentation just > missing these items? > docs is not yet competely updated. contributions would be welcome there too.

4 3

syslog-ng help
by Manimaran kk 17 Jan '13

17 Jan '13

Could some please help me to setup an log collection for the following logging services like telnet, ftp, ssh, smpd using syslog-ng. Is there any blog or article available? Thank you. Regards, Mani

1 0

syslog-ng help
by Manimaran kk 17 Jan '13

17 Jan '13

Hi, Could some please help me to setup and

1 0

Flow-control
by Anton Koldaev 16 Jan '13

16 Jan '13

Could you please explain the following statement: > If the control window is full, syslog-ng stops reading messages from the source until some messages are successfully sent to the destination. What does that mean - "stops reading messages from the source"? My applications is still sending messages to this souce so where will all the logs at that moment? Where will it start reading the source? From the same point it stopped or not? -- Best regards, Koldaev Anton

2 6

Stats levels
by Anton Koldaev 16 Jan '13

16 Jan '13

I'm using stats_level(0) on my syslog-ng clients and I'm able to see processed, stored and dropped messages. But there are only processed messages counters at my central syslog-ng server when using stats_level(0). Setting stats_level to 1 gives me dropped and stored counters but the total number of counters on my central syslog-ng server at stats_level(1) is about 70 000 and it takes too long to get it. Any advices? -- Best regards, Koldaev Anton

1 0

"make check" in a restricted environment
by Peter Czanik 15 Jan '13

15 Jan '13

Hello, I'm building my packages mainly in the openSUSE build service (OBS: http://build.opensuse.org/) My problem is, that while "make check" runs without problems on my local machine, it fails in the OBS. You can check where it fails (most likely hiding other failures) at https://build.opensuse.org/package/live_build_log?arch=x86_64&package=syslo… The OBS builds packages in a very restricted virtualized environment, with no networking, etc. to make sure, that even harmful sources can't manipulate the build infrastructure. The above problem also seems to be related. Could you check the tests and make sure, that no name resolution, etc. is needed to pass the tests? Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

syslog-ng OSE configurator
by András Mitzki 15 Jan '13

15 Jan '13

Hello, My name is Andras Mitzki, I work for BalaBit as a syslog-ng test engineer. I started to develop a syslog-ng OSE configurator web application. My goal with this web app is to generate a syntactically correct syslog-ng OSE config with only a few clicks. Please try it, and let me know if something is not clear, or you have found a bug. Of course any ideas are welcome. syslog-ng OSE configurator app: http://mitzkia.github.com/syslog-ng-ose-configurator syslog-ng OSE configurator source: https://github.com/mitzkia/syslog-ng-ose-configurator I have also written a short blog post about it: http://micek.blogs.balabit.com/en/2013/01/syslog-ng-ose-configurator/ Thanks for trying it.

7 15

syslog-ng 3.3.7 DNS resolving Problem
by Daniel Neubacher 09 Jan '13

09 Jan '13

Hello there, I've got a little trouble with the DNS resolving of syslog-ng. Last week I patched my syslog installation with the threaded dns bugfix (https://bugzilla.balabit.com/show_bug.cgi?id=212) and it seems like most of my problems are gone but one is still remaining. Many times a day messages are sorted into a folder with the DNS name of my syslog-ng server instead of the real host where the log is coming from. The log line still has the right host in the text and most of the time it is working but I could not find any way to reproduce the problem on demand yet. For debugging I've disabled any logging for the server itself but it still happens. My destinations are configured like this: destination d_syslog { file("/log/syslog/${R_YEAR}/${R_MONTH}/${R_DAY}/$FULLHOST_FROM/$PROGRAM" template(t_plain)); }; And my dns options: use_fqdn(yes); dns_cache(yes); dns_cache_size(16384); dns_cache_expire(300); dns_cache_expire_failed(10); I've tried disabling the syslog-ng cache,installing a local caching bind and after that a nscd but with no success. With 750 servers sending 30k-40k logs per second the dns querys are too expensive and I need the internal syslog-ng caching. With local bind caching the logs per second are dropping down to 2500. Does anybody has an idea to fix this? -- Daniel Neubacher, Network Administrator daniel.neubacher(a)xing.com<mailto:daniel.neubacher@xing.com> XING AG Gaensemarkt 43, 20354 Hamburg, Germany Tel. +49 40 419131-28, Fax +49 40 419131-11 Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 98807 Exec. Board (Vorstand): Dr. Stefan Groß-Selbeck (Vorsitzender), Dr. Thomas Vollmoeller, Ingo Chu, Dr. Helmut Becker, Jens Pape Chairman of the Supervisory Board (Aufsichtsratsvorsitzender): Dr. Neil Sunderland This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden and may be unlawful.

2 9

loggen read-file tokens?
by Clayton Dukes 08 Jan '13

08 Jan '13

What is the expected file format used for loggen? i.e: which tokens should each line consist of? >From the docs: http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.3-guid… I'd like to pump a bunch of messages (other than PADPADPAD) through loggen to generate tests. But it doesn't say anything about what format to use. I've tried using just a text file with a message per line, but I get Invalid line X for some messages an not others. Am I just misunderstanding what this command line option is for? Sample Messages: 2012-11-27 08:48:29 16.22.1.42 %SYS-3-CPUHOG: the CPU is hogged 2012-11-27 08:48:29 16.22.2.42 %IR-6-INSCARD: Card inserted in slot 0, interfaces administratively shut down 2012-11-27 08:48:29 16.22.2.42 %OIR-6-REMCARD: Card removed from slot 0, interfaces disabled So it seem I should be able to use: loggen --size 300 --rate 6000 --interval 600 192.168.28.29 514 -R ./myfile.txt but then only some lines work and some don't. Sometimes the host comes in on the receiving server as an invalid IP. Is there a way to specify tokens, like f1=datetime -f2=host -f3=PRI -f4=msg. Would really be nice if we could also specify the delimiter between them, like: f1=datetime -f2=host -f3=PRI -f4=msg --delim=\t ______________________________________________________________ Clayton Dukes ______________________________________________________________

2 2

AUTO: James B Horwath is out of the office (returning 01/09/2013)
by James B Horwath 08 Jan '13

08 Jan '13

I am out of the office until 01/09/2013. Note: This is an automated response to your message "syslog-ng Digest, Vol 93, Issue 7" sent on 1/8/2013 15:43:58. This is the only notification you will receive while this person is away. ----------------------------------------- This message, and any attachments to it, may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are notified that any use, dissemination, distribution, copying, or communication of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately by return e-mail and delete the message and any attachments. Thank you.

2 1