- syslog-ng - lists.balabit.hu

Suppress almost-identical messages?
by Evade Flow 23 May '14

23 May '14

I'm trying to clean up the ad hoc logging in a collection of applications started on an embedded device using the daemon utility. Example: /usr/bin/daemon --name=myapp --output=user.debug --chdir=/MyApp -- /MyApp/bin/myapp There are 10 different applications started this way, written by different individuals/teams, and our syslog-ng.conf file currently looks something like this: @version:3.3.6 source s_local { system(); internal(); }; destination d_localfile { file("/var/log/messages" suppress(30)); }; log { source(s_local); destination(d_localfile); }; Because the --output=user.debug option to the daemon utility will cause the applications' stdout/stderr to be redirected to syslog, I'm a little worried about 'rogue' printf()'s filling up the logs with stuff like: Jan 1 00:00:54 (none) myapp: Battery voltage is 11.07 volts Jan 1 00:00:54 (none) myapp: Battery voltage is 11.10 volts Jan 1 00:00:54 (none) myapp: Battery voltage is 11.07 volts Jan 1 00:00:54 (none) myapp: Battery voltage is 11.14 volts Jan 1 00:00:54 (none) myapp: Battery voltage is 11.13 volts Jan 1 00:00:55 (none) myapp: Battery voltage is 11.09 volts Jan 1 00:00:55 (none) myapp: Battery voltage is 11.14 volts ... Does syslog-ng support suppression of almost-but-not-quite identical messages? It would be nice to see something like this in the logs: Jan 1 00:55:11 myapp: Battery voltage is 11.07 volts Jan 1 00:55:14 myapp: Last message 'Battery voltage is 1' repeated with nearly identical content 418 times, suppressed by syslog-ng on flahblargle A few of my colleagues have suggested that this is probably more trouble than it's worth, and we should just fix the spammy app(s) rather than try to 'outsmart' them. And I think I agree with them, but... it was my idea to move to syslog-ng, so there'll be a bit more egg on my face than theirs if we accidentally allow a rogue printf() in a hot loop somewhere to escape into production. `:-} Any advice/insight much appreciated, thanks!

4 6

Email
by Anwar El fatayri 22 May '14

22 May '14

syslog-ng(a)lists.balabit.hu El Fatayri Anwar

1 0

Handing over syslog-ng maintenance
by Balazs Scheidler 21 May '14

21 May '14

Dear syslog-ng users, I would like to announce that Gergely Nagy (aka @algernoone on Twitter and algernon on github), is taking over the maintenance of syslog-ng master from me. He was maintaining the older stable versions for a few years now, was also active in creating syslog-ng binaries and started the syslog-ng Incubator, and is also behind the Google Summer of Code effort. I have a lot of distractions in my personal life, and although I don't want to retreat from the syslog-ng community entirely, I wanted to leave syslog-ng in Gergely's able hands. Happy logging, Cheers, Bazsi -- Bazsi

4 3

insider 2014-05: GsoC; perl & python; web interfaces; protecting logs;
by Peter Czanik 15 May '14

15 May '14

Dear syslog-ng users, This is the 34th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news. FEATURED NEWS Four GsoC students are working on syslog-ng ------------------------------------------- Thanks to Google, there are four students working on extending syslog-ng with new features during the summer. These are features, which were often requested on the mailing list or at different conferences: - integration with configuration management systems - ZMQ transport, both source and destination - AMQP source driver - TLS support for the mongodb destination Read more about GSoC at https://algernon.blogs.balabit.com/2014/04/gsoc2014-syslog-ng-accepted-proj… Python and Perl support in incubator ------------------------------------ It is still only available in git, as it needs some more polish, but the syslog-ng incubator gained Perl and Python support during the last month. Both the perl and python destinations use the value-pairs framework to get data transferred from syslog-ng to the script, and thus, work differently from the Lua destination. With value-pairs, one can select what parts of the message will be transferred to the script. The script will need to have a queue function (settable with the queue-func() option), which will receive a hash-map of values. Additionally, one can set an init and a deinit function too, to run whenever the driver starts or shuts down. Check it out and let us know your experiences! It's available athttps://github.com/balabit/syslog-ng-incubator Web based user interfaces for syslog-ng --------------------------------------- One of the most popular BalaBit blogs is about syslog-ng web based graphical user interfaces (web GUIs). It's already three years old, and many things have changed. At that time, only a single Logging as a Service solution was available, now a new one pops up regularly. Also, there were a lot less logging-related GUIs, so some not strictly syslog-ng related solutions were included as well. You can read an updated version of the blog, focusing on syslog-ng based solutions at https://czanik.blogs.balabit.com/2014/05/web-based-user-interfaces-for-sysl… Protecting log data against targeted attacks -------------------------------------------- BalaBit has been saying that SIEM and other analytic tools are only as good as the underlying data. Attackers are also aware of this, and often target log management and SIEMs to hide their presence. Read this blog post for some logging best practices and how syslog-ng can help to secure your logging infrastructure: https://jluby.blogs.balabit.com/2014/05/06/protecting-log-data-against-targ… NEW RELEASES * Check git if you are impatient :) Your feedback and news tips about the next issue is welcome at _documentation(a)balabit.com <mailto:documentation@balabit.com>_To read this newsletter on-line, visit: _http://insider.blogs.balabit.com/_ -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

weird disk usage problems -- syslog-ng/ELSA
by Russell Fulton 11 May '14

11 May '14

HI Folks It would seem that the issue here is with syslog-ng but it manifested itself in the context of an ELSA instance so I am sending this to both lists, apologies to those who get it twice. I just discovered that my ELSA machine had run out of disk some 18 hours ago. /dev/mapper/datavg-d01 985G 943G 0 100% /d01 [rful011@itslogprd05 ~]$ sudo du -sh /d01/* 671G /d01/data 16K /d01/lost+found 43G /d01/mysql But these figures don’t match — there is about 250GB of disk unaccounted for (although I did not pick up this discrepancy at first — instead I went looking in ELSA node.log to try and figure out why logs were not getting deleted). After failing to find any evidence of problems in the logs I then stopped syslog-ng and set about moving the tmp/buffers to another disk. I copied them but when I went to delete the originals they had vanished! I was dazed and confused ;) until I realised that the usage of /d01 had dropped to 77% which allowed cron.pl to process all the waiting buffers and delete them. It would seem that syslog-ng was sitting on some very large files that had been unlinked from the directory (so du did not see them) stopping syslog-ng would have closed the files allowing the disk to be released. Hmmm…. with syslog-ng it might have been fifos (the ELSA syslog-ng config uses fifos to deliver logs to the ELSA perl scripts) but they should not have had buffers that size. Has anyone seen anything like this before or have any idea what might trigger this issue? Russell

2 1

Issue regarding the configurations of syslog-ng application on Centos
by Christian Hangi 09 May '14

09 May '14

Hello Guys, I am new in Syslog-ng application and also Linux and I need your help regarding the functionality of syslog-ng on Linux. I am building a syslog-ng server capable of receiving syslog messages from syslog-ng clients. In doing so, I would like to those syslog messages to be received and stored inside the Syslog MySQL database of the server inside the syslog_incoming table so that I can be able view those messages. I am using the Centos 6.5 of Redhat distributions and the Syslog-ng application running on that PC acts as a Syslog-ng server. This is my first attempt to allow the syslog messages storage on the database: @version:3.2 # syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; #source s_sys { #file ("/proc/kmsg" program_override("kernel: ")); # unix-stream ("/dev/log"); # internal(); # udp(ip(0.0.0.0) port(514)); #}; #A change was made here! source network { syslog(ip(0.0.0.0) port(514) transport(udp));}; destination db_mysql{sql(type(mysql) host("localhost") username("cacti") password("P@ssw0rd") database("syslog") table("syslog_incoming") columns("host", "facility", "priority", "date", "time", "message") values("$HOST","$FACILITY","$PRIORITY","$YEAR-$MONTH-$DAY","$HOUR:$MIN:$SEC","$MSG") indexes("host", "facility", "priority", "date", "time", "message") );}; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #A change was made here! log { source(s_sys);source(network); destination(d_mysql);}; #log { source(network); destination(d_mysql);}; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; #log { source(s_sys); filter(f_kernel); destination(d_kern); }; #log { source(s_sys); filter(f_default); destination(d_mesg); }; #log { source(s_sys); filter(f_auth); destination(d_auth); }; #log { source(s_sys); filter(f_mail); destination(d_mail); }; #log { source(s_sys); filter(f_emergency); destination(d_mlal); }; #log { source(s_sys); filter(f_news); destination(d_spol); }; #log { source(s_sys); filter(f_boot); destination(d_boot); }; #log { source(s_sys); filter(f_cron); destination(d_cron); }; # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #A change was made here! log { source(network); destination(d_mysql);}; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; #log { source(s_sys); filter(f_kernel); destination(d_kern); }; #log { source(s_sys); filter(f_default); destination(d_mesg); }; #log { source(s_sys); filter(f_auth); destination(d_auth); }; #log { source(s_sys); filter(f_mail); destination(d_mail); }; #log { source(s_sys); filter(f_emergency); destination(d_mlal); }; #log { source(s_sys); filter(f_news); destination(d_spol); }; #log { source(s_sys); filter(f_boot); destination(d_boot); }; #log { source(s_sys); filter(f_cron); destination(d_cron); }; # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: And this is the message I got when I tried to restart the application : Stopping syslog-ng: [FAILED] Starting syslog-ng: Error in configuration, unresolved destination reference; destination='d_mysql' [FAILED] Then I tried to use a named pipe to accept and send thoses messages to the syslog_incoming table of the Syslog MySQL database through a template: @version:3.2 # syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # options { flush_lines (0); time_reopen (10); log_fifo_size (1000); long_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; #source s_sys { # file ("/proc/kmsg" program_override("kernel: ")); # unix-stream ("/dev/log"); # internal(); # udp(ip(0.0.0.0) port(514)); #}; destination d_mysql{ pipe("/var/log/mysql.pipe" template("INSERT INTO syslog_incoming(host, facility, priority, date, time, message) VALUES('$HOST', '$FACILITY', '$PRIORITY', '$ YEAR-$MONTH-$DAY', '$HOUR:$MIN:$SEC', '$MSG');\n") template-escape(yes)); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #A change was made here! log { source(network); destination(d_mysql);}; #log { source(network); destination(d_mysql);}; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; #log { source(s_sys); filter(f_kernel); destination(d_kern); }; #log { source(s_sys); filter(f_default); destination(d_mesg); }; #log { source(s_sys); filter(f_auth); destination(d_auth); }; #log { source(s_sys); filter(f_mail); destination(d_mail); }; #log { source(s_sys); filter(f_emergency); destination(d_mlal); }; #log { source(s_sys); filter(f_news); destination(d_spol); }; #log { source(s_sys); filter(f_boot); destination(d_boot); }; #log { source(s_sys); filter(f_cron); destination(d_cron); }; # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: And I got this message assuming that the syslog-ng server is working Stopping syslog-ng: [ OK ] Starting syslog-ng: [ OK ] But when I checked to the table of the database it was empty. So, I would like to know, do you miss something on the first attempt to allow the syslog-server to work perfectly or should I just not to uncomment the source system (s_sys) to accept the syslog messages from the Syslog-clients? On the 2nd attempt, what needs to be done to acctually see those messages? Please I need some advice on those two attempts. Kindly regards, Chris Hangi

2 1

[ANN]: syslog-ng Incubator maintenance
by Gergely Nagy 07 May '14

07 May '14

Hi! While it has effectively happened about a month ago, I didn't get around to announce the good news yet: syslog-ng Incubator maintenance has been handed over to Viktor Tusa <tusa(a)balabit.hu>, author of (in no particular order) the Lua and RSS destinations, graphite-output template function, and more. He has been doing a terrific job so far, and I'm very happy with the maintainer switch. The Incubator is in the best hands possible. Thank you, Viktor! -- |8]

1 0

AMQP source
by MÓZES Ádám István 06 May '14

06 May '14

Hi All, My name is MÓZES Ádám István, and I'll develop the AMQP source this summer. A short introduction about me and the progress of the development can be found at this blog: http://dnsjts.blogspot.hu/search/label/gsoc2014 . Best regards, Ádám

1 0

Re: [syslog-ng] Ubuntu Precise -ng filling out buffer, dropping messages
by Chaman Chakalaka 05 May '14

05 May '14

Just to close the circle on this, I found the problem. After having increased the rmem_max I was no longer seeing any UDP packet drops. I was still not seeing some log messages, and at this point it was a configuration issue. I noticed I had filters such as: filter f_info { level(info);} Obviously this was only catching info level messages. Changed to: filter f_info {level(info...emerg);} Sorry for the noise, but I still wanted to bring some closure in case someone runs into this. The initial packet drop blinded me into looking into some basic configs, thanks all for the suggestions! > From: syslog-ng-request(a)lists.balabit.hu > Subject: syslog-ng Digest, Vol 108, Issue 27 > To: syslog-ng(a)lists.balabit.hu > Date: Wed, 30 Apr 2014 12:00:02 +0200 > > Send syslog-ng mailing list submissions to > syslog-ng(a)lists.balabit.hu > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.balabit.hu/mailman/listinfo/syslog-ng > or, via email, send a message with subject or body 'help' to > syslog-ng-request(a)lists.balabit.hu > > You can reach the person managing the list at > syslog-ng-owner(a)lists.balabit.hu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of syslog-ng digest..." > > > Today's Topics: > > 1. Re: Ubuntu Precise -ng filling out buffer, dropping messages > (Evan Rempel) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 29 Apr 2014 12:38:52 -0700 > From: Evan Rempel <erempel(a)uvic.ca> > Subject: Re: [syslog-ng] Ubuntu Precise -ng filling out buffer, > dropping messages > To: syslog-ng(a)lists.balabit.hu > Message-ID: <535FFFCC.8000203(a)uvic.ca> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > On 04/29/2014 12:20 PM, Chaman Chakalaka wrote: > > I fixed the issue with udp being dropped at the system level by changing the linux core files but this time restarting the system, now I know they are missing somewhere between getting to the system and syslog-ng :( > > > > I'm lost once again... > > > > -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > trysource s_network_udp { udp(so_rcvbuf(33554432) log_fetch_limit(20000) log_iw_size(1000000) ); }; > > > This assumes that you have a large net.core.rmem_max > > net.core.rmem_max = 52428800 > > See how that goes. > > > ------------------------------ > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > End of syslog-ng Digest, Vol 108, Issue 27 > ******************************************

1 0

ZMQ transport, source and destination
by Laci Mészáros 04 May '14

04 May '14

Hy all, I am László Mészáros a student at Szechenyi Istvan University learning software engineering. I've made a proposal for Google Summer of Code to implement the zeromq transport, source and destination. The organization accepted it, so during this summer I will work on it. You can find the proposal here: https://github.com/balabit/syslog-ng/wiki/GSoC2014-Proposal:-ZMQ-transport,… I made a blog where you can follow the status, it can be found here: http://lmesz.tumblr.com/post/84705072064/introduction-aims I think one part it will be fun and on another it will be difficult, so I would like to ask all of you, if you have some use cases on your shelf about how you would/will use zmq for logging via network, please share it with us. You can send it to the list or to me personally it would be very useful during the implementation. Br, Laci

1 0