- syslog-ng - lists.balabit.hu

Per-destination log_msg_size?
by David Schweikert 21 Feb '14

21 Feb '14

Hi, We have some setups where we want to have a large log_msg_size, but need to send some logs to an external party which only can handle standard BSD syslog messages with a maximum size of 1024 bytes. Question: would it be possible to add a per-destination log-msg-size to truncate the messages on a per-destination basis? I think that it would be very useful. Something like: destination d_remote { udp("192.168.65.10" port(514) log_msg_size(1024)); }; Do you find it a good idea and is there any chance that this might be implemented? Thanks David

5 7

Re: [syslog-ng] syslog-ng Digest, Vol 106, Issue 12
by Terry Slattery 18 Feb '14

18 Feb '14

Why not stand up a test server and add it to the logging configs of the non-working and a couple of working switches? It could be a VM, since it won't have much load. You could also capture the log messages and play them back at the test server without impacting your production server or data archives. -tcs On 2/18/14 3:12 PM, syslog-ng-request(a)lists.balabit.hu wrote: > Date: Tue, 18 Feb 2014 15:12:50 -0500 > From: Chris Moody<chris(a)node-nine.com> > Subject: [syslog-ng] need help debugging some network received logs > that aren't writing to files > To: Syslog-ng users' and developers' mailing list > <syslog-ng(a)lists.balabit.hu> > Message-ID:<5303BEC2.9010405(a)node-nine.com> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Hello. > > First off, thanks a __TON__ for syslog-ng. I've sworn by this awesome > code for years now. I've built all sorts of logging infrastructure with > it. > > I seem to have hit on something though that's got me scratching my head > and lacking for explanation. Perhaps I've just been staring at it and > debugging it too long and am missing something obvious. -- Terry Slattery CCIE# 1026

1 0

log path with two destinations - flow control problem
by Tomáš Novosad 18 Feb '14

18 Feb '14

Hello to all i'm trying to set syslog to store sendmaill messages into PostgreSQL tables. One simple table (tableA) which stores all messages, and another table (tableB) which stores only some relevant iformation. All messages are parsed in my parser, then logged into tableA, then checked against the filter, an those messages which pass are logged into tableB. it all looks like this: log { source(pipe_load); filter(f_sendmail); #-- basic filter for sendmail program parser(maillog); #-- parse message details destination(maillog_base); #-- store message in tableA - basic info only filter(f_sendmail_from_or_to); #-- filter only messages which have "from" or "to" address defined destination(maillog_detail); #-- strore the deatiled info into tableB flags(flow-control); } filterinhg itself works fine, but i have problem with flow-control Messages are send from file (old archived maillog) into syslog pipe source. Problemm is, that alhough flow-control is enabled, messages get droped on the maillog_base destination. Im doing "cat old_maillog > syslog-pile" where old_maillog has approx 1 800 000 records, but only about 400 000 records gets logged into tableA. Flow control does not seem to work in this scenario. In case when i comment-out lines with filter(f_sendmail_from_or_to) and destination(maillog_detail) effectively disabling logging into the 2nd table, all messages from input file are processed and stored into tableA. The flow control works fine in this case. Is there something wrong with such filtering and storing only some messages in the 2nd destination? I also thought about having two different log path for each table, but in that case, the parsing of the message should be done twice, which i want to avoid. My question is, if is it OK to use filtering like in my example, - filter - than destination - then filter again - then another destination + flow control Many thanks in advance -- Tomáš Novosad LinuxBox.cz, s.r.o. 28. října 168, 709 00 Ostrava tel.: +420 591 166 221 mobil: +420 737 238 655 email: tomas.novosad(a)linuxbox.cz jabber: novosad(a)linuxbox.cz www.linuxbox.cz mobil servis: +420 737 238 656 email servis: servis(a)linuxbox.cz

3 3

source plugin udp not found
by Costa Farber 16 Feb '14

16 Feb '14

Hello, I have compiled syslog-ng ver 3.4.3 from source. syslog-ng -V syslog-ng 3.4.3 Installer-Version: 3.4.3 Revision: ssh+git://algernon@git.balabit /var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.4#no_branch#64d670f3cbfb90769f3c7f0fdd9c70bb9136ec5b Compile-Date: Feb 10 2014 15:39:36 Available-Modules: Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off Enable-Pcre: on When I run it: syslog-ng --cfgfile /opt/wix-log-service/configs/wix-log-service.conf I get: Error parsing source, source plugin udp not found in /opt/wix-log-service/configs/wix-log-service.conf at line 66, column 9: udp(port(514) so_rcvbuf(33554430)); What is wrong? Regards, Constantin -

2 3

Adding syslog-ng OSE 3.5.1 to buildroot
by Tamayo, Andres 16 Feb '14

16 Feb '14

Hello, I am working on adding syslog-ng OSE 3.5.1 to a cross-compiled image using buildroot. I was wondering if anybody knows if there is a repository where I can get the Config.in and syslog-ng.mk files needed to configure buildroot. Thank you for your help.

2 1

Multiple rewrite conditions under 3.4.x
by C. L. Martinez 14 Feb '14

14 Feb '14

Hi all, Is it possible to add multiple rewrite conditions in syslog-ng 3.4.x like this: rewrite r_rewrite_set{ set("myhost1", value("HOST") condition(program("myapplication1"))); set("myhost2", value("HOST") condition(program("myapplication2"))); set("myhost3", value("HOST") condition(program("myapplication3"))); set("myhost4", value("HOST") condition(program("myapplication4"))); }; ?? Is this a good option or maybe a performance penalty?? Thanks.

3 4

Re: [syslog-ng] [RFC] procfile Dezso Bagi
by Balazs Scheidler 14 Feb '14

14 Feb '14

Algernon is working on the debian subdir. Thanks for the feedback. Hope your experience is going to be bugfree :) On Feb 11, 2014 3:52 PM, "Valentijn Sessink" <valentyn(a)blub.net> wrote: > Hi, > > Finally found some time to test this. My setup is rather simple (I > think), the config-file now says: > destination d_syslogblock { pseudofile > ("/proc/net/xt_recent/syslogblock" template("+${usracct.device}\n")); > file("/var/log/syslogblock"); file("/tmp/blocker" > template("+${usracct.device}\n")); }; > > The "blocker" file is just for testing purposes. and "syslogblock" is to > log all evil to a regular log file as well. > > I got it to work rather easily and will report my findings. > > In the mean time, a question: is there anyone who guards the debian/ > directory inside the source tree? There seem to be some (mostly minor) > issues, and as I'm fixing them locally in order to get syslog-ng up and > running, I may as well send them upstream. If anyone cares ;-) that is. > > Best regards, > > Valentijn > > On 12/30/13 19:16, Balazs Scheidler wrote: > > I have now submitted a pull request to merge the "pseudofile" > > destination, I've now skipped the support for destination file > > templates, but that can be added later on when a more concrete usecase > > shows up. It already solves Valentijn's use-case, so it is useful as it > is. > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > >

1 0

insider 2014-02: FIPS; GSoC; Incubator; Conferences;
by Peter Czanik 13 Feb '14

13 Feb '14

Dear syslog-ng users, This is the 31th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news. FEATURED NEWS FIPS ---- Certain departments of the US government and those conducting business on certain regulated markets are required to use tools utilizing cryptography certified according to the FIPS 140-2 standard to handle sensitive data. As logs often contain sensitive information, this regulation can apply to the log management system, too. Starting with the 5.0.4 release, we are offering a version of syslog-ng Premium Edition to customers that is built and shipped with the OpenSSL FIPS Object Module and as thus usable in environments where such validation is required. (For more information about the OpenSSL FIPS Object Module, see _http://openssl.com/fips/_.) This is a separate version and the original version is available that uses the standard OpenSSL library. It is currently available on major Linux platforms and has the same functionality as the non-compliant version apart from a few limitations listed in the documentation. Please visit _http://www.balabit.com/lp/fips_if for more information. Google Summer of Code (GsoC) ---------------------------- Based on the success of last years syslog-ng Gsoc participation (in co-operation with the openSUSE project), this year we plan to participate in GsoC again with a number of syslog-ng related projects. The current list of ideas range from new sources and destination through under the hood improvements to integration with configuration management systems. The syslog-ng GsoC page is available at _https://github.com/balabit/syslog-ng/wiki/GSoC2014_ The ideas page is available at _https://github.com/balabit/syslog-ng/wiki/GSoC2014-Idea-&-Project-list_ We also created a central hub for syslog-ng OSE related material, as part of our preparations for Google Summer of Code and onwards: it is available at_http://syslog-ng.org/_ syslog-ng incubator updates --------------------------- The syslog-ng incubator is a collection of tools and modules which are not (yet) part of the official repository. Since last month there are many smaller changes and brand new features. The master branch goes now hand in hand with syslog-ng's master branch, which is currently 3.6 pre-alpha code. If you are working on syslog-ng 3.5, use the 3.5/master branch of syslog-ng-incubator. On the feature side the Lua destination was enhanced, a monitoring source was added, together with a Graphite output. Among the Lua examples one can also find a script to push logs into ElasticSearch. To try the new features, check out _https://github.com/balabit/syslog-ng-incubator_from git. You can read more about the Lua destination at _https://talien.blogs.balabit.com/2014/02/lua-the-undiscovered-country/_ syslog-ng at conferences ------------------------ In the past few weeks we visited three conferences. While there we talked about syslog-ng to many people, some during organized presentations, and many more between presentations and social events of the conferences. A look back at FOSDEM and Config Management Camp: _https://czanik.blogs.balabit.com/2014/02/fosdem-2014-and-config-management… The next confirmed event is: Open Source Data Center Conference, 8-10 April, Berlin, Germany: _http://www.netways.de/osdc/_, Giving a talk titled "Monitoring with syslog-ng, Riemann and Kibana" There will be two developers present, so if anyone attends, and has Incubator, Lua, Riemann, etc. related questions, we'll be happy to answer. SHORT NEWS: * Measuring log collection performance: _https://pzolee.blogs.balabit.com/2014/01/measuring-log-collection-performa… NEW RELEASES * syslog-ng 5.0.4: _https://lists.balabit.hu/pipermail/syslog-ng-announce/2014-February/000180.html_ Your feedback and news tips about the next issue is welcome at _documentation(a)balabit.com <mailto:documentation@balabit.com>_To read this newsletter on-line, visit: _http://insider.blogs.balabit.com/_ -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

debian init files gone wrong? (Due to git reorg?)
by Valentijn Sessink 12 Feb '14

12 Feb '14

Hi, Building syslog-ng from source posed a few problems, one of them being debian/syslog-ng.init that missed a crucial option: It's function syslogng_start() in Git currently says: start-stop-daemon --start --quiet --exec "$SYSLOGNG" --pidfile "$PIDFILE" -- $SYSLOGNG_OPTS \ ... while the Debian source I viewed says: start-stop-daemon --start --quiet --exec "$SYSLOGNG" --pidfile "$PIDFILE" -- -p "$PIDFILE" $SYSLOGNG_OPTS \ As the Git repository tells the file is from 2007, I suspect the git reorganisation from november to have caused this - but I'm not sure. Also, the Debian init files seem to be much more up to date than this file - the file in Git still starts with the "skeleton" stuff. Best regards, Valentijn

2 3

[RFC] procfile destination
by Balazs Scheidler 11 Feb '14

11 Feb '14

Hi, Based on a request I have now created a new destination driver, called "procfile". A working PoC is available on this branch: https://github.com/balabit/syslog-ng/tree/3.6/f/procfile It can be used to change kernel settings dynamically, based on a trigger within syslog-ng. The difference between procfile() and file(): - no template in the name of the file - processing is performed immediately as read by the source, no queueing is performed (e.g. no loss is possible, but takes CPU time from the source) - opens and closes the destination file after each and every write, writes to offset zero e.g. it doesn't append further data. I'd be grateful if you could provide feedback around these questions or the feature in general: - name of the driver, right now it is called procfile as it was the original use-case, however it works with stuff in /dev too, and might work with simple named pipes. - right now the file must exist and is not created (e.g. it really requires an existing file); do we need that? - do we need append mode in some cases - whether we need append mode (if we think of general files, this would probably be needed) Thanks in advance, Bazsi

5 6