June 2011 - syslog-ng - lists.balabit.hu

[PATCH (3.4)] affile: Fix NULL dereference when opening the destination fails.
by Gergely Nagy 23 Jun '11

23 Jun '11

Viktor Tusa discovered an error in the affile destination driver: if opening the destination failed (for example, due to insufficient permissions), next was set to NULL, and later on, at the end of affile_dd_open_writer() tried to dereferrence it. A simple workaround is to return NULL in case next is NULL. Reported-by: Viktor Tusa <tusa(a)balabit.hu> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/affile/affile.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/modules/affile/affile.c b/modules/affile/affile.c index 87767aa..57ccb7a 100644 --- a/modules/affile/affile.c +++ b/modules/affile/affile.c @@ -1052,6 +1052,9 @@ affile_dd_open_writer(gpointer args[]) } } } + if (!next) + return NULL; + next->queue_pending = TRUE; /* we're returning a reference */ return &next->super; -- 1.7.2.5

1 1

Twelve days of Log: A short animation
by Gergely Nagy 23 Jun '11

23 Jun '11

Hi! A day or two ago, I had a rush of inspiration, and ended up doing something fairly pointless (yet, at least in my opinion, funny): I took twelve days worth of my logs, and using Gource[0], I built a little animation out of them: http://vimeo.com/25483686 There's still a lot of knobs to turn and tweak, but I thought I'd share a short demo, before I finish up this part of my log visualisation project. The fun thing is, that one does not need anything else to make these kind of animations, but syslog-ng and gource. Pretty much any recent version (possibly even 3.0) should work out of the box! I wrote a little HOWTO on it, which is available on my blog at http://algernon.blogs.balabit.com/2011/06/twelve-days-of-log/. 0: https://code.google.com/p/gource/ -- |8]

2 2

Turning off TCP connection logs
by JP Vossen 22 Jun '11

22 Jun '11

This is probably totally obvious, but it's not jumping out at me in the admin guide or Google. How can I turn off the TCP connection logs like: Jun 22 01:40:47 centos5 syslog-ng[31034]: Syslog connection accepted; fd='10', client='AF_INET(192.168.1.148:46400)', local='AF_INET(0.0.0.0:514)' [...] Jun 22 01:40:47 centos5 syslog-ng[31034]: Syslog connection closed; fd='10', client='AF_INET(192.168.1.148:46400)', local='AF_INET(0.0.0.0:514)' I assume these are coming from the syslog-ng internal() source, but I'm not seeing how to turn only those off without affecting other internal syslog-ng messages. What am I being totally dense about and missing? Thanks, JP ----------------------------|:::======|------------------------------- JP Vossen, CISSP |:::======| http://bashcookbook.com/ My Account, My Opinions |=========| http://www.jpsdomain.org/ ----------------------------|=========|------------------------------- "Microsoft Tax" = the additional hardware & yearly fees for the add-on software required to protect Windows from its own poorly designed and implemented self, while the overhead incidentally flattens Moore's Law.

2 1

New on syslog-ng
by Josu Lazkano 22 Jun '11

22 Jun '11

Hello list, I am new on this list. I start working on a IT department and I need to update and check the actual syslog system. We have more than 50 Ubuntu servers, locals and remotes, on each servers there is rsyslog installed and configured this way: $ cat /etc/rsyslog.d/99-rsyslog.conf auth.*,authpriv.* @logserver kern.warn @logserver kern.err @logserver mail.* @logserver There is server (logserver) with syslog-ng to manage and save all logs. It receive on UDP port without secure, this the source section on syslog-ng configuration: source s_all { internal(); unix-stream("/dev/log"); file("/proc/kmsg" log_prefix("kernel: ")); }; source logs_externs{ udp(); }; I have some questions about how to secure it: 1. How can I secure the logs on the net? I must use TCP to secure? 2. I have lot of data (5-10GB at week) to store, which is the best method to manage it? Logrotate? Scripts? I want to move logs to NAS monthly. 3. Is it a good idea to make files for each host and services? Something like: /var/log/host1/auth.log, /var/log/host2/auth.log, /var/log/host1/mail.log, /var/log/host2/mail.log These are my newby questions, thanks for all your help and best regards.

3 4

Having some compile errors on ubuntu lucid-amd64
by Mike Gracy 21 Jun '11

21 Jun '11

I get the same error when compiling the latest stable also: libtool: link: gcc -std=gnu99 -Wno-pointer-sign -g -O2 -Wall -o .libs/pdbtool pdbtool-pdbtool.o -Wl,--export-dynamic -pthread libsyslog-ng-patterndb.a ../../lib/.libs/libsyslog-ng.so -lnsl /usr/lib/ libgmodule-2.0.so /usr/lib/libgthread-2.0.so -lrt /usr/lib/libglib-2.0.so/usr/lib/libevtlog.so -ldl -lssl -lcrypto -pthread libsyslog-ng-patterndb.a(libsyslog_ng_patterndb_a-patterndb.o): In function `pattern_db_set_time': /home/mgracy/ftp/syslog-ng/syslog-ng-3.2.3/modules/dbparser/patterndb.c:1379: undefined reference to `evt_tag_long' libsyslog-ng-patterndb.a(libsyslog_ng_patterndb_a-patterndb.o): In function `pattern_db_timer_tick': /home/mgracy/ftp/syslog-ng/syslog-ng-3.2.3/modules/dbparser/patterndb.c:1352: undefined reference to `evt_tag_long' libsyslog-ng-patterndb.a(libsyslog_ng_patterndb_a-patterndb.o): In function `pattern_db_expire_entry': /home/mgracy/ftp/syslog-ng/syslog-ng-3.2.3/modules/dbparser/patterndb.c:1315: undefined reference to `evt_tag_long' collect2: ld returned 1 exit status make[5]: *** [pdbtool] Error 1 make[5]: Leaving directory `/home/mgracy/ftp/syslog-ng/syslog-ng-3.2.3/modules/dbparser' make[4]: *** [all-recursive] Error 1 make[4]: Leaving directory `/home/mgracy/ftp/syslog-ng/syslog-ng-3.2.3/modules/dbparser' make[3]: *** [all] Error 2 make[3]: Leaving directory `/home/mgracy/ftp/syslog-ng/syslog-ng-3.2.3/modules/dbparser' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/home/mgracy/ftp/syslog-ng/syslog-ng-3.2.3/modules' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/home/mgracy/ftp/syslog-ng/syslog-ng-3.2.3' make: *** [all] Error 2 I am trying to compile to a deb. I didn't have a problem with libevtlog.

2 2

[RFC]: MongoDB destination plans (for 3.4 and beyond)
by Gergely Nagy 21 Jun '11

21 Jun '11

Hi! I've been working on a few mongodb destination related features recently, and I thought I'll ask for comments here, before I proceed further, to see if my ideas can be improved, and if there's anyone actually interested in the stuff I play with (mostly out of sheer curiosity; I'll scratch my own itches even if noone else has similar needs :P). Before I go further, let me introduce the current mongodb destination features, available in the syslog-ng 3.3 branch: * We can connect to a single MongoDB server per destination, which we expect to be the master (or a standalone server). * We can send logs to it, structured in various interesting ways via value-pairs(). ...aaand that's about it. Pretty bare bones, but it's enough for a lot of stuff. Now, I plan to extend this in two ways: first, it will be possible to connect to a ReplicaSet, which is basically a set of MongoDB servers that replicate from a master. The advantage of this, is that if the master goes down, one of the secondaries will automatically take over, and the mongodb driver will reconnect to it automatically aswell. In case that fails too, the destination driver will fall back to queuing within syslog-ng, and retry after a configured interval. Another advantage will be the use of "safe mode", which when turned on, will verify that the message could be inserted into a MongoDB collection, and it will not be ACKed on the syslog-ng side until it is. After a number of retries, it can be dropped along with an appropriate log message, so that these won't fill up the queue. (This, of course, would be optional, with never dropping messages by default, unless the internal queue is full.) So we could have a destination configured like this: d_mongo { mongodb( servers("10.0.10.1:27017" "10.0.10.2:27017" "10.0.10.3:27017") safe-mode(on) ); }; This would connect to 10.0.10.1 on port 27017 by default, and if that becomes inaccessible, it'd retry with .2 and .3, in that order. If any of them listed other servers as part of their replicaset, the driver would retry with those aswell. It would also turn on safe-mode, which are a bunch of extra checks to ensure that data arrived to MongoDB safe and sound. We'd get more reliability this way, and with safe-mode, better data safety aswell. Now, the question is: is there anything else that may be worth adding? If anyone here used MongoDB, is there something else you'd like to see added to the mongodb destination? (I could add GridFS support aswell, but so far, I haven't found an acceptable use-case for that yet.) -- |8]

2 2

rotating log files on every 5 minutes
by Omer Faruk SEN 21 Jun '11

21 Jun '11

Hi Is it possible to rotate logs for every 5 minutes For example instead of using ${HOST}_${R_YEAR}${R_MONTH}${R_DAY}${R_HOUR}${R_MIN}.log I want to use ${HOST}_${R_YEAR}${R_MONTH}${R_DAY}${R_HOUR}${R_MIN%5}.log (which rotates every 5 minues ) Regards.

2 1

syslog-ng & AppArmor
by Peter Czanik 21 Jun '11

21 Jun '11

Hello, As announced a few weeks ago, the FAQ for recent syslog-ng versions is now maintained within BalaBit: http://czanik.blogs.balabit.com/2011/06/syslog-ng-faq-moved-and-updated/ Today I added a new entry about AppArmor: http://www.balabit.com/wiki/syslog-ng-faq-apparmor It was tested on the latest openSUSE, so please let me know, if you have different experiences on other distributions, or any other ideas to extend the article! Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

syslog-ng logging in timezone?
by Matthias Runge 20 Jun '11

20 Jun '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello list, can please someone verify this: ssh'ing to a box and disconnecting results in something like this: Jun 15 08:20:45 mrungexp userhelper[4063]: running '/usr/sbin/hddtemp -n - -q /dev/sda' with root privileges on behalf of 'mrunge' Jun 15 06:57:34 mrungexp sshd[12047]: Connection closed by 79.204.170.139 Jun 15 09:00:27 mrungexp su: pam_systemd(su-l:session): Moving new user session for mrunge into control group /user/mrunge/1. (taken from /var/log/secure) Time offset of disconnect messages may vary. There are two bugs reported: https://bugzilla.redhat.com/show_bug.cgi?id=231326 https://bugzilla.redhat.com/show_bug.cgi?id=712151 - -- Matthias Runge <mrunge(a)matthias-runge.de> <mrunge(a)fedoraproject.org> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJN+GH7AAoJEOnz8qQwcaIWq2oIAJxQz9ZzvGogswicJCjDqVCf ei0Gj48BgpFE/47bFnexoV45rU0ct/TWSWvoIENoKL5dkusIoSjpT8LGbUrEDlGx sAQbDAXQf5XKcUESlWC0HEQmGA1u/Qww7ETiDTfYS8Sa/QCQCJYjqNrKWlrr/UnN qikBDGcjGoqbQLkPbXvLbx9BCZIHN11DkAvXTe2iTAek8JlZmgASF1a5nHYKizFQ 1uG2L9ZRy00HmGTSy9wYPWgjrtHVebVk6O3HSNOmbdXP+Dgv1uoyjE2qe5VCd8M/ l11LtzUEMzoaeaSK2XIesMUNMGJJ5vIBAi8L/Gs21k4QtGgYohjSmJs1D4mRq6Y= =Oldo -----END PGP SIGNATURE-----

1 1

quick question on host() in syslog-ng 3.1.4
by Mishou Michael 16 Jun '11

16 Jun '11

Folks, I want host() to do a literal, exact string match instead of regex. For message() you can do this: message("foo" type("string")) and get an exact string match. Does this type() and flags() (so you can do substrings) also work on the host() operator? This is on syslog-ng 3.1.4. I finally moved off of Solaris (easier to just get a Linux box up and running it turned out) and am attempting to modify the insane config I have to be more performance friendly. I've got about 300 host() matches going on, dumping hosts to various folder structures, and figured that if I could get it to do an exact match instead of regex it would be that much more efficient. Of course, I could always do host("^192\.168\.1\.1$") but that seems kind of silly when I don't need regex at all. FWIW, syslog-ng -s doesn't complain about this in the config, and it starts/runs fine, I just can't tell if it's still using regex or not. Thanks, --Mike

2 2