August 2012 - syslog-ng - lists.balabit.hu

3.4 alpha: Deadlock in MongoDB plugin?
by Conrad Hoffmann 14 Aug '12

14 Aug '12

Hi, it's me bugging again :) (btw, I'm on the list, no need to CC me) In my quest to test my patches with the actual 3.4 version syslog, I noticed today that the mongodb plugin doesn't seem to work very at all on todays master branch (vanilla, without any patches). No log messages are ever written to the DB, and after a few messages syslog locks up. The mongodb threads seems to idle in its normal event loop while the main thread seems to be stuck in some afmongodb-related lock. Once again, I need to go to bed before I can take a closer look, but I just thought I'd let you know. Find here a backtrace of the main thread in the state it is stuck in: http://pastebin.com/QZSam3CH At the bottom is also the backtrace for the mongodb thread, as far as I can tell it looks as always. Good night for now, Conrad

2 4

[PATCH] [0/1] Allow mongodb destination to use Unix sockets
by Conrad Hoffmann 12 Aug '12

12 Aug '12

As promised this is actually the second part of the previous patch, but since the other one was for a different repo I decided to keep them seperate. Yet, this one depends on the former. This patch is also for syslog-ng-3.4, master branch. Like I mentioned, I already used Unix sockets successfully, though I must add that this was with a modified version of the path for syslog-ng-3.3, because I had trouble getting 3.4 to work at all. I will take another look at this tomorrow, but maybe you can give some feedback already? Conrad Hoffmann (1): Allow Unix sockets for MongoDB connection. modules/afmongodb/afmongodb-grammar.ym | 2 ++ modules/afmongodb/afmongodb-parser.c | 1 + modules/afmongodb/afmongodb.c | 25 +++++++++++++++++++++---- modules/afmongodb/afmongodb.h | 1 + 4 files changed, 25 insertions(+), 4 deletions(-) -- 1.7.11.4

3 4

[PATCH] tfhash: New template function to calculate the SHA1 of its input
by Peter Gyongyosi 12 Aug '12

12 Aug '12

tfhash: New template function to calculate the SHA1 of its input This patch adds a new template function called $(hash) which can be used to calculate the SHA1 hash of its inputs. As it uses the SHA1() function from OpenSSL, it is only available if syslog-ng is compiled using --enable-ssl. One use case for it can be when you'd like to anonymize sensitive parts of the log message (eg. the user name) you parsed out using patterndb before storing or forwarding it for further analysis but would like to retain the ability to correlate messages along this value. Beware though that such simple anonymization can be broken if there's enough data available, see http://arxiv.org/pdf/cs/0610105v2.pdf for references and examples. Thanks for the original idea to Martin Grauel. Signed-off-by: Peter Gyongyosi <gyp(a)balabit.hu> --- [see the patch itself at https://github.com/gyp/syslog-ng-3.4/commit/5c6e97d94694eef2aa76fa61a1c8928… ] <https://github.com/gyp/syslog-ng-3.4/commit/5c6e97d94694eef2aa76fa61a1c8928…>

5 12

lib/ivykis submodule pointing to stale commit?
by Conrad Hoffmann 12 Aug '12

12 Aug '12

Hi folks, apparently the commit that lib/ivykis submodule is set to (syslog-ng-3.4, master branch) does not exist in the tree anymore? GitHub actually still shows it, but it's not checked out: (See lines beginning with *) dionysos:~/tmp $>git clone https://github.com/balabit/syslog-ng-3.4.git Cloning into 'syslog-ng-3.4'... remote: Counting objects: 13157, done. remote: Compressing objects: 100% (2834/2834), done. remote: Total 13157 (delta 10256), reused 13100 (delta 10213) Receiving objects: 100% (13157/13157), 3.18 MiB | 1.12 MiB/s, done. Resolving deltas: 100% (10256/10256), done. dionysos:~/tmp $>cd syslog-ng-3.4/ dionysos:~/tmp/syslog-ng-3.4 $>git submodule init Submodule 'lib/ivykis' (git://github.com/buytenh/ivykis.git) registered for path 'lib/ivykis' Submodule 'modules/afmongodb/libmongo-client' (git://git.balabit.hu/bazsi/libmongo-client.git) registered for path 'modules/afmongodb/libmongo-client' dionysos:~/tmp/syslog-ng-3.4 $>git submodule update Cloning into 'lib/ivykis'... remote: Counting objects: 3678, done. remote: Compressing objects: 100% (917/917), done. remote: Total 3678 (delta 2764), reused 3635 (delta 2732) Receiving objects: 100% (3678/3678), 1.06 MiB | 682 KiB/s, done. Resolving deltas: 100% (2764/2764), done. *fatal: reference is not a tree: 73c0c27a19755f700b268081a5a59acdd04b421e Cloning into 'modules/afmongodb/libmongo-client'... remote: Generating pack... remote: Done counting 4658 objects. remote: Deltifying 4658 objects... remote: 100% (4658/4658) done remote: Total 4658 (delta 3123), reused 4585 (delta 3053) Receiving objects: 100% (4658/4658), 702.36 KiB | 1.17 MiB/s, done. Resolving deltas: 100% (3123/3123), done. Submodule path 'modules/afmongodb/libmongo-client': checked out '58f3814cad94bcd78216c7ac971c8435d17a9242' *Unable to checkout '73c0c27a19755f700b268081a5a59acdd04b421e' in submodule path 'lib/ivykis' I can't find a similar in the tree neither. Any ideas? Regards, Conrad

3 2

Re: [syslog-ng] Issue about using perl script in Receiving messages from external applications
by Evan Rempel 10 Aug '12

10 Aug '12

You are probably suffering from perl's buffering. I forget the exact syntax right now but a quick google of disabling Perl auto flush should give you what you need. Jian Zhu <zhujian0805(a)gmail.com> wrote: Hello, Could anyone please help me on this below issue? my syslog-ng.conf file look like below: The test.sh script work fine, I can see it's output in the test.sh.log file. But the test.pl<http://test.pl> doesn't work. the test.sh and test.pl<http://test.pl> have the same function of printing something to STDOUT. ------------------------------ ------------------------------------------------------------------------------------------------------------------- @version: 3.0 #Default configuration file for syslog-ng. # # For a description of syslog-ng configuration file directives, please read # the syslog-ng Administrator’s guide at: # # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html # options { # Number of syslog lines stored in memory before being written to files flush_lines (0); log_fifo_size (2048); create_dirs (yes); perm (0640); dir_perm (0750); }; source ppp { program("/root/perl/test.pl<http://test.pl>"); }; destination ppp { file ("/var/log/syslog-ng/$HOST/perl.pl.log" create_dirs(yes)); }; log { source(ppp); destination(ppp); }; source sss { program("/root/shell/test.sh"); }; destination sss { file ("/var/log/syslog-ng/$HOST/test.sh.log" create_dirs(yes)); }; log { source(sss); destination(sss); }; ------------------------------------------------------------------------------------------------------------------------------------------------- -- Thanks & Best Regards James Zhu Mobile: +86 186 8482 2818 Email: zhujian0805(a)gmail.com<mailto:zhujian0805@gmail.com>

3 3

[Bug 178] New: Spoofed source address bug introduced in 3.3.5
by bugzilla＠bugzilla.balabit.com 10 Aug '12

10 Aug '12

https://bugzilla.balabit.com/show_bug.cgi?id=178 Summary: Spoofed source address bug introduced in 3.3.5 Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Solaris Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: marvin.nipper(a)stream.com Type of the Report: regression Estimated Hours: 0.0 Hi. Sorry if this is not the right way to do this. There was an email thread regarding this on 4/30, but I figured that the right way to do this way to simply open an official bug report. As this is my 1st one for syslog-ng, apologies if I screw it up! This all pertains to 3.3.5, on Solaris 10 x86. Per my original email, changes made between 3.3.4 and 3.3.5 have caused a situation wherein forwarded UDP packets, with spoofed source addressing, all revert to null addresses, the instant that a HUP is issued against syslog-ng (e.g. during log rotation). Also, as per that email, with Gergely's excellent (and educated) guess, I backed out the patch noted here: http://git.madhouse-project.org/debian/syslog-ng/patch/?id=a898014482f733e9…, and the result was that the null addressing issue was resolved. Obviously, as there were (no doubt) other reasons for that patch, I'm just wanting to be sure that this problem doesn't get lost in the shuffle, and that the patch gets reworked (to achieve its originally desired results, while managing to also resolve the null address problem that it caused with spoofing). Let me know if you need any further information regarding this problem. And, as always, I'm MORE than happy to be a test guinea pig, if need be. THANKS. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 21

syslog-ng upgrade
by James McDonald 10 Aug '12

10 Aug '12

Does anyone know what needs to be done to successfully upgrade from 4.1 to 4.2 (F2)? I tried simply stopping the service and running 'rpm -ivh syslog-ng..rpm -force' but it did not complete successfully. I added the -force command because it was complaining of file conflicts. Balabit has requested I upgrade to this version, as they try to help me figure out my other problem I'm having with the syslog-ng service, but I was unable to find any documentation or get a response from Balabit yet. I thought I would see if anyone here had performed the upgrade and knew what needed to be done. Here is the output I got when I tried to run it: rpm -ivh syslog-ng-premium-edition-4.2.3-1.rhel5.i386.r pm --force warning: syslog-ng-premium-edition-4.2.3-1.rhel5.i386.rpm: Header V4 DSA signature: NOKEY, key ID 2aa28252 Preparing... ########################################### [100%] *IMPORTANT NOTICE - NEW LICENSE REQUIRED FOR MULTI-CPU OPERATION* Please note that the 4.2 version of syslog-ng Premium Edition includes multi-threaded processing with a 2 CPU core limit. If you are upgrading your syslog-ng server from version 4.1 to 4.2 and your syslog-ng deployment requires more than 2 CPU cores, you will need to obtain a new license file, free of charge, from the BalaBit sales department. Please contact the sales department at sales(a)balabit.com. Shutting down kernel logger: [FAILED] Shutting down system logger: [FAILED] Stopping syslog-ng: 1:syslog-ng-premium-editiwarning: /opt/syslog-ng/etc/syslog-ng.conf created as /opt/syslog-ng/etc/syslog-ng.conf.rpmnew ########################################### [100%] Starting syslog-ng: WARNING: window sizing for network sources has changed in syslog-ng 4.1. The window size is the value of the log_iw_size() option divided by the value of max_connections(). If log_iw_size()/max_connections() is lower than 100, it is automatically increased to 100. Adjust the log_fifo_size() option if needed to avoid message loss.; orig_window_size='1', new_window_size='100', min_log_fifo_size='100000' [ OK ] Thanks, James McDonald System Administrator RHCSA, Security+ jmcdonald(a)lce.com

3 2

Issue about using perl script in Receiving messages from external applications
by Jian Zhu 10 Aug '12

10 Aug '12

Hello, Could anyone please help me on this below issue? my syslog-ng.conf file look like below: The test.sh script work fine, I can see it's output in the test.sh.log file. But the test.pl doesn't work. the test.sh and test.pl have the same function of printing something to STDOUT. ------------------------------ ------------------------------------------------------------------------------------------------------------------- @version: 3.0 #Default configuration file for syslog-ng. # # For a description of syslog-ng configuration file directives, please read # the syslog-ng Administrator’s guide at: # # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html # options { # Number of syslog lines stored in memory before being written to files flush_lines (0); log_fifo_size (2048); create_dirs (yes); perm (0640); dir_perm (0750); }; source ppp { program("/root/perl/test.pl"); }; destination ppp { file ("/var/log/syslog-ng/$HOST/perl.pl.log" create_dirs(yes)); }; log { source(ppp); destination(ppp); }; source sss { program("/root/shell/test.sh"); }; destination sss { file ("/var/log/syslog-ng/$HOST/test.sh.log" create_dirs(yes)); }; log { source(sss); destination(sss); }; ------------------------------------------------------------------------------------------------------------------------------------------------- -- *Thanks & Best Regards* *James Zhu* *Mobile**:* *+86 18**6 8482 2818* *Email: *zhujian0805(a)gmail.com

1 0

Query On configuring Centralized Audit server with Auditd daemon
by Koresh... 09 Aug '12

09 Aug '12

Hi Folks, Need your help ! Want to configure a centralized Audit server (Currently the centralized server is running Octopussy Web interface, which receives logs from remote hosts by Rsyslog ). The challenge and confusion here is .. all my linux clients are configured with syslog-ng and the daemon is sending all the system logs and kernel logs like messages,secure,cron logs etc ... with out any trouble. The problem is the syslog-ng daemon is not able to send the auidtd logs (/var/log/audit.log) to the Rsyslog server, Hence request your help to guide me how to setup the syslog-ng to forward the audit.log to the remote Rsyslog server. It would be great if i can get client side and server side configuration guidelines. -- Thanks in Advance - Koresh

3 4

syslog-ng service not starting
by James McDonald 08 Aug '12

08 Aug '12

I returned from being out of the office for week and realized something was "off" with syslog-ng. I was unable to restart the service. I did a 'service syslog-ng status' and it says it's stopped. When I try 'service syslog-ng start' I get the following error: Error binding socket; addr='AF_INET(0.0.0.0:443)', error='Address already in use (98)' Error initializing source driver; source='inputs', id='inputs#3' Error initializing message pipeline; There is only instance of syslog-ng installed on the server but I did a reboot to try and clear any conflicting settings, but I still get the same error. Can someone help me clear this error and get the service started again? Thanks, James McDonald System Administrator RHCSA, Security+ jmcdonald(a)lce.com

5 8