- syslog-ng - lists.balabit.hu

password mailing-list
by laugello 28 Oct '10

28 Oct '10

Hello Sorry of the o-topic I forgot password of the mailing-list How I can recover it tanks Luigi -- Augello Luigi Amministratore di Sistema Poli didattici di Agrigento, Caltanissetta e Trapani Università degli Studi di Palermo tel 093420928 VoIP 09123865802

2 1

AppArmor
by Peter Czanik 26 Oct '10

26 Oct '10

Hello, I wonder, if anyone here uses syslog-ng together with AppArmor. SCL calls external applications, or using program() in syslog-ng.conf also calls an external app. I spent quite some time with AppArmor documentation, tried logprof many times, asked on openSUSE mailing lists, but no luck. I could not execute external apps when AppArmor was enabled. Does anybody has a success story here? I'd be very interested in your AppArmor profile for syslog-ng! BTW: if you use openSUSE, you could test my 3.2beta1 packages. See my blog for details! Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

Should configure error out if lex/flex is missing?
by Lars Kellogg-Stedman 19 Oct '10

19 Oct '10

After successfully running ./configure for syslog-ng 3.2beta1, the build immediately failed because I didn't have flex installed. I see that configure is already calling AM_PROG_LEX...so maybe this is just the expected behavior? After installing flex the build completed successfully.

2 3

[Bug 52] CAP_DAC_OVERRIDE is missing when capabilities are enabled, unable to write to files created with the owner() option.
by bugzilla＠bugzilla.balabit.com 19 Oct '10

19 Oct '10

https://bugzilla.balabit.com/show_bug.cgi?id=52 Balazs Scheidler <bazsi(a)balabit.hu> changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution| |FIXED Status|NEW |RESOLVED --- Comment #4 from Balazs Scheidler <bazsi(a)balabit.hu> 2010-10-19 17:23:38 --- I've integrated this as well, thanks Zbigniew commit 967b1720c8487f3cbe49292c7e5ed3b871ab2de5 Author: Zbigniew Krzystolik <zbyniu(a)pld-linux.org> Date: Tue Oct 19 17:21:23 2010 +0200 fix capability in the file() destination fixes for settings like owner(), group(), perm() and analogous dir_* if create_dirs=yes for destination files - CAP_DAC_OVERRIDE force changes if parent dir has 000 perm, ie vservers (is inerhitted for dir_* too) - CAP_CHOWN - needed if dir_owner() or dir_group() are in use - CAP_FOWNER - to force chmod() for dirs with owner != root -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 0

Buffering AF_UNIX Destination, Batch Post Processing Messages
by Matthew Hall 19 Oct '10

19 Oct '10

Hello All, I want to configure an AF_UNIX SOCK_DGRAM syslog-ng destination which sends certain log messages to an external program for further processing and analysis. This program should batch up the messages into 60 second batches for processing. Currently I am running into an architectural challenge in how I should process the 60 second batch without slowing down the select which is collecting the messages from the destination. In the past when creating a similar kind of application in Java I handled this by creating a huge dynamic array to store objects creating from each incoming message, then passed the array reference to another background thread for processing, and began building a new array in the select thread. Currently I am trying to solve this same basic problem in Perl, which has poor threading support. I am investigating a few different options: * use threads anyway-- not recommended by more expert Perl devs I asked * prefork a process which listens to the AF_UNIX from syslog-ng, and writes to some kind of buffered non blocking pipe with a really big buffer-- not sure if such a pipe device actually exists, many pipes block * postfork a worker process which handles the 60 second batch-- problem here is that you want to have a whole lot of long term state data which is maintained between batches to help separate the needles from the haystacks, and you could get weird behavior on the duplicated FDs that are still being select()ed in the parent process which are copied into the child. * use some kind of message or job queue to copy things from a producer process to a consumer process-- gearman, theschwartz, beanstalk, rabbitmq, activemq, and poe::component::mq have been suggested-- this would probably cause a lot of context switching and unwanted buffer copies * see if there is something existing in syslog-ng that can help with this situation. can it somehow be convinced to buffer things internally for my process when my process is busy on a 60 second batch, or send in 60 second batch, etc. / whatever other clever people can dream up? Is this a problem other people have dealt with before? What did you do about this one? I want to get this right and avoid making a big mess or reinventing the wheel. Matthew.

4 18

Re: [syslog-ng] [patch] fix linking issue in 3.2beta1
by Guillaume Rousse 19 Oct '10

19 Oct '10

Le 18/10/2010 10:18, Balazs Scheidler a écrit : > Hi, > > Thanks for the patch, but I it was my intent not to link the plugins > against GLIB & other dependencies. > > Do you have an error message that this patch actually fixes? Build failure, when -Wl,--no-undefined flag is used. > The reason that the plugins are not linked against GLIB and other is > that I have to support an almost static syslog-ng, in which case GLib is > statically linked into the libsyslog-ng.so shared object and plugins > pull it from there. I may indeed disable this flag here, it's primary target is shared library, not plugin. -- BOFH excuse #107: The keyboard isn't plugged in

2 1

pdbtool patternize update and my syslog-ng 3.2 branch
by Peter Gyongyosi 19 Oct '10

19 Oct '10

Hello, As the patterndb project is starting to gain some momentum I thought it'd be the right time to port my patternize tool to the new, plugin-based 3.2 codebase as the first step towards getting it integrated --- and to be able to use the fancy new pdbtool features along with patternize. To those who are unfamiliar with it, patternize is an addition to pdbtool that makes it possible to automatically generate a pattern database from raw logs using statistical data clustering methods: you can read more about it in this blog post: http://gyp.blogs.balabit.com/2010/01/introducing-pdbtool-patternize/ Besides the port to the new codebase, it's received some fixes and new features since my original post: * multiple small internal bugfixes to get rid of weird errors * added the option "/--named-parsers/" that names the found @ESTRING@s like "/.dict.string0,1,2,3.../" * Balint Kovacs has sent three contributions: added support for reading the logfile from the standard input, escaping special characters in the output and putting examples in the XML that can be used for self-testing. It can be found in my public syslog-ng 3.2 tree: http://git.balabit.hu/?p=gyp/syslog-ng-3.2.git;a=summary If you're already using it (I've received some feedback so I guess some of you do), please note that most probably this 3.2-based branch will get the fixes and new features from now on. It's only received a basic sanity check and the unit tests do pass, so as usual, handle it with care and all feedback is welcome. greets, Peter ps.: the branch also contains a patch that fixes a wrong section name in pdbtool's man page and I'll try to update the whole manpage a bit when adding a section for patternize soon -- Bazsi, you might want to pull those to the mainline.

6 24

decoding messages from sockd (SOCKS proxy)
by Matthew Hall 18 Oct '10

18 Oct '10

Hello all, I am running into some headaches with the poor formatting of sockd messages. How should I decode messages like this? Note I have not applied XML escapes to these yet as that's hard to read but I will do so when inserting them into a patterndb to prevent parse errors. Every message in this group begins with this string on one line: sockd[@NUMBER:pid:@]: @ESTRING:action::@ @ESTRING:phase::@ @IPv4:src:@.@NUMBER:srcport:@ -> Then there are a few different endings which happen in some messages that are giving me problems to decode. Here are three examples from my collected logs: smarthost.company.com.25 host.team.division.company.com.18050: invalid address: 0.0.0.0.18050 company.com.443: Connection reset by peer I am having a hard time figuring out how to break these up into domain name (src / dst as appropriate) and port (srcport / dstport). My best thought so far was to detect this and rewrite them using PCRE before applying patterndb matching. I could find the .[0-9]+ and replace with :\1, then I have the port delimited with ':' and I can pull it apart using: @ESTRING:src::@@NUMBER:srcport:@ Is it possible to do PCRE replacement using backreferences? Or is there another way to get this to work? Thanks, Matthew.

2 2

pattern matching on xxx#
by Bill Anderson 18 Oct '10

18 Oct '10

I have hostnames of the format xxxx# such as host1, hostb1, hostc1. I need to split that into two fields such as (host,1). Unfortunately, since @@ escapes the @ and STRING and it's followers ALSO match digits, I've not found the obvious means to get that out. Conceptually something like @LETTER:host.name@@NUMBER:host.id@ woudl do it, save that LETTER doesn't exist and @@ escapes. The end goal is as follows (pseudo-code): I need to have a destination for each (HOST). For example all files from hosta## go to /var/log/hosta/ and entries for hostb## go to /var/log/hostb/ I suppose I *might* be able to do a rewrite to add say, a hyphen, and then use csv-parser, but we're talking some heavy traffic and I suspect that doing rewrites on that much traffic would be a performance killer. I'm open to suggestions (that don't involve changing server names, preferably ;) ) as to how to accomplish this. Cheers, Bill

4 10

[Bug 95] New: Missing capabilities support for unix-stream() source
by bugzilla＠bugzilla.balabit.com 18 Oct '10

18 Oct '10

https://bugzilla.balabit.com/show_bug.cgi?id=95 Summary: Missing capabilities support for unix-stream() source Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: zbyniu(a)pld-linux.org Type of the Report: --- Estimated Hours: 0.0 Operations on unix-stream() sources need capabilities: - CAP_CHOWN - needed if owner() or group() are in use - CAP_FOWNER - to force chmod() for sockets with owner != root - CAP_DAC_OVERRIDE - force changes if some parent dir has 000 perm, ie vservers Attached patch fixes all these problems. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6