- syslog-ng - lists.balabit.hu

[Bug 243] New: patterndb rule should allow action upon context timeout
by bugzilla＠bugzilla.balabit.com 09 Aug '13

09 Aug '13

https://bugzilla.balabit.com/show_bug.cgi?id=243 Summary: patterndb rule should allow action upon context timeout Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.balabit(a)faxm0dem.org Type of the Report: enhancement Estimated Hours: 0.0 Currently in an event correlation configuration, when a certain context times out, the whole rule is thrown away. It would be very useful to be able to trigger an action when a rule's context-timeout is reached. Of course the ability to add a timeout_action for every rule would further enhance the idea. Example: <ruleset name='dummy_ruleset' id='03eb0142-4b0c-4226-ac98-6bcb03e59e00'> <pattern>dummy_program</pattern> <rules> <rule provider="dummy_provider" id="0cc9a000-2a4e-41f2-b30b-09d67af68ddc" class='dummy_class' context-timeout="300" context-scope="program" context-id="dummy_context"> <patterns> <pattern>First message has @ANYSTRING:dummy_string@</pattern> </patterns> <timeout_actions> <action> <message> <values> <value name="MESSAGE">Timeout: Failed to correlate ${dummy_string}@1 with anything</value> </values> </message> </action> </timeout_actions> </rule> <rule provider="dummy_provider" id="6fbefe59-3448-4b29-8c4f-7d9c1ab65a4c" class='dummy_class' context-scope="program" context-id="dummy_context"> <patterns> <pattern>Second message has @ANYSTRING:dummy_string@</pattern> </patterns> <actions> <action> <message> <values> <value name="MESSAGE">Correlated ${dummy_string}@1 with ${dummy_string}@2</value> </values> </message> </action> </actions> </rule> </rules> </ruleset> -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

Time Conversion and String delimiter
by Kyaw Kyi 07 Aug '13

07 Aug '13

Hi, I have a non-standard (not rfc 5414) log message format coming into syslog-ng client. I am trying to use csv-parser to parse into custom macros and send it off to syslog-ng host in the correct format. With no-parse option, how do I make syslog-ng handle this incoming timestamp: [July 30, 2013 10:19:06 AM] ? It would be so much easier to simply use the timestamp for the time message received in syslog-ng (e.g $DATE marco) but I have to extract the timestamp from the message itself somehow. Example: [July 30, 2013 10:19:06 AM EDT] DisplayType=CLEAR ClassName=Host.... Secondly, the documentation of syslog-ng seems to hint that I can use string delimiter in parser. "Currently only the csv-parser is implemented, which can separate columns based on delimiter characters and strings." but it also says that "If you specify multiple characters, every character will be treated as a delimiter." . Is there a way for me to use a string, not single characters, as delimiters in csv-parser? Thanks. ------------------------------------------------------------------------------ Please consider the environment before printing this email. Visit our website at http://www.nyse.com <http://www.nyse.com> ***************************************************************************** Note: The information contained in this message and any attachment to it is privileged, confidential and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by replying to the message, and please delete it from your system. Thank you. NYSE Euronext.

2 1

system() source problem.
by Nemeth Gyorgy 07 Aug '13

07 Aug '13

I compiled a syslog-ng 3.4 and when starting I get teh following error message: Error parsing source, source plugin system not found in /etc/syslog-ng/syslog-ng.conf at line 31, column 2: system(); The part of syslog-ng.conf is the following: source s_src { # message generated by Syslog-NG internal(); # standard Linux log source (this is the default place for the syslog() # function to send logs to) unix-stream("/dev/log"); # messages from the kernel file("/proc/kmsg" program_override("kernel")); # use the following line if you want to receive remote UDP logging messages # (this is equivalent to the "-r" syslogd flag) # udp(); system(); }; According to the documentation the system() source is correct.. What is wrong here? When I comment the system() source out, the syslog-ng daemon starts without errors. # syslog-ng -V syslog-ng 3.4.2 Installer-Version: 3.4.2 Revision: Compile-Date: Aug 5 2013 11:00:20 Available-Modules: syslogformat,afsocket-notls,afsocket-tls,afsql,affile,afprog,afuser,afamqp,afmongodb,csvparser,confgen,system-source,basicfuncs,cryptofuncs,dbparser,afsocket Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: off Enable-Linux-Caps: off Enable-Pcre: on -- Friczy

3 3

problem tagging with patterndb (syslog-ng 3.4.1)
by mailing lists 06 Aug '13

06 Aug '13

Hello all, I am having a problem understanding patterndb and tags, for the following rule the log line is matched but tags are not settled in pdbtool output. What am i missing here? <rule id="dad57bd5-6f9e-47b8-9e9f-401e3eb34334" provider="user" class="system"> <patterns> <pattern>@ESTRING:postfix.qid::@ to=@QSTRING:postfix.to:<>@, orig_to=@QSTRING:postfix.orig_to:<>@, relay=@ESTRING:postfix.relay.hostname:[@@ESTRING:postfix.relay.path:]@, delay=@ESTRING:postfix.delay:,@ delays=@ESTRING:postfix.delays.1:/@@ESTRING:postfix.delays.2:/@@ESTRING:postfix.delays.3:/@@ESTRING:postfix.delays.4:,@ dsn=@ESTRING:postfix.dsn:,@ status=@ESTRING:postfix.status: @@QSTRING:postfix.statusmsg:()@</pattern> <pattern>@ESTRING:postfix.qid::@ to=@QSTRING:postfix.to:<>@, orig_to=@QSTRING:postfix.orig_to:<>@, relay=@ESTRING:postfix.relay.hostname:[@@ESTRING:postfix.relay.path:]@:24, delay=@ESTRING:postfix.delay:,@ delays=@ESTRING:postfix.delays.1:/@@ESTRING:postfix.delays.2:/@@ESTRING:postfix.delays.3:/@@ESTRING:postfix.delays.4:,@ dsn=@ESTRING:postfix.dsn:,@ status=@ESTRING:postfix.status: @@QSTRING:postfix.statusmsg:()@</pattern> <pattern>@ESTRING:postfix.qid::@ to=@QSTRING:postfix.to:<>@, orig_to=@QSTRING:postfix.orig_to:<>@, relay=@ESTRING:postfix.relay.hostname:[@@ESTRING:postfix.relay.path:]@:24, delay=@ESTRING:postfix.delay:,@ delays=@ESTRING:postfix.delays.1:/@@ESTRING:postfix.delays.2:/@@ESTRING:postfix.delays.3:/@@ESTRING:postfix.delays.4:,@ dsn=@ESTRING:postfix.dsn:,@ status=@ESTRING:postfix.status: @(@ESTRING:postfix.status.code1: @@ESTRING:postfix.status.code2: @@EMAIL:postfix.status.recipient:<> @@ESTRING:postfix.status.qid: @Saved)</pattern> </patterns> <tags> <tag>postfix</tag> <tag>lmtp</tag> </tags> </rule> $ pdbtool match -P 'postfix' -p postfix.pdb -D -c -f mail.log Pattern matching part: @ESTRING:postfix.qid=B5BBAADB@ to=@QSTRING:postfix.to=user002@example.com@, orig_to=@QSTRING:postfix.orig_to=noreply@example.com@, relay=@ESTRING:postfix.relay.hostname=lmtp.example.com@@ESTRING:postfix.relay.path=10.180.242.142@:24, delay=@ESTRING:postfix.delay=0.07@ delays=@ESTRING:postfix.delays.1=0.04@@ESTRING:postfix.delays.2=0@@ESTRING:postfix.delays.3=0@@ESTRING:postfix.delays.4=0.03@ dsn=@ESTRING:postfix.dsn=2.0.0@ status=@ESTRING:postfix.status=sent@(@ESTRING:postfix.status.code1=250@@ESTRING:postfix.status.code2=2.0.0@@EMAIL:postfix.status.recipient=noreply@example.com@@ESTRING:postfix.status.qid=XP52K7Bp+1G/FAAAtCZERg@Saved) Matching part: B5BBAADB: to=<user002(a)example.com>, orig_to=<noreply(a)example.com>, relay=lmtp.example.com[10.180.242.142]:24, delay=0.07, delays=0.04/0/0/0.03, dsn=2.0.0, status=sent (250 2.0.0 <noreply(a)example.com> XP52K7Bp+1G/FAAAtCZERg Saved) Values: HOST=mailserver MESSAGE=B5BBAADB: to=<user002(a)example.com>, orig_to=<noreply(a)example.com>, relay=lmtp.example.com[10.180.242.142]:24, delay=0.07, delays=0.04/0/0/0.03, dsn=2.0.0, status=sent (250 2.0.0 <noreply(a)example.com> XP52K7Bp+1G/FAAAtCZERg Saved) PROGRAM=postfix/lmtp PID=29484 LEGACY_MSGHDR=postfix/lmtp[29484]: .classifier.class=system .classifier.rule_id=dad57bd5-6f9e-47b8-9e9f-401e3eb34334 postfix.qid=B5BBAADB postfix.to=user002(a)example.com postfix.orig_to=noreply(a)example.com postfix.relay.hostname=lmtp.example.com postfix.relay.path=10.180.242.142 postfix.delay=0.07 postfix.delays.1=0.04 postfix.delays.2=0 postfix.delays.3=0 postfix.delays.4=0.03 postfix.dsn=2.0.0 postfix.status=sent postfix.status.code1=250 postfix.status.code2=2.0.0 postfix.status.recipient=noreply(a)example.com postfix.status.qid=XP52K7Bp+1G/FAAAtCZERg TAGS=

2 1

Multi-line support issue
by Satish Patel 03 Aug '13

03 Aug '13

We have tomcat shop and at everyone know tomcat has a java call trace in logs with tab space but syslog-ng doesn't know about it and printing lines as a new line. I have read here syslog-ng 3.x does support multi-line logs http://www.balabit.com/sites/default/files/documents/syslog-ng-pe-4.0-guide… But does this feature available in Open Source syslog-ng? If yes then why its not working for me?

3 18

Re: [syslog-ng] question on parsed syslog-ng config files and command line options
by Jim Hendrick 27 Jul '13

27 Jul '13

Thanks! Sent from my Verizon Wireless 4G LTE Smartphone -------- Original message -------- From: Balazs Scheidler <bazsi77(a)gmail.com> Date: 07/26/2013 5:35 PM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] question on parsed syslog-ng config files and command line options --preprocess-into Not a perfect name, and should probably exit after having done its work. On Jul 26, 2013 9:46 PM, <jrhendri(a)roadrunner.com> wrote: Hi, I remember seeing (somewhere...) a command line option for syslog-ng that created an output file that was essentially the "resulting configuration" after all #include files were processed. I am trying to debug a problem with message parsing, and my filters are maintained in separate files. I was hoping I could remember this option so I could see what syslog-ng "thought" the resultant config was but (neither myself nor google) can remember it. essentially it expands all the include files "inline" and produces one output file. Help? Thanks, Jim ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

1 0

question on parsed syslog-ng config files and command line options
by jrhendri＠roadrunner.com 26 Jul '13

26 Jul '13

Hi, I remember seeing (somewhere...) a command line option for syslog-ng that created an output file that was essentially the "resulting configuration" after all #include files were processed. I am trying to debug a problem with message parsing, and my filters are maintained in separate files. I was hoping I could remember this option so I could see what syslog-ng "thought" the resultant config was but (neither myself nor google) can remember it. essentially it expands all the include files "inline" and produces one output file. Help? Thanks, Jim

2 1

Time zone problem in Mongodb driver
by Kyaw Kyi 25 Jul '13

25 Jul '13

Hi, I tried to use time_zone() option in mongodb destination driver but it is throwing an error. Error parsing afmongodb, syntax error, unexpected KW_TIME_ZONE, expecting ')' in /usr/local/etc/syslog-ng.conf at line 60, column 2: time_zone("America/New_York") It doesn't seem to a syntax error. It seems more like syslog-ng doesn't understand the time_zone() option in mongodb driver. This is my configuration: @version: 3.4 @include "scl.conf" options { ts_format(iso) ; keep_hostname(yes); keep_timestamp(yes); time_reopen(10); }; destination d_mongodb1 { mongodb( time_zone("America/New_York") ); }; I tried using ("-04:00") instead but no avail. Mongodb destination driver is working well without the time_zone() option. Thanks, Joe ------------------------------------------------------------------------------ Please consider the environment before printing this email. Visit our website at http://www.nyse.com <http://www.nyse.com> ***************************************************************************** Note: The information contained in this message and any attachment to it is privileged, confidential and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the sender immediately by replying to the message, and please delete it from your system. Thank you. NYSE Euronext.

4 5

XMPP destination for syslog-ng
by IKAC 23 Jul '13

23 Jul '13

Hi, My name is Ilija, and I'm a undergraduate student of computer science. I'm interested if this project is open for contribution apart from GSOC, since as I saw that there is no activity on it's github repository? Best regards, Ilija

5 5

AUTO: James B Horwath is out of the office (returning 07/29/2013)
by James B Horwath 20 Jul '13

20 Jul '13

I am out of the office until 07/29/2013. I will be out of the office until 07/29/2013 with limited access to e-mail. If you have an urgent issue please contact Sunil Gupta at (610)-807-6710 or Dylan Yoder at (610)-807-7230. Regards Jim Horwath IT Security Services & Governance, Corporate BTS and IT Security/Risk Tel 610-807-8795 Mobile 610-533-6972 3900 Burgess Place, Bethlehem, Pa. 18017 Jim_Horwath(a)glic.com Note: This is an automated response to your message "syslog-ng Digest, Vol 99, Issue 20" sent on 7/20/2013 6:00:02. This is the only notification you will receive while this person is away. ----------------------------------------- This message, and any attachments to it, may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are notified that any use, dissemination, distribution, copying, or communication of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately by return e-mail and delete the message and any attachments. Thank you.

1 0