- syslog-ng - lists.balabit.hu

R: Re: Closing destination file when date changes.
by andrea.meconi＠libero.it 22 Mar '13

22 Mar '13

Thank. I'm reconfiguring syslog-ng using macro and sending HUP. Regards. Andrea >----Messaggio originale---- >Da: algernon(a)balabit.hu >Data: 22/03/2013 15.44 >A: "Syslog-ng users' and developers' mailing list"<syslog-ng(a)lists.balabit. hu> >Ogg: Re: [syslog-ng] Closing destination file when date changes. > >Sandor Geller <Sandor.Geller(a)morganstanley.com> writes: > >> On Fri, Mar 22, 2013 at 9:37 AM, andrea.meconi(a)libero.it >> <andrea.meconi(a)libero.it> wrote: >>> Hello. >>> I'm using a very simple configuration on Ubuntu 12.04.2 LTS >>> >>> >>> @version: 3.3 >>> @include "scl.conf" >>> source from-network { udp(ip("0.0.0.0") port(514)); }; >>> destination local-file { >>> file("/var/log/temporary-encrypted/$YEAR-$MONTH-$DAY.log"); }; >>> log { source(from-network); destination(local-file);}; >>> >>> >>> Log on a destination file using date as filename. >>> I need to close this file when date changes at 00:00. How? >> >> Unused file destinations could get reaped, see time_reap(). However >> you should take into account that the macros you used in the filename >> are dependent of keep_timestamp() and by default use the timestamp in >> the log message so the destination file could get opened when further >> logs arrive with timestamps referring to the given date... If you want >> to use files based on the receiver's time then I suggest using the R_ >> macros. > >That, and you can force syslog-ng to reinitialize the destinations by >sending it a HUP signal. So using the R_* family of macros, and sending >a SIGHUP shortly after 00:00 will result in syslog-ng closing the old >files. > >-- >|8] > >______________________________________________________________________________ >Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >Documentation: http://www.balabit.com/support/documentation/?product=syslog- ng >FAQ: http://www.balabit.com/wiki/syslog-ng-faq > >

1 0

AUTO: James B Horwath is out of the office (returning 03/25/2013)
by James B Horwath 22 Mar '13

22 Mar '13

I am out of the office until 03/25/2013. I will be out of the office until 03/25/2013 with limited access to e-mail. If you have an urgent issue please contact Sunil Gupta at (610)-807-6710 or Dylan Yoder at (610)-807-7230. Regards Jim Horwath IT Security Services & Governance, Corporate BTS and IT Security/Risk Tel 610-807-8795 Mobile 610-533-6972 3900 Burgess Place, Bethlehem, Pa. 18017 Jim_Horwath(a)glic.com Note: This is an automated response to your message "syslog-ng Digest, Vol 95, Issue 14" sent on 3/22/2013 7:00:02. This is the only notification you will receive while this person is away. ----------------------------------------- This message, and any attachments to it, may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are notified that any use, dissemination, distribution, copying, or communication of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately by return e-mail and delete the message and any attachments. Thank you.

1 0

flags(final) in version 3.4
by Fanselow, William 22 Mar '13

22 Mar '13

According to this report, the flags(final) does not work in 3.4 as one got used to in prior versions. https://lists.balabit.hu/pipermail/syslog-ng/2013-February/020039.html In version 3.3 I used flags(final) in each of my log{ } statements so that a message was not unnecessarily processed by anything beyond a matching filter. My last entry in the config used flags(fallback) so that any message not previously sent to a destination was caught by this final stanza. With version 3.4, the same syntax does not work, and all messages appear to be passing through every filter and getting duplicated in the first match and my final log destination. Surely, there must be another way to handle this sort of thing. Not only is it useful for processing efficiency, but it is also a useful way to identify "unmatched" filters in the config. Any suggestions for replicating the 3.3 behavior would be appreciated. Thanks Bill Fanselow

5 4

syslog-ng Insider - March 2013
by Peter Czanik 14 Mar '13

14 Mar '13

Dear syslog-ng users, This is the 22th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news. Your feedback and news tips about the next issue is welcome at documentation(a)balabit.com <mailto:documentation@balabit.com> FEATURED NEWS Video: networking scenarios and filters --------------------------------------- There is a video series in the work to introduce the basics of syslog-ng. The third video is online now, which explains networking scenarios and filters. You can watch it at https://www.youtube.com/watch?v=VR3t7p04GUI Masking credit card numbers in log messages with syslog-ng ---------------------------------------------------------- Compliance with different regulations is becoming more and more important recently also in logging. One of these, PCI-DSS requires credit card numbers to be masked in logs. Our CTO describes, how it can be achieved using the freshly released syslog-ng 3.4 at http://marci.blogs.balabit.com/2013/02/masking-credit-card-numbers-in-log-m… Brand new syslog-ng Superhero T-shirt available ----------------------------------------------- The syslog-ng Superhero T-shirt was designed for open source fan geeks. Can't wait more to get one? Simply tell us which version of syslog-ng (OSE, PE or SSB) you use, and be our public syslog-ng reference. Please send an email to Peter Czanik for more information and don't forget to include your t-shirt size.;-) T-shirt pictures on Facebook: http://www.facebook.com/media/set/?set=a.10151698258059867.609510.124671654… T-shirt pictures on the web: http://bit.ly/ZFmmVq SHORT NEWS: * openSUSE 12.3 was released yesterday and is the first Linux distribution to include syslog-ng 3.4.1 NEW RELEASES: * SSB 3F1: http://gyp.blogs.balabit.com/2013/03/introducing-syslog-ng-store-box-3f1/ ARCHIVE http://insider.blogs.balabit.com/ -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

[Bug 228] New: match regex not working
by bugzilla＠bugzilla.balabit.com 13 Mar '13

13 Mar '13

https://bugzilla.balabit.com/show_bug.cgi?id=228 Summary: match regex not working Product: syslog-ng Version: 3.3.x Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: blackluck(a)ktk.bme.hu Type of the Report: bug Estimated Hours: 0.0 Sziasztok! Ubuntu 8.04 lts-rol megfelelo lepesekben frissitve lett rendszer 12.04 lts-re, abban talalhato syslog-ng 3.3.4 -ben nem mukodik jol a match regex szures. Konkretan ugy viselkedik hogy sima stringre szures csak akkor ervenyesul ha bejovo logban string elott van space. Filter: filter f_EQAM_BW_OVERSUB { netmask("172.18.96.0/255.255.224.0") and match("Bandwidth Oversubscribed" value("MESSAGE")); }; ha netcat-el kuldok bele teszt uzenetet, csak akkor ervenyeul a filter ha van szokoz a keresett szoveg es az elotte talalhato tartalom kozott, pl : "test Bandwidth Oversubscribed" mukodik "testBandwidth Oversubscribed" nem mukodik Korabban 8.04-es ubuntu alatt ez mukodott rendben (csak akkor match-nek nem kellett megadni a value("MESSAGE") reszt), szintaktikai hibat nem ad a konfigra. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

Writing JSON message to a MongoDB destination
by Al Itchon 07 Mar '13

07 Mar '13

I tried using I'm getting an error trying to use json-parser here: parser p_json { json-parser(prefix(".json.")); }; But I get this error: Error parsing parser expression, parser plugin json-parser not found in /usr/local/etc/syslog-ng.conf at line 38, column 3: json-parser(prefix(".json.")); I'm pretty sure I'm using syslog-ng 3.4.1. Any idea on what I'm missing?

3 4

strange error message: Invalid byte sequence or other error while converting input
by Lucas, Sascha 01 Mar '13

01 Mar '13

Hi, I'm running a self-compiled syslog-ng-3.4.1 on SLES11-SP2. Compilation is mostly identical to the rpm-spec from OBS home:czanik:syslog-ng34, expect spoof-source is disabled. I have the following configuration enabled, to avoid non-UTF8 chars in mongodb: source s_network { tcp( max-connections(10) port(2012) encoding(UTF-8)); udp( port(2012) encoding(UTF-8)); }; Feb 27 12:54:13 gisu1000 syslog-ng[3335]: Syslog connection accepted; fd='11', client='AF_INET(x.x.x.x:50928)', local='AF_INET(0.0.0.0:2012)' Feb 27 12:54:13 gisu1000 syslog-ng[3335]: Invalid byte sequence or other error while converting input, skipping character; encoding='UTF-8', char='0xd2' Feb 27 12:54:13 gisu1000 syslog-ng[3335]: Syslog connection closed; fd='11', client='AF_INET(x.x.x.x:50928)', local='AF_INET(0.0.0.0:2012)' Can someone point me to what this means and how to avoid this? Thanks, Sascha. Vorsitzender des Aufsichtsrates: Ralf Hiltenkamp Geschäftsführung: Michael Krüger (Sprecher), Stephan Drescher Sitz der Gesellschaft: Halle/Saale Registergericht: Amtsgericht Stendal | Handelsregister-Nr. HRB 208414 UST-ID-Nr. DE 158253683 Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Empfänger sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail oder des Inhalts dieser Mail sind nicht gestattet. Diese Kommunikation per E-Mail ist nicht gegen den Zugriff durch Dritte geschützt. Die GISA GmbH haftet ausdrücklich nicht für den Inhalt und die Vollständigkeit von E-Mails und den gegebenenfalls daraus entstehenden Schaden. Sollte trotz der bestehenden Viren-Schutzprogramme durch diese E-Mail ein Virus in Ihr System gelangen, so haftet die GISA GmbH - soweit gesetzlich zulässig - nicht für die hieraus entstehenden Schäden.

1 1

credit card masking in log messages
by Marton ILLES 01 Mar '13

01 Mar '13

Hello Loggers! I was playing around with the latest 3.4.1 release and created two rewrite configuration blocks to mask and hash credit card numbers in log messages. I think it would be a good idea to add these config blocks to SCL so anyone could use it latter easily. These are fairly simple two blocks: block rewrite hash_cc(value("MESSAGE")) { subst( "(?P<1>:4[0-9]{12}(?:[0-9]{3})?|5[1-5][0-9]{14}|6(?:011|5[0-9][0-9])[0-9]{12}|3[47][0-9]{13}|3(?:0[0-5]|[68][0-9])[0-9]{11}|(?:2131|1800|35\d{3})\d{11})", "$(sha1 --length 16 $1)", value(`value`), flags(global, store-matches), type(pcre)); }; block rewrite mask_cc(value("MESSAGE")) { subst( "(?P<1>:4[0-9]{12}(?:[0-9]{3})?|5[1-5][0-9]{14}|6(?:011|5[0-9][0-9])[0-9]{12}|3[47][0-9]{13}|3(?:0[0-5]|[68][0-9])[0-9]{11}|(?:2131|1800|35\d{3})\d{11})", "$(substr $1 0 6)******$(substr $1 -4 4)" value(`value`), flags(global, store-matches), type(pcre)); }; You can use it in your rewrite statements easily: log { source(s_local); rewrite { mask_cc(); }; #rewrite { hash_cc(value(".sdata.my-cc")); }; destination(d_local); }; And it turns a message like this: Transaction approved; checkout-id='46255763', amount='38.43', cc='5542043004559005' Into this, in case of masking: Transaction approved; checkout-id='46255763', amount='38.43', cc='554204******9005' And in case of hashing: Transaction approved; checkout-id='46255763', amount='38.43', cc='986d97fc95435b22' For anyone interested in more details on credit card number masking/hashing/tokenization I also wrote a blogpost about this: http://marci.blogs.balabit.com/2013/02/masking-credit-card-numbers-in-log-m… The regexps could be enhanced further, but I guess it is good for a start. Bazsi, Algernon could you please add this to SCL? Sorry, I was lazy to generate a git pull request. :( Happy Masking! cheers, Marton

2 2

[Bug 220] New: Version 3. 4 logical configuration incompatible with previous versions
by bugzilla＠bugzilla.balabit.com 01 Mar '13

01 Mar '13

https://bugzilla.balabit.com/show_bug.cgi?id=220 Summary: Version 3.4 logical configuration incompatible with previous versions Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 In version 3.3 the top level configuration structure treated all log statements as if they were inside a single block. If there was a log statement with a flags(final) then no other log statements were processed. This allowed @include statements to include snippets that included logs statements with flags(final). In version 3.4 the top level configuration structure treats all log statements independently. If there was a log statement with a flags(final) it only affects processing within that log statement and processing does continue with then next log statement. Including files with log statements and flags(final) can'r really be done and the @include can not be used inside of log statements. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

storing an array in mongodb
by Lucas, Sascha 27 Feb '13

27 Feb '13

Hi, I'm experimenting with patterndb and the classifier (syslog-ng 3.4.1). ATM the classifier is stored in the TAGS key as a comma separated string. For searching on different classes it would be nice to store it in an array. This can be indexed and searched (per element). Is this possible? TIA, Sascha. Vorsitzender des Aufsichtsrates: Ralf Hiltenkamp Geschäftsführung: Michael Krüger (Sprecher), Stephan Drescher Sitz der Gesellschaft: Halle/Saale Registergericht: Amtsgericht Stendal | Handelsregister-Nr. HRB 208414 UST-ID-Nr. DE 158253683 Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Empfänger sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail oder des Inhalts dieser Mail sind nicht gestattet. Diese Kommunikation per E-Mail ist nicht gegen den Zugriff durch Dritte geschützt. Die GISA GmbH haftet ausdrücklich nicht für den Inhalt und die Vollständigkeit von E-Mails und den gegebenenfalls daraus entstehenden Schaden. Sollte trotz der bestehenden Viren-Schutzprogramme durch diese E-Mail ein Virus in Ihr System gelangen, so haftet die GISA GmbH - soweit gesetzlich zulässig - nicht für die hieraus entstehenden Schäden.

3 2