- syslog-ng - lists.balabit.hu

Syslog-ng writes log entries to 2 different logs
by Burton Simonds 29 Sep '10

29 Sep '10

I am using syslog-ng 3.1.2 and I am having a problem that I can not figure out. Our network gear sends messages to syslog-ng, and then it filters them based on device type, and name. There is one firewall, that is matching both the firewalls filter and the switches filter, and is subsequently writing to both locations. in the example below, host 1.2.3.5 is logging to both the switches log dir and the firewalls log dir even though it is only referenced in the firewalls filter. Other then the obvious possibilities, (Yes, I have confirmed that the ip address is only referenced in one filter) does anyone have any ideas on what I should look at? Thanks, Burton our filters look like this (each host filter has several addresses in it): filter f_firewalls { host(1.2.3.4) or host(1.2.3.5); }; filter f_switches {host(9.8.7.6) or host(9.8.7.5); }; destination d_firewalls {file( "/path/to/logs/fw/$YEAR/MONTH/$HOST-$YEAR$MONTH$DAY" create_dirs(yes) owner(log_owner) group(log_group) dir_perm(0775) perm(0664) ); }; destination d_switches {file( "/path/to/logs/sw/$YEAR/MONTH/$HOST-$YEAR$MONTH$DAY" create_dirs(yes) owner(log_owner) group(log_group) dir_perm(0775) perm(0664) ); }; log { source(s_network); filter(f_firewalls); destination(d_firewalls); }; log { source(s_network); filter(f_switches); destination(d_switches); }; Version info: syslog-ng 3.1.2 Installer-Version: 3.1.2 Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.1#master#8bf13c304b6ab5fc1a372b49d55c78370efe14ca Compile-Date: Aug 3 2010 11:16:08 Enable-Threads: on Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-Sun-Door: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: off Enable-SSL: on Enable-SQL: on Enable-Linux-Caps: on Enable-Pcre: on

4 4

vsftpd login/login failure events
by Peter Czanik 29 Sep '10

29 Sep '10

Hello, Attached are the vsftpd login/login failure events I found. There was no trace of logout in the logs. Question: what is the correct way dealing with the last rule? It has "vsftpd" twice in it. I checked, and obviously only the last appearance counts. Is it worth to define it twice? linux-6y8u:/local/czanik/tmp/syslog-ng-patterndb/file-service # pdbtool match -D -p vsftpd.pdb -P vsftpd -M "pam_listfile(bla1:auth): Refused user root for service bla2" Pattern matching part: pam_listfile(@STRING:usracct.service=bla1@:auth): Refused user @ESTRING:usracct.username=root@for service @ANYSTRING:usracct.service=bla2@ Matching part: pam_listfile(bla1:auth): Refused user root for service bla2 Values: MESSAGE=pam_listfile(bla1:auth): Refused user root for service bla2 PROGRAM=vsftpd .classifier.class=system .classifier.rule_id=7256a6d6-c720-11df-8a1d-000c298c9ba2 usracct.username=root usracct.service=bla2 usracct.type=login usracct.sessionid= usracct.application=vsftpd secevt.verdict=REJECT Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

2 2

[Bug 94] New: too many "Configuration file format is too old..." messages
by bugzilla＠bugzilla.balabit.com 29 Sep '10

29 Sep '10

https://bugzilla.balabit.com/show_bug.cgi?id=94 Summary: too many "Configuration file format is too old..." messages Product: syslog-ng Version: 3.2.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: czanik(a)balabit.hu Type of the Report: --- Estimated Hours: 0.0 When using my regular 3.1 configuration, I get the following message 1000+ times on screen: " WARNING: Configuration file format is too old, please update it to use the 3.2 format as some constructs might operate inefficiently; " It should only be printed once. Changing the first line of configuration to: "@version:3.2" silents the warning messages, and everything works as expected. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

AUTO: James B Horwath/TheGuardian is out of the office. (returning 10/04/2010)
by James B Horwath 29 Sep '10

29 Sep '10

I am out of the office until 10/04/2010. I will be out of the office until October 4th and will not have access to e-mail. If there are urgent logging issues please contact Sunil Gupta at 610-807-6717. Any other issues please contact Christopher Clarke. Thanks, Jim Note: This is an automated response to your message "syslog-ng Digest, Vol 65, Issue 48" sent on 9/29/10 5:34:59. This is the only notification you will receive while this person is away. ----------------------------------------- This message, and any attachments to it, may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are notified that any use, dissemination, distribution, copying, or communication of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately by return e-mail and delete the message and any attachments. Thank you.

1 0

one more sshd rule
by Peter Czanik 29 Sep '10

29 Sep '10

Hello, While checking my logs with pdbtool, I ran into this log message: Sep 23 13:10:03 linux-6y8u sshd[21420]: error: PAM: Authentication failure for root from 192.168.2.52 The attached rule seems to find it correctly: HOST=linux-6y8u MESSAGE=error: PAM: Authentication failure for root from 192.168.2.52 PROGRAM=sshd PID=21420 LEGACY_MSGHDR=sshd[21420]: .classifier.class=system .classifier.rule_id=55ec76e0-c709-11df-b62d-000c298c9ba2 usracct.username=root usracct.device=192.168.2.52 usracct.type=login usracct.sessionid=21420 usracct.application=sshd secevt.verdict=REJECT Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

2 2

Re: [syslog-ng] Converting filtering from 2.1 to 3.0?
by Worsham, Michael 29 Sep '10

29 Sep '10

I must be doing something wrong or the syntax is really dependant on the program name now. Orginally under the v2.1 build, the intent of the filter was to strip any incoming message going to the syslog-ng server from the data being sent from the syslog-ng client. There was no one specific program attached to the incoming data. It could come from the client's syslog-ng side via logger command, apache data, kernel, etc. Server-side syslog-ng.conf filter section: filter M_audit { not match("Auditing" value("Audit daemon rotating log files") flags(ignore-case)); }; filter M_repeat { not match("Repeating" value("last message repeated") flags(ignore-case)); }; filter M_stats { not match("Stats" value("Log statistics") flags(ignore-case)); }; On the client side, I would just do 'logger daemon' to send over the message to the syslog-ng server. On the server side, running syslog-ng -d reveals this output: Incoming log entry; line='<13>Sep 20 17:16:31 drupal root: daemon' Filter rule evaluation begins; filter_rule='M_audit' No such value known; value='Audit daemon rotating log files' Filter node evaluation result; filter_result='match' Filter rule evaluation result; filter_result='match', filter_rule='M_audit' Filter rule evaluation begins; filter_rule='M_repeat' No such value known; value='last message repeated' Filter node evaluation result; filter_result='match' Filter rule evaluation result; filter_result='match', filter_rule='M_repeat' Filter rule evaluation begins; filter_rule='M_stats' No such value known; value='Log statistics' Filter node evaluation result; filter_result='match' Filter rule evaluation result; filter_result='match', filter_rule='M_stats' Server-side syslog output being seen in the data file: Sep 20 17:16:31 drupal root: daemon ________________________________ CONFIDENTIALITY NOTICE: This email and any attachments are intended solely for the use of the named recipient(s). This email may contain confidential and/or proprietary information of Scientific Research Corporation. If you are not a named recipient, you are prohibited from reviewing, copying, using, disclosing or distributing to others the information in this email and attachments. If you believe you have received this email in error, please notify the sender immediately and permanently delete the email, any attachments, and all copies thereof from any drives or storage media and destroy any printouts of the email or attachments. EXPORT COMPLIANCE NOTICE: This email and any attachments may contain technical data subject to U.S export restrictions under the International Traffic in Arms Regulations (ITAR) or the Export Administration Regulations (EAR). Export or transfer of this technical data and/or related information to any foreign person(s) or entity(ies), either within the U.S. or outside of the U.S., may require advance export authorization by the appropriate U.S. Government agency prior to export or transfer. In addition, technical data may not be exported or transferred to certain countries or specified designated nationals identified by U.S. embargo controls without prior export authorization. By accepting this email and any attachments, all recipients confirm that they understand and will comply with all applicable ITAR, EAR and embargo compliance requirements.

5 18

pub_date
by Peter Czanik 27 Sep '10

27 Sep '10

Hello, All of the pdb files in git have pub_date set to 2010-07-13. Should I also use this date, or it's just a coincidence and I can use the actual date? Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

2 1

[Bug 93] New: filter() functionality between 2.1 to 3.0 not consistent
by bugzilla＠bugzilla.balabit.com 23 Sep '10

23 Sep '10

https://bugzilla.balabit.com/show_bug.cgi?id=93 Summary: filter() functionality between 2.1 to 3.0 not consistent Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: mworsham(a)scires.com Type of the Report: bug Estimated Hours: 0.0 In the older syslong-ng v2.1, this line works perfectly: filter M_audit { not match("Audit daemon rotating log files"); }; Under 3.0.8, none of the following are working (if added one line at a time) and the daemon restarted: filter M_audit { not match("Audit daemon rotating log files" value("MSGONLY") flags(ignore-case)); }; filter M_audit { not match("MSGONLY" value("Audit daemon rotating log files") flags(ignore-case)); }; filter M_audit { not match("Audit daemon rotating log files" value(MSGONLY) flags(ignore-case)); }; filter M_audit { not match("Audit daemon rotating log files" value(MSGONLY)); }; filter M_audit { not match("MSGONLY" value("Audit daemon rotating log files")); }; What I am looking to do is if any incoming data has the following keywords (i.e. "Audit daemon rotating log files") being detected, it should be filtered (i.e. dropped), and then not show up in the actual message log file. For example, if I go over on the syslog-ng client and do 'logger daemon': - The older v2.1 syslog-ng server detects the embedded keyword and drops the message. <-- This is correct - On the v3.0.8, the message passes through and found in the data file. <-- This is incorrect Attempted to use the following URLs to correct the formatting, but it still not working: - http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.1-gui… - http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.1-gui… -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

5 10

Converting filtering from 2.1 to 3.0?
by Worsham, Michael 20 Sep '10

20 Sep '10

We are running syslog-ng v2.1 in a multiple client-server configuration. Due to new encryption requirements, we planning to upgrade to v3.0.8 soon and use the TLS capabilities. Upon one of our test servers, I discovered that the following filtering works just fine under syslog-ng v2.1 and blocks the messages from reaching the remote syslog-ng server: filter M_audit { not match("Audit daemon rotating log files"); }; filter M_snmp { not match("snmpd"); }; filter M_repeat { not match("last message repeated"); }; filter M_stats { not match("Log statistics"); }; log { source(s_localhost); filter(M_audit); filter(M_snmp); filter(M_repeat); filter(M_stats); destination(d_general); # Remote Syslog server }; However under syslog-ng v3.0.8, these messages are getting passed over to the remote syslog server. What do I need to do to strip these messages from getting passed through? I tried using "not message" but that doesn't seem to work. -- Michael ________________________________ CONFIDENTIALITY NOTICE: This email and any attachments are intended solely for the use of the named recipient(s). This email may contain confidential and/or proprietary information of Scientific Research Corporation. If you are not a named recipient, you are prohibited from reviewing, copying, using, disclosing or distributing to others the information in this email and attachments. If you believe you have received this email in error, please notify the sender immediately and permanently delete the email, any attachments, and all copies thereof from any drives or storage media and destroy any printouts of the email or attachments. EXPORT COMPLIANCE NOTICE: This email and any attachments may contain technical data subject to U.S export restrictions under the International Traffic in Arms Regulations (ITAR) or the Export Administration Regulations (EAR). Export or transfer of this technical data and/or related information to any foreign person(s) or entity(ies), either within the U.S. or outside of the U.S., may require advance export authorization by the appropriate U.S. Government agency prior to export or transfer. In addition, technical data may not be exported or transferred to certain countries or specified designated nationals identified by U.S. embargo controls without prior export authorization. By accepting this email and any attachments, all recipients confirm that they understand and will comply with all applicable ITAR, EAR and embargo compliance requirements.

2 1

[patterndb] classification
by Balazs Scheidler 20 Sep '10

20 Sep '10

Hi, As you probably know one goal for patterndb is to implement message classification. E.g. in addition to extracting information from log messages, it also associates a "class", later available in the "${.classifier.class}" value. Right now, syslog-ng doesn't really care what this string is. But the XML schema validating patterndb file lists the following four classes (taken from the logcheck project) violation - security violation security - other security events system - system information unknown - no rule matches One one hand, the tagging functionality (e.g. the ability to also associate tags with each message) is superior to classes. On the other hand, all tags are equivalent, thus if a message has 5 tags, then currently syslog-ng only provides functions to _filter_ based on tags, but not use it as a macro. So for example it is possible to do: file d_class_files { file("/var/log/messages.${.classifier.class}.log"); }; But it is difficult to do with tags (except for using filters and different destinations), as there's no such functionality. Another problem is that tags/classes are completely independent, in order to filter on the class of the message, one would have to use a match() filter like this: filter f_class { match("violation" value(".classifier.class")); }; My conclusion is that classes are better when used in templates, tags are better when filtering. The two should be merged somehow. So I'm thinking on how to move forward. Here are the alternatives I'm considering: 1) the class of the message is always a tag in some generated format (e.g. if a message has class XXX, then a tag named ".class.XXX" would be automatically associated with the message. This is somewhat cumbersome. 2) the class of the message is created as a tag as well, with the same name as the class. e.g. we'd have a tag named "violation", but that'd preclude the use of the "violation" name as a tag. 3) drop the class stuff and implement a macro trick that makes it possible to use tags in macro context One way to do this: file d_class_files { file("/var/log/messages.$(expand-tag-name violation security system unknown).log"); }; The "expand-tag-name" macro function would try to look for the tags listed as parameters, and if the message matches it'd expand to the tagname. This is not intuitive and if someone wants to use such an expansion in a lot of templates, it is also irritating and difficult to get right. On an independent matter, the set of classes may need some thought. As I said the original list is borrowed from logcheck, but I think it probably needs to be expanded. Last time I got patterns for DNS queries, and although I could shove them into "system", right now I feel that the point of classification is to categorize events by "importance", in a similar spirit to syslog severity, but one that works even if the application developer uses a bogus severity when sending syslog messages. So one email, two questions, feedback appreciated. Thanks. -- Bazsi

4 17