- syslog-ng - lists.balabit.hu

syslog-ng and network problem
by Giovanni Mancuso 11 Jun '10

11 Jun '10

Hello guys I've installed two syslog-ng server instances behind a load balancer (LB) and configured N syslog-ng clients pointing to my virtual ip address. The clients doesn't send too much messages to the servers at this time. For some reason the LB closes the connections and when my client have to send a message to the server it realize that the connection was closed and it reopen a new one. The problem is that the messages even if the connection is stablished once again doesn't arrive to the server. I tryed also to enable the mark_freq(10) option but nothing changed. It starts to work if I restart the client or if I generate a lot of messages ( echo "MESSAGE" >> FILE.log ) I realize that the problem is in the network, but is someone have another idea to workaround this issue i'll be thankfull! Here is the config of one of my clients: @version: 3.0 #Default configuration file for syslog-ng. # # For a description of syslog-ng configuration file directives, please read # the syslog-ng Administrator's guide at: # # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html ######### # options options { time_reopen(10); keep_hostname(yes); log_msg_size(65535); flush_lines(0); }; ######### # sources source s_balabit { internal(); }; source s_source { file("FILE.log"); }; filter f_filter { match ("OUTGOING" value("MESSAGE")); }; ######### # destinations destination d_internal { file("/var/log/balabit.log"); }; destination d_network { tcp("vip" port(10001) tls( ca_dir("/etc/ssl/syslog-ng/ca.d")) log_fifo_size(4194304) ); }; ######### # logs log { source(s_source); filter(f_filter); destination(d_network);}; log { source(s_balabit); destination(d_internal); };

2 2

syslog-ng 3.0.7 has been released
by devel＠balabit.hu 11 Jun '10

11 Jun '10

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.0.7 SUMMARY : new stable release DATE : Jun 7, 2010 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.0.7) has been released. For latest fixes in the 3.0.x branch you are recommended to upgrade to this version. CHANGES: 3.0.7 Sun, 06 Jun 2010 13:27:38 +0200 Bugfixes: * When syslog-ng is reloaded, the local hostname value was not refreshed, causing syslog-ng to remember the hostname until the next restart. This may not play nice with DHCP configured hostnames, which may change dynamically. * When several SIGHUP signals are received in quick succession, the last one may have been dropped. This problem is fixed. * Make sure that numbers never get resolved using getpwnam/getgrnam (e.g. libnss functions), because this may cause deadlocks if the NSS provider is LDAP and the LDAP server is trying to log a message to syslog about invalid usernames. * Fixed flush_timeout() handling in for non-file destinations. Enabling flush_lines()/flush_timeout() for a non-file destination, either using a global option, or a per-destination option could cause excess CPU usage as long as the destination wasn't writable. Other changes: * Introduced a new, cleaner syntax to specify that the permissions of a file are not to be changed. Use owner/group/perm options of a file without any parameters instead of "-1". The old mechanism had the problem that both per-file and global settings had to use "-1" to effectively stop syslog-ng from changing the uid/gid/perm values. The new syntax will make it possible to specify defaults in the global options section and customize those on a per-file basis. * syslog-ng will not go into the background if either the --syntax-only or --debug options are specified. DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system…

1 0

Message uniqueness
by Dragan Zubac 09 Jun '10

09 Jun '10

Hello I'm using syslog-ng to log messages into PostgreSQL via file logging. I have a problem,because syslog-ng actually sends two messages from log file,instead one at a time. In attachment,You may see two messages that has the same ID number and were sent together,as 'one' message into database procedure. Is it possible to configure syslog-ng to send one message at a time only ? Sincerely Dragan Zubac

3 4

[Bug 85] New: wrong $_YEAR after logrotate
by bugzilla＠bugzilla.balabit.com 09 Jun '10

09 Jun '10

https://bugzilla.balabit.com/show_bug.cgi?id=85 Summary: wrong $_YEAR after logrotate Product: syslog-ng Version: 2.0.x Platform: PC OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: cedric.jeanneret(a)camptocamp.com Type of the Report: --- Estimated Hours: 0.0 Hello, We're using syslog-ng on debian lenny. We use macro like this : destination df_haproxy { file("/srv/syslog/archive/haproxy/$S_YEAR-$S_MONTH-$S_DAY-haproxy.log" create_dirs(yes) group(toto) ); }; We see there's an error when date changes, syslog recreate a new file with the wrong year (i.e. one year too old...) This happens each 1st of month: -rw-r----- 1 root toto 15782746 2010-05-31 23:59 2009-06-01-haproxy.log -rw-r----- 1 root toto 1756029229 2010-06-01 23:39 2010-06-01-haproxy.log More over, syslog-ng seems to use the wrong file for about 20 minutes, then "see" it's the wrong one, and create the good one just later... As said, this behavior is seen only on the 1st of each month... It's really strange. I have to confess, I really don't know if it's a problem for debian, or upstream. Here are informations about version and package : apt-cache show syslog-ng Package: syslog-ng Priority: extra Section: admin Installed-Size: 448 Maintainer: SZALAY Attila <sasa(a)debian.org> Architecture: amd64 Version: 2.0.9-4.1 If you think it's a debian problem, let me know so that I'll open a bug on debian's tracker. Thank you. Best regards, C. Jeanneret -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

Which of these config.php files should I use?
by Wagner Pereira 07 Jun '10

07 Jun '10

Hi, folks. I posted this issue in the cacti mailing list, but maybe the Syslog mailing list should be the better place to post this: ------------------ Hi, friends. After I've installed the syslog-ng plugin for cacti, there are much more files config.php than I'd like! See below: ------------------ /etc/syslog-ng/syslog/config.php /usr/share/cacti/site/include/config.php /var/www/cacti-0.8.1/include/config.php /var/www/cacti-0.8.1/plugins/syslog/config.php ------------------ In the /var/www/cacti-0.8.1/include/config.php one, I've the lines as follows (which is responsible to connect to the cacti database, right?): ------------------ $database_type = "mysql"; $database_default = "cacti"; $database_hostname = "localhost"; $database_username = "admin"; $database_password = "xxxxxxxxx"; $database_port = ""; ------------------ Even after I've made this below, I am still not able to see the Syslog tab in the Cacti web page: /mysqladmin --user=root -p create syslog / /mysql syslog///--user=root -p/// < syslog.sql/ Thanks in advance. Att, -- Wagner Pereira PoP-SP/RNP - Ponto de Presença da RNP em São Paulo CCE/USP - Centro de Computação Eletrônica da Universidade de São Paulo http://www.pop-sp.rnp.br Tel. (11) 3091-8901

1 1

Quick Questions
by Dyne Lai 04 Jun '10

04 Jun '10

Hi Syslog-ng Developers, I have a few general questions regarding syslog-ng as below. 1) What is the type of transmission (Asynchronous/Synchronous) used for the transfer of log messages between server and remote client? 2) I understand that Syslog-ng is able to store unsent messages from remote client to server in the local hardisk during network/system outrage. Will syslog-ng be able to resume with the unsent messages upon the revive of the network/system? If possible, how can we configure/setup to achieve that? 3) In view of security, how can we configure remote client and server to perform authentication (using X.509 certificates) during log message forwarding? 4) Is the forwarding of log message between remote client and server consider real-time? (assume syn=0) Do pardon if the questions appears ignorant.Thanks for the time! Best Regards, Dyne Lai

5 7

mis-claimed features: network link compression w/ TLS
by Robin H. Johnson 04 Jun '10

04 Jun '10

The comparison table: http://www.balabit.com/network-security/syslog-ng/comparing/detailed/ Lists: "Support for network link compression when using TLS" But I can't find it in the OSE at all. There are NO calls to the OpenSSL COMP* functions, nor any usage of compression functions that I can find. The Flex and Yacc grammar DO include a 'compress' keyword, but data from it is not actually used. I'm looking for some reasonable compression to cut down on network traffic one some Gentoo boxes where there is a very limited upstream network connection. Traffic is about 2.4GiB/day, over a 1Mbit link. Plain gzip after the fact gets >90% compression, but I'd like the logs in near-realtime to the remote logging destinations (multiple). I do also realize that we're going to be limited by packet compression, unless we keep some form of running dictionary, but I think even with that, we should get a good size reduction. -- Robin Hugh Johnson Gentoo Linux: Developer, Trustee & Infrastructure Lead E-Mail : robbat2(a)gentoo.org GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85

2 1

[Bug 83] New: sql (oracle) queries dont escape single quotes
by bugzilla＠bugzilla.balabit.com 03 Jun '10

03 Jun '10

https://bugzilla.balabit.com/show_bug.cgi?id=83 Summary: sql (oracle) queries dont escape single quotes Product: syslog-ng Version: 3.1.x Platform: Other OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.syslogng(a)feystorm.net Type of the Report: --- Estimated Hours: 0.0 SQL inserts with values containing single quotes dont escape the quotes causing the inserts to fail. Jun 1 22:08:24 slider syslog-ng.515[20417]: Error running SQL query; type='oracle', host='', port='', user='report_agent', database='oru23', error='1400: ORA-01400: cannot insert NULL into ("CMS_DBO"."MESSAGE_TRACKING"."MSGID")\x0a', query='INSERT INTO cms_dbo.message_tracking (host, msg_dt, pname, message, msgid, xid, from_name, from_domain, to_name, to_domain, addr) VALUES (\'iceman.dev.usa.net\', \'2010-06-01 22:08:22\', \'smtad\', \'[4155186080] 000000007 ok XID: XID007oFawiw8111Xem MsgID: <4C0584D4.50007(a)devo0402.dev.usa.net> Dest: cms:gwsin IP: 165.212.101.13 From: <phemmer(a)devo0402.dev.usa.net> Rcpt: <phemmer(a)devo0402.dev.usa.net> Subject: \'\'Re: asdf\'\' Size: 22400 Response: 250 Mail accepted (292oFawiw3072Mem)\', NULL, \'XID007oFawiw8111Xem\', \'phemmer\', \'devo0402.dev.usa.net\', \'phemmer\', \'devo0402.dev.usa.net\', \'165.212.101.13\')' The part that is causing the problem is the 'message' field with value: [4155186080] 000000007 ok XID: XID007oFawiw8111Xem MsgID: <4C0584D4.50007(a)devo0402.dev.usa.net> Dest: cms:gwsin IP: 165.212.101.13 From: <phemmer(a)devo0402.dev.usa.net> Rcpt: <phemmer(a)devo0402.dev.usa.net> Subject: ''Re: asdf'' Size: 22400 Response: 250 Mail accepted (292oFawiw3072Mem) ^^that is what is passed in the insert statement. As you can see, there are 2 single-quotes around the subject. The string as it gets passed to syslog-ng is: ...Subject: 'Re: asdf' Size... So instead of escaping the single quote, it adds another quote. The afsql.c code calls dbi_conn_quote_string_copy to escape it, but this is obviously failing to do so (and not only fails, but for some reason adds a second quote. I'm attempting to find a simple fix for this, but there doesnt appear to be a clean & proper way of doing this. But I'll keep digging as this is a fairly significant flaw which I have to get fixed before using this in production. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

syslog-ng
by Kaushal Shriyan 03 Jun '10

03 Jun '10

Hi, Whats the message size a syslog-ng application can accommodate ? Thanks, Kaushal

2 1

log smtp mail messages to a specific file
by Thanasis 02 Jun '10

02 Jun '10

Postfix messages were logged in /var/log/messages. Here is how syslog-ng.conf was (before my changes): ---------------------------------------------------- # cat /etc/syslog-ng/syslog-ng.conf.orig @version: 3.0 # $Header: /var/cvsroot/gentoo-x86/app-admin/syslog-ng/files/syslog-ng.conf.gentoo.3,v 1.1 2010/04/06 02:11:35 mr_bones_ Exp $ # # Syslog-ng default configuration file for Gentoo Linux options { chain_hostnames(no); # The default action of syslog-ng is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # how many messages syslog-ng missed (0). stats_freq(43200); }; source src { unix-stream("/dev/log" max-connections(256)); internal(); file("/proc/kmsg"); }; destination messages { file("/var/log/messages"); }; # By default messages are logged to tty12... destination console_all { file("/dev/tty12"); }; # ...if you intend to use /dev/console for programs like xconsole # you can comment out the destination line above that references /dev/tty12 # and uncomment the line below. #destination console_all { file("/dev/console"); }; log { source(src); destination(messages); }; log { source(src); destination(console_all); }; ------------------------------------------------- I wanted to log postfix messages to a new file: /var/log/mail.log I changed /etc/syslog-ng/syslog-ng.conf like so: # diff syslog-ng.conf.orig syslog-ng.conf 15a16,23 > filter mail { > facility(mail); > }; > > filter notmail { > not facility(mail); > }; > 22a31 > destination mail { file("/var/log/mail.log"); }; 31,32c40,42 < log { source(src); destination(messages); }; < log { source(src); destination(console_all); }; --- > log { source(src); filter(mail); destination(mail); }; > log { source(src); filter(notmail); destination(messages); }; > log { source(src); filter(notmail); destination(console_all); }; ---------------------------------------------------------------------- It worked, or at least I think so, by looking at both /var/log/messages and /var/log/mail.log. So ,my questions are: 1) Are my edits OK? Did I do anything wrong? 2) Should I have configured it otherwise, perhaps more efficiently? Thanks. Thanasis

3 4