November 2011 - syslog-ng - lists.balabit.hu

[PATCH] logrewrite: Fix the logrewrite cloning functions.
by Gergely Nagy 25 Nov '11

25 Nov '11

Repair the rewrite clone funcions, to correctly clone the value_handle and conditions aswell. Without this, if we use rewrite in multiple logpaths, only the first will behave correctly, the cloned ones will all rewrite the $MESSAGE macro instead, because that's the default. Signed-off-By: Zoltan Fried <deirf(a)balabit.hu> Signed-off-By: Gergely Nagy <algernon(a)balabit.hu> --- lib/logrewrite.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/lib/logrewrite.c b/lib/logrewrite.c index d282f6f..a578439 100644 --- a/lib/logrewrite.c +++ b/lib/logrewrite.c @@ -165,6 +165,8 @@ log_rewrite_subst_clone(LogProcessPipe *s) cloned = (LogRewriteSubst *) log_rewrite_subst_new(self->replacement->template); cloned->matcher = log_matcher_ref(self->matcher); + cloned->super.value_handle = self->super.value_handle; + cloned->super.condition = self->super.condition; return &cloned->super.super.super; } @@ -227,6 +229,8 @@ log_rewrite_set_clone(LogProcessPipe *s) LogRewriteSet *cloned; cloned = (LogRewriteSet *) log_rewrite_set_new(self->value_template->template); + cloned->super.value_handle = self->super.value_handle; + cloned->super.condition = self->super.condition; return &cloned->super.super.super; } -- 1.7.7.3

2 1

[PATCH] Fixed compounded filter initialization.
by Szalay Attila 25 Nov '11

25 Nov '11

If a filter link to an other filter an init function called to set up this link. But if this filter is compunded then this callback is not called so the filter is not initialized but silently evaluated to false. For example in the following situation the f_message is always true: filter f_auth { facility(auth, authpriv); }; filter f_cron { facility(cron); }; filter f_mail { facility(mail); }; filter f_messages { not filter(f_auth) and not filter(f_cron) and not filter(f_mail); }; Signed-off-by: Szalay Attila <sasa(a)balabit.hu> --- lib/filter.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 39 insertions(+), 0 deletions(-)

2 1

preserving tags across the network
by Matt Zagrabelny 24 Nov '11

24 Nov '11

Hi, I am trying to ship a bunch of apache log files across the network and on the syslog-ng server side then break them into their individual files again. I am not sure the best way to do this, but it looks like tags might be helpful. On my apache system I have the following snippets: -----{syslog-ng client begin}----- source s_apache_access { file("/var/log/apache2/access.log" tags("main_access") ); }; source s_apache_other_vhost_access { file("/var/log/apache2/other_vhost.log" tags("other_vhost_access") ); }; destination d_server { syslog( "10.0.0.1" transport("tls") port(6514) tls( peer-verify(required-trusted) ca_dir('/etc/syslog-ng/ssl/ca.d') key_file('/etc/syslog-ng/ssl/server.key') cert_file('/etc/syslog-ng/ssl/server.crt') ) ); }; log { source(s_apache_access); source(s_apache_other_vhost_access); destination(d_server); }; -----{syslog-ng client end}----- On my server I have the following: -----{syslog-ng server begin}----- source s_tls { syslog( ip(0.0.0.0) port(6514) transport("tls") tls( peer-verify(required-trusted) ca_dir('/etc/syslog-ng/ssl/ca.d') key_file('/etc/syslog-ng/ssl/server.key') cert_file('/etc/syslog-ng/ssl/server.crt') ) max_connections(1000) keep_hostname(yes) ); }; filter f_main_apache_access { tags("main_access"); }; destination d_main_access { file("/var/log/apache2/access.log"); }; log { source(s_tls); filter(f_main_apache_access); destination(d_main_access); }; -----{syslog-ng server end}----- It doesn't look like the "tag" is being preserved through the network. Does anyone know if tags is the right mechanism for splitting apart (filtering) the aggregate stream coming in on the server? Any other advice would be greatly appreciated. Thanks! -matt zagrabelny

4 6

How to detect loss of connection of remote syslog-ng client.
by Pramod Pillai 24 Nov '11

24 Nov '11

Hi I am using syslog-ng server on Solaris box and the syslog-ng clients are running on Linux box. When client ( linux ) box crashes the streaming stops. But the syslog-ng server assumes that the client is connected and keeps the connection. Is there any way to detect this condition and close the socket connection to the client which no longer exists. Regards Pramod

2 2

[PATCH 0/1]: value-pairs: key rewriting support
by Gergely Nagy 24 Nov '11

24 Nov '11

After going over the code a few more times, and after a quick chat with Bazsi, I cleaned up the patch considerably. So much so, that I'm actually starting to like it! Functionality remained the same, but the code behind it is shorter and contains less magic. Instead of playing tricks with both scratch buffers and allocating gchar* strings all over the place, the transformation functions now operate directly on scratch buffers, and we steal the results. The patch coming in a follow-up message is something I consider ready for review. The only thing left to do is adding a couple of test cases (there's already a simple one included, but I'd like a few more).

1 1

Enterprise Log Search and Archive (ELSA) Beta Available
by Martin Holste 23 Nov '11

23 Nov '11

The new auto-installer script is working well enough that I think most people should be able to get the beta of ELSA installed now. I put up a short post with the details and screenshots on my blog here: http://ossectools.blogspot.com/2011/11/elsa-beta-available.html . >From the project page (http://code.google.com/p/enterprise-log-search-and-archive/) Features: * High-volume receiving/indexing (a single node can receive > 30k logs/sec, sustained) * Full Active Directory/LDAP integration for authentication, authorization, email settings * Instant ad-hoc reports/graphs on arbitrary queries even on enormous data sets * Email alerting, scheduled reports * Plugin architecture for web interface * Distributed architecture for clusters * Ships with normalization for some Cisco logs, Snort/Suricata, Bro, and Windows via Eventlog-to-Syslog or Snare Many thanks to Peter for helping me test the auto-installer, as well as all of Balabit for providing such an amazingly fast syslog parser with PatternDB. Please let me know if you run into issues installing. Ubuntu, openSUSE, and CentOS have been tested, but variations of those distros should work fine.

1 0

Query regarding syslog-ng.conf and creation of logging files
by Ankit Mahawar 23 Nov '11

23 Nov '11

Hi, Please help regarding doubts of syslog-ng.conf file 1 . The destination files specified in syslog-ng.conf is created by syslog-ng ?? 2. If they are created by syslog-ng then , are they created only when the log gets filtered for that destination file ? 3 . Does this happen in syslog as well,because I think in syslog the file gets created even before logging into the file I am new to syslog utilities and it is just my assumption that the files might be created using the conf file ,because when I killed syslog daemon and restarted it ,the file I removed got created again but not in the case of syslog-ng. Please help regarding this issue . Thanks Ankit =====-----=====-----===== Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you

5 5

rpmbuild on centos6
by Hajdu Tamás Márk 23 Nov '11

23 Nov '11

Hi everyone! I wanted to build rpm-s from syslog-ng (and maintain it in the future). I've using eventlog_0.2.12 (from source) which builds with some fiddling, but syslog-ng_3.3.3 (also from source) won't build. [rpmbuilder@svn ~]$ rpmbuild -ta rpmbuild/SOURCES/syslog-ng_3.3.3.tar.gz error: File /home/fathom/rpmbuild/SOURCES/ivykis-0.21.tar.gz: No such file or directory I have not found anything like ivykis in the .spec file, but I have downloaded it: [rpmbuilder@svn ~]$ rpmbuild -ta rpmbuild/SOURCES/syslog-ng_3.3.3.tar.gz error: line 135: second %prep Somehow it makes one spec file from the syslog-ng spec and the ivykis spec file... I tried to other way as well. Building it from the spec file cries for dependencies, including libevtlog and libnet-devel. The first is ok, but I do not really want to install that on the rpm building system, but the second one is totally missing from centos6. Has anyone some clues about this? It would be a pain in the ass to manually install/update everywhere. Hajdu Tamás Márk

2 2

Query regarding syslog-ng.conf and creation of logging files
by Ankit Mahawar 23 Nov '11

23 Nov '11

Hi, Please help regarding doubts of syslog-ng.conf file 1 . The destination files specified in syslog-ng.conf is created by syslog-ng ?? 2. If they are created by syslog-ng then , are they created only when the log gets filtered for that destination file 3 . Does this happen in syslog as well,because I think in syslog the file gets created even before logging into the file I am new to syslog utilities and it is just my assumption that the files might be created using the conf file ,because when I killed syslog daemon and restarted it ,the file I removed got created again but not in the case of syslog-ng. Please help regarding this issue .. Thanks Ankit =====-----=====-----===== Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you

1 0

Cisco IOS message format
by Wells, James 22 Nov '11

22 Nov '11

Hi All, I am struggling to get syslog-ng to output the correct format for Cisco IOS devices. I am using syslog-ng to forward message to a NMS system. The issue I see is syslog-ng upon forwarding adds more data to the message as I am assuming it does not understand the Cisco IOS syslog format. Version of syslog-ng: Name : syslog-ng Relocations: (not relocatable) Version : 2.1.4 Vendor: Fedora Project Release : 9.el5 Build Date: Mon 16 May 2011 15:09:47 BST Install Date: Fri 21 Oct 2011 12:26:04 BST Build Host: x86-01.phx2.fedoraproject.org Group : System Environment/Daemons Source RPM: syslog-ng-2.1.4-9.el5.src.rpm Cisco IOS statements: service timestamps log datetime msec localtime show-timezone logging trap notifications logging facility local6 Syslog format in the local file: Nov 10 10:18:44.102 UTC: %SYS-5-CONFIG_I: Configured from console by testuser on vty0 (1.2.3.4) Syslog-ng conf file: options { sync(0); time_reopen(10); log_fifo_size(1000); long_hostnames(off); check_hostname(yes); keep_hostname(yes); chain_hostnames(no); use_time_recvd(yes); }; template("$MSGONLY\n") When I perform a TCPDUMP and view the incoming message and then the forwarded message I can see that syslog-ng adds more data to the MESSAGE aspect of the syslog. Has anyone been able to create a filter or template that manages this format, so that the forwarding of the syslog onto another receiver is not changed as syslog-ng is adding the $DATE and $HOST to the message. Thanks in advance James _______________________________________________________________________ This email is intended only for the use of the individual(s) to whom it is addressed and may be privileged and confidential. Unauthorised use or disclosure is prohibited. If you receive this e-mail in error, please advise immediately and delete the original message without copying, using, or telling anyone about its contents. This message may have been altered without your or our knowledge and the sender does not accept any liability for any errors or omissions in the message. This message does not create or change any contract. Royal Bank of Canada and its subsidiaries accept no responsibility for damage caused by any viruses contained in this email or its attachments. Emails may be monitored. RBC Capital Markets is a business name used by branches and subsidiaries of Royal Bank of Canada, including Royal Bank of Canada, London branch and RBC Europe Limited. In accordance with English law requirements, details regarding RBC Europe Limited are set out below: RBC EUROPE LIMITED Registered in England and Wales 995939 Registered Address: Riverbank House, 2 Swan Lane, London, EC4R 3BF. Authorised and regulated by the Financial Services Authority. Member of the London Stock Exchange.

2 2