- syslog-ng - lists.balabit.hu

preserving tags across the network
by Matt Zagrabelny 24 Nov '11

24 Nov '11

Hi, I am trying to ship a bunch of apache log files across the network and on the syslog-ng server side then break them into their individual files again. I am not sure the best way to do this, but it looks like tags might be helpful. On my apache system I have the following snippets: -----{syslog-ng client begin}----- source s_apache_access { file("/var/log/apache2/access.log" tags("main_access") ); }; source s_apache_other_vhost_access { file("/var/log/apache2/other_vhost.log" tags("other_vhost_access") ); }; destination d_server { syslog( "10.0.0.1" transport("tls") port(6514) tls( peer-verify(required-trusted) ca_dir('/etc/syslog-ng/ssl/ca.d') key_file('/etc/syslog-ng/ssl/server.key') cert_file('/etc/syslog-ng/ssl/server.crt') ) ); }; log { source(s_apache_access); source(s_apache_other_vhost_access); destination(d_server); }; -----{syslog-ng client end}----- On my server I have the following: -----{syslog-ng server begin}----- source s_tls { syslog( ip(0.0.0.0) port(6514) transport("tls") tls( peer-verify(required-trusted) ca_dir('/etc/syslog-ng/ssl/ca.d') key_file('/etc/syslog-ng/ssl/server.key') cert_file('/etc/syslog-ng/ssl/server.crt') ) max_connections(1000) keep_hostname(yes) ); }; filter f_main_apache_access { tags("main_access"); }; destination d_main_access { file("/var/log/apache2/access.log"); }; log { source(s_tls); filter(f_main_apache_access); destination(d_main_access); }; -----{syslog-ng server end}----- It doesn't look like the "tag" is being preserved through the network. Does anyone know if tags is the right mechanism for splitting apart (filtering) the aggregate stream coming in on the server? Any other advice would be greatly appreciated. Thanks! -matt zagrabelny

4 6

How to detect loss of connection of remote syslog-ng client.
by Pramod Pillai 24 Nov '11

24 Nov '11

Hi I am using syslog-ng server on Solaris box and the syslog-ng clients are running on Linux box. When client ( linux ) box crashes the streaming stops. But the syslog-ng server assumes that the client is connected and keeps the connection. Is there any way to detect this condition and close the socket connection to the client which no longer exists. Regards Pramod

2 2

[PATCH 0/1]: value-pairs: key rewriting support
by Gergely Nagy 24 Nov '11

24 Nov '11

After going over the code a few more times, and after a quick chat with Bazsi, I cleaned up the patch considerably. So much so, that I'm actually starting to like it! Functionality remained the same, but the code behind it is shorter and contains less magic. Instead of playing tricks with both scratch buffers and allocating gchar* strings all over the place, the transformation functions now operate directly on scratch buffers, and we steal the results. The patch coming in a follow-up message is something I consider ready for review. The only thing left to do is adding a couple of test cases (there's already a simple one included, but I'd like a few more).

1 1

Enterprise Log Search and Archive (ELSA) Beta Available
by Martin Holste 23 Nov '11

23 Nov '11

The new auto-installer script is working well enough that I think most people should be able to get the beta of ELSA installed now. I put up a short post with the details and screenshots on my blog here: http://ossectools.blogspot.com/2011/11/elsa-beta-available.html . >From the project page (http://code.google.com/p/enterprise-log-search-and-archive/) Features: * High-volume receiving/indexing (a single node can receive > 30k logs/sec, sustained) * Full Active Directory/LDAP integration for authentication, authorization, email settings * Instant ad-hoc reports/graphs on arbitrary queries even on enormous data sets * Email alerting, scheduled reports * Plugin architecture for web interface * Distributed architecture for clusters * Ships with normalization for some Cisco logs, Snort/Suricata, Bro, and Windows via Eventlog-to-Syslog or Snare Many thanks to Peter for helping me test the auto-installer, as well as all of Balabit for providing such an amazingly fast syslog parser with PatternDB. Please let me know if you run into issues installing. Ubuntu, openSUSE, and CentOS have been tested, but variations of those distros should work fine.

1 0

Query regarding syslog-ng.conf and creation of logging files
by Ankit Mahawar 23 Nov '11

23 Nov '11

Hi, Please help regarding doubts of syslog-ng.conf file 1 . The destination files specified in syslog-ng.conf is created by syslog-ng ?? 2. If they are created by syslog-ng then , are they created only when the log gets filtered for that destination file ? 3 . Does this happen in syslog as well,because I think in syslog the file gets created even before logging into the file I am new to syslog utilities and it is just my assumption that the files might be created using the conf file ,because when I killed syslog daemon and restarted it ,the file I removed got created again but not in the case of syslog-ng. Please help regarding this issue . Thanks Ankit =====-----=====-----===== Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you

5 5

rpmbuild on centos6
by Hajdu Tamás Márk 23 Nov '11

23 Nov '11

Hi everyone! I wanted to build rpm-s from syslog-ng (and maintain it in the future). I've using eventlog_0.2.12 (from source) which builds with some fiddling, but syslog-ng_3.3.3 (also from source) won't build. [rpmbuilder@svn ~]$ rpmbuild -ta rpmbuild/SOURCES/syslog-ng_3.3.3.tar.gz error: File /home/fathom/rpmbuild/SOURCES/ivykis-0.21.tar.gz: No such file or directory I have not found anything like ivykis in the .spec file, but I have downloaded it: [rpmbuilder@svn ~]$ rpmbuild -ta rpmbuild/SOURCES/syslog-ng_3.3.3.tar.gz error: line 135: second %prep Somehow it makes one spec file from the syslog-ng spec and the ivykis spec file... I tried to other way as well. Building it from the spec file cries for dependencies, including libevtlog and libnet-devel. The first is ok, but I do not really want to install that on the rpm building system, but the second one is totally missing from centos6. Has anyone some clues about this? It would be a pain in the ass to manually install/update everywhere. Hajdu Tamás Márk

2 2

Query regarding syslog-ng.conf and creation of logging files
by Ankit Mahawar 23 Nov '11

23 Nov '11

Hi, Please help regarding doubts of syslog-ng.conf file 1 . The destination files specified in syslog-ng.conf is created by syslog-ng ?? 2. If they are created by syslog-ng then , are they created only when the log gets filtered for that destination file 3 . Does this happen in syslog as well,because I think in syslog the file gets created even before logging into the file I am new to syslog utilities and it is just my assumption that the files might be created using the conf file ,because when I killed syslog daemon and restarted it ,the file I removed got created again but not in the case of syslog-ng. Please help regarding this issue .. Thanks Ankit =====-----=====-----===== Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you

1 0

Cisco IOS message format
by Wells, James 22 Nov '11

22 Nov '11

Hi All, I am struggling to get syslog-ng to output the correct format for Cisco IOS devices. I am using syslog-ng to forward message to a NMS system. The issue I see is syslog-ng upon forwarding adds more data to the message as I am assuming it does not understand the Cisco IOS syslog format. Version of syslog-ng: Name : syslog-ng Relocations: (not relocatable) Version : 2.1.4 Vendor: Fedora Project Release : 9.el5 Build Date: Mon 16 May 2011 15:09:47 BST Install Date: Fri 21 Oct 2011 12:26:04 BST Build Host: x86-01.phx2.fedoraproject.org Group : System Environment/Daemons Source RPM: syslog-ng-2.1.4-9.el5.src.rpm Cisco IOS statements: service timestamps log datetime msec localtime show-timezone logging trap notifications logging facility local6 Syslog format in the local file: Nov 10 10:18:44.102 UTC: %SYS-5-CONFIG_I: Configured from console by testuser on vty0 (1.2.3.4) Syslog-ng conf file: options { sync(0); time_reopen(10); log_fifo_size(1000); long_hostnames(off); check_hostname(yes); keep_hostname(yes); chain_hostnames(no); use_time_recvd(yes); }; template("$MSGONLY\n") When I perform a TCPDUMP and view the incoming message and then the forwarded message I can see that syslog-ng adds more data to the MESSAGE aspect of the syslog. Has anyone been able to create a filter or template that manages this format, so that the forwarding of the syslog onto another receiver is not changed as syslog-ng is adding the $DATE and $HOST to the message. Thanks in advance James _______________________________________________________________________ This email is intended only for the use of the individual(s) to whom it is addressed and may be privileged and confidential. Unauthorised use or disclosure is prohibited. If you receive this e-mail in error, please advise immediately and delete the original message without copying, using, or telling anyone about its contents. This message may have been altered without your or our knowledge and the sender does not accept any liability for any errors or omissions in the message. This message does not create or change any contract. Royal Bank of Canada and its subsidiaries accept no responsibility for damage caused by any viruses contained in this email or its attachments. Emails may be monitored. RBC Capital Markets is a business name used by branches and subsidiaries of Royal Bank of Canada, including Royal Bank of Canada, London branch and RBC Europe Limited. In accordance with English law requirements, details regarding RBC Europe Limited are set out below: RBC EUROPE LIMITED Registered in England and Wales 995939 Registered Address: Riverbank House, 2 Swan Lane, London, EC4R 3BF. Authorised and regulated by the Financial Services Authority. Member of the London Stock Exchange.

2 2

[PATCH 0/1]: value-pairs key rewriting for 3.4
by Gergely Nagy 21 Nov '11

21 Nov '11

The patch I'll be sending in a followup implements key rewriting for syslog-ng 3.4. I wrote about this quite a bit in the past, and this time, there are no functional changes - it's just the work I've done so far merged into a single patch. ...and updated a bit, because during the merge, I discovered that the transformations are not thread safe, so I took the opportunity to convert them to scratch buffers too. This, however, was done in a bit of a hurry, so I'm not entirely happy with the result. Nevertheless, I'm submitting it as a single patch, mostly so that it can be reviewed in its entirety. I don't think it's ripe for a merge yet, there's a bit of work left to done still, at least in the code part. The functionality itself is - I believe - solid by now. The former branch I worked on (feature/3.4/value-pairs/rekey) will be deleted soon, as it does not contain the latest fixes I applied during the commit merging process. -- |8]

1 4

Fwd: bug in syslog-ng 3.2 and 3.3
by Gergely Nagy 21 Nov '11

21 Nov '11

The mail below was originally sent to one of our internal addresses, but since it's a bug report against 3.2 and 3.3, it is better to have it posted here too (with original files attached): Da Mota Sam <da.mota.sam(a)gmail.com> wrote: > > Hi, > > I think I found a bug in syslog-ng starting from version 3.2. > > You will find in attachment the smallest configuration files for which > the bug appears. > These files must be placed in /etc/syslog-ng "to work". > > The problem seems to be in a bad handling of characters with accents. > > With the attached configuration files, when syslog-ng receives a > message containing just "1 é" > (for exemple by calling "logger '1 é') one should expect the creation > of a file /var/log/message_with_id_matched > This file isn't actually created. > > The given file pattern_rules.xml contains two patterns. If I remove > one of them, then I get the creation of the file > /var/log/message_with_id_matched, which makes me think that one of the > pattern shadows the other one. > > > About: > I got this erroneous behaviour with: > - syslog-ng 3.2.4 on debian wheezy (from debian packages) > - syslog-ng 3.2.4 on debian lenny (from syslog-ng source code) > - syslog-ng 3.2.4 on debian squeeze (from syslog-ng source code) > - syslog-ng 3.2.5 on debian squeeze (from syslog-ng source code) > - syslog-ng 3.3.1 on debian wheezy (using the packages of debian sid) > > the bug was not present on syslog-ng 3.1 on debian squeeze (from > debian packages) > > Could you please check that you get the same errors, and let me know about it? > (so I could bug report this to the debian team if it is not a syslog-ng bug). > > > Best regards, > -- > Samuel Da Mota -- |8]

1 0