- syslog-ng - lists.balabit.hu

Anyone Going to Cisco Live/Networkers in Las Vegas next week?
by Clayton Dukes 22 Jul '11

22 Jul '11

Hey everyone, If any of you are attending the Cisco Live conference in Las Vegas next week and are interested in learning more about building scalable syslog management solutions, please feel free to sign up for my session: BRKNMS-2031. I will be covering step-by-step instructions on how to set up a system using syslog-ng. If you can't make the session, there is also a Lunch/Table topic you can sign up for just to pick my brain (what little there is, heh). See you there! ______________________________________________________________ Clayton Dukes ______________________________________________________________

2 5

Severity/Pri/Level
by Paul Muther 22 Jul '11

22 Jul '11

Hello all, Just joined the list. I was wondering if there was a method to rewrite the severity of an event based on the text of the entry? I know I can use regex to read the message but from what I have seen I can't set the severity level of an event. In reading posts it would appear it was a feature under consideration in the past. Thanks much, PM

3 4

3.3b1: "EOF occurred while idle"
by Nix 22 Jul '11

22 Jul '11

I'm seeing a lot of these: May 29 18:59:48 fold err: POLLERR occurred while idle; fd='25' [...] May 29 19:04:13 fold err: EOF occurred while idle; fd='25' (It also just coredumped, but this led to an autoreboot and the log was not preserved across the reboot :( ) -- NULL && (void)

2 7

cannot filter by program name
by Luis Pugoy 21 Jul '11

21 Jul '11

Hello. I am having trouble filtering by program name and was wondering if anyone could help. I am using syslog-ng v3.1.2 on Ubuntu 10.04 machines. I installed syslog-ng by installing the provided deb package. In Machine 1, the relevant lines in syslog-ng.conf are below: source s_tag_a { file('/home/ubuntu/a.tag' flags(no-parse) program_override('atag')); }; destination d_tag_a { tcp('machine2' port(40000)); }; log { source(s_tag_a); destination(d_tag_a); }; source s_tag_b { file('/home/ubuntu/b.tag' flags(no-parse) program_override('btag')); }; destination d_tag_b { tcp('machine2' port(40000)); }; log { source(s_tag_b); destination(d_tag_b); }; In Machine 2, the relevant lines in syslog-ng.conf are below: source s_tag { tcp(ip(0.0.0.0) port(40000)); }; destination d_tag_a { file('/home/ubuntu/a.tag'); }; filter f_tag_a { program('atag'); }; log { source(s_tag); destination(d_tag_a); filter(f_tag_a); }; destination d_tag_b { file('/home/ubuntu/b.tag'); }; filter f_tag_b { program('btag'); }; log { source(s_tag); destination(d_tag_b); filter(f_tag_b); }; Now when I execute the following commands in Machine 1: # echo 'atag' >> a.tag # echo 'btag' >> b.tag In Machine 2, the files a.tag and b.tag both contain the given lines. # cat a.tag atag btag # cat b.tag atag btag Is there something wrong in my configuration? Thank you.

1 1

Lot of log lost
by maxime.denier＠orange-ftgroup.com 20 Jul '11

20 Jul '11

Hello, I have recently installed syslog-ng OSE 3.1 as log collector and I face a problem. A great number of logs arrive on the server, but a little part of them arrive in the destination files, but all the destination files have some logs on it. I have enabled the verbose mode and I see this: Jul 20 07:52:04 sparte1 syslog-ng[2557]: Initializing destination file writer; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/zidane2/201107202400.csv' Jul 20 07:52:31 sparte1 syslog-ng[2557]: Reaping unused destination files; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Reaping unused destination files; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Destination timed out, reaping; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/peony2/201107202400.csv' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Closing log transport fd; fd='31' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Destination timed out, reaping; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/decca2/201107202400.csv' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Closing log transport fd; fd='19' Jul 20 07:53:16 sparte1 syslog-ng[2557]: Initializing destination file writer; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/hyenne2/201107202400.csv' Jul 20 07:53:17 sparte1 syslog-ng[2557]: Initializing destination file writer; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/olive2/201107202400.csv' Jul 20 07:53:31 sparte1 syslog-ng[2557]: Reaping unused destination files; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv' Jul 20 07:53:31 sparte1 syslog-ng[2557]: Destination timed out, reaping; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/zidane2/201107202400.csv' Jul 20 07:53:31 sparte1 syslog-ng[2557]: Closing log transport fd; fd='24' I haven't found information about the root cause of these timed out. This seems to be a writing problem. Before using syslog-ng, Logs were processed by a application owned by the firewall publisher on the same type of hardware without this great number of log lost. If any body have already faced this problem and have a solution, it would be wonderful. Regards, Maxime Denier ******************************************************************************** IMPORTANT.Les informations contenues dans ce message electronique y compris les fichiers attaches sont strictement confidentielles et peuvent etre protegees par la loi. Ce message electronique est destine exclusivement au(x) destinataire(s) mentionne(s) ci-dessus. Si vous avez recu ce message par erreur ou s il ne vous est pas destine, veuillez immediatement le signaler a l expediteur et effacer ce message et tous les fichiers eventuellement attaches. Toute lecture, exploitation ou transmission des informations contenues dans ce message est interdite. Tout message electronique est susceptible d alteration. A ce titre, le Groupe France Telecom decline toute responsabilite notamment s il a ete altere, deforme ou falsifie. De meme, il appartient au destinataire de s assurer de l absence de tout virus. IMPORTANT.This e-mail message and any attachments are strictly confidential and may be protected by law. This message is intended only for the named recipient(s) above. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender and delete this e-mail message. Any unauthorized view, usage or disclosure ofthis message is prohibited. Since e-mail messages may not be reliable, France Telecom Group shall not be liable for any message if modified, changed or falsified. Additionally the recipient should ensure they are actually virus free. ********************************************************************************

3 2

max-connections(1000)
by Josu Lazkano 20 Jul '11

20 Jul '11

Hello list, this morning I had some problems with my syslog system I am adding lots of server to send TCP logs to the syslog-ng server. But there are not on the server. After readingon the web, I notice that there is a max-connections parameter on the TCP. So I fix to 50. With this there are more logs, then I added 100 and there more. So I fix it to 1000: source s_all { internal(); unix-stream("/dev/log"); file("/proc/kmsg" log_prefix("kernel: ")); tcp(max-connections(1000)); }; Is this a good idea? How can I check if this is a good maximum connection value? Thanks for all your help and best regards.

4 5

Securing syslog-ng
by Kārlis Repsons 20 Jul '11

20 Jul '11

All, I've been sorting system information with syslog-ng just fine, but it happened in a LAN environment. Now I plan starting off with a public IPv6 address and that raises some concerns. What would you advice to check to be sure? How should remote logging be set up so that some mutual (or at least client) cryptographic authentication happens? Thanks...

4 3

syslogd as well as syslog-ng running simultaneously
by Rakesh Rajasekharan 20 Jul '11

20 Jul '11

Hi, I have compiled syslog-ng 3.0.7 successfully on an AIX machine. But I have noticed that I now have both traditional syslog syslogd as well as syslog-ng daemons running simultaneously. root 237722 139372 0 Mar 24 - 2:34 /opt/soe/local/syslog_ng/sbin/syslog-ng -F -f /opt/soe/local/syslog_ng/etc/syslog-ng.conf -p /var/run/syslog-ng.pid root 307444 139372 0 Apr 04 - 0:03 /usr/sbin/syslogd Is this fine or ther is something that has to be done to fix it. Please let me know. Thanks, Rakesh

3 2

AIX Syslog Messages
by Ricardo Oliveira 20 Jul '11

20 Jul '11

Hi, I'm having some problems properly storing messages received from AIX servers. The format which they come in is like this: "Jul 5 19:30:59 Message forwarded from server2: su: from root to ..." According to a thread on this mailing list (https://lists.balabit.hu/pipermail/syslog-ng/2006-October/009372.html) and if I understood correctly, this should be OK, and I should get the expected behaviour of replacing this with the form: "Jul 5 19:30:59 server2 su: from root to ..." However, what I get in the log is: "Jul 5 19:30:59 192.168.1.1 su: from root to ..." Where the 192.168.1.1 is the IP of the machine I got the message from and not the name of the server (server2 in this case). The issue here is that these messages belong to several machines which are sending their syslog messages to a NIM server which in turn forwards them to our syslog server, so the IP we end up with is not the machine's IP, but rather the NIM server IP, which is not what we need. I tried parsing the message on arrival, but it doesn't work, I suppose it's because syslog-ng processes it before the parsers kick in. Is there a way to do this? TIA, Ricardo.

3 4

Third-party module collection updates
by Gergely Nagy 20 Jul '11

20 Jul '11

Hi! A repository for third-party and experimental modules was announced in May[0], and I'd like to give a quick update about what's been brewing there since then: * The SMTP destination has been updated a few times for newer versions of syslog-ng 3.3. As of this writing, it's up to date with current 3.3. * The MongoDB destination was copied there too, as its easier to maintain one copy, instead of a branch for 3.3 and 3.4 each. It's also been updated to use recent versions of the libmongo-client library, so it can take advantage of features added in the past few months, such as the ability to use Replica Sets and safe mode (see my earlier RFC[1] about these - what's mentioned there is implemented now). The host() and port() parameters are now gone, and no backward compatibility exists yet. That will be fixed at a later time. * On the modules/rss branch, there is an initial RSS destination by Viktor Tusa <tusa(a)balabit.hu>. It has a few serious issues, like not escaping the log messages, and outputting an invalid format, but those will be corrected Soon. There's not much one can use it for just yet, but it's going to be a neat toy, nevertheless! * The log visualisation helper[2] is included in tools/visualisation/. By the way, a new demo video is also available at http://vimeo.com/25700679, which shows ~3 months of logs in 50 minutes. Quite a bit longer than the previous video, and made with different gource settings, to be a bit less messy. For the time being, the module collection is primarily developed against syslog-ng 3.3. The modules have a high chance of working with 3.4 aswell, but that's not tested, and only minimal effort is made to be compatible with both. I'll try to figure out an easy way to properly support both versions in the future, but at the moment, the focus is on 3.3. 0: http://article.gmane.org/gmane.comp.syslog-ng/11403 1: http://article.gmane.org/gmane.comp.syslog-ng/11557 2: http://article.gmane.org/gmane.comp.syslog-ng/11569 -- |8]

2 1