- syslog-ng - lists.balabit.hu

Rapid filling of /var/log/syslog-ng.log with "--MARK--" messages?
by mike＠lague.org 25 Jul '11

25 Jul '11

On a system running syslog-ng 3.0.5, we observed the following: * * * Jul 21 11:53:37.724586 sys89 -- MARK -- Jul 21 11:53:37.724623 sys89 -- MARK -- Jul 21 11:53:37.724660 sys89 -- MARK -- Jul 21 11:53:37.724696 sys89 -- MARK -- Jul 21 11:53:37.724733 sys89 -- MARK -- Jul 21 11:53:37.724770 sys89 -- MARK -- Jul 21 11:53:37.724807 sys89 -- MARK -- Jul 21 11:53:37.724843 sys89 -- MARK -- Jul 21 11:53:37.724880 sys89 -- MARK ? The log was written to at this frequency, with these "MARK" messages, until the partition filled up. It appears that the first of these messages was logged very close in time to when the system date was set back by a couple of minutes. However, I was unable to reproduce this result on a different system (by setting the date back in time.) However, is it possible that this problem is the same as: SYSLOG-NG] [BUG 110] NEW: ADVANCING THE SYSTEM DATE AND TIME BEYOND 2038 CAUSES FLOOD OF MESSAGES (i.e., https://lists.balabit.hu/pipermail/syslog-ng/2011-January/015754.html) It appears that this is fixed in 3.2.3; it is fixed in other versions? In versions where this is not fixed, is it possible to avoid this problem by turning off the "mark" feature, i.e., by putting mark_freq(0); in the options clause of the configuration? Thank you, --Mike Lague

1 0

Re: [syslog-ng] log file size limit
by Hendrik Visage 25 Jul '11

25 Jul '11

Better to make use of monthly, weekly, daily, hourly type logs in syslog-ng, and the to use cront to clean up as needed ;) On Mon, Jul 25, 2011 at 5:50 PM, William Tambellini <William.Tambellini(a)galaxysemi.com> wrote: > Dear Syslog-ng team, > > We are currently evaluating syslog-ng in order to satisfy some requests > from our customers regarding logs from our software. > > One of their requests is to limit log file size using some kind of rotation > algo : if size > 6Mo then rotate to another file. > > We have found some options in file destination like fifo buffer control but > nothing about files rotation. > > Is there anything for that kind of feature ? > > Regards > > WT. > > > > ---------------------- > > William Tambellini > > Lead Technical Organizer > > www.galaxysemi.com > > > > “Sometimes I think the surest sign that intelligent life exists elsewhere in > the universe is that none of it has tried to contact us.” > > Bill Waterson (Calvin & Hobbes) > > > > > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > >

1 0

Re: [syslog-ng] log file size limit
by Patrick H. 25 Jul '11

25 Jul '11

Syslog-ng doesnt perform file rotation. This would the the job of something like 'logrotate <https://fedorahosted.org/logrotate/>'. Sent: Mon Jul 25 2011 09:50:42 GMT-0600 (MST) From: William Tambellini <William.Tambellini(a)galaxysemi.com> To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] log file size limit > > Dear Syslog-ng team, > > We are currently evaluating syslog-ng in order to satisfy some > requests from our customers regarding logs from our software. > > One of their requests is to limit log file size using some kind of > rotation algo : if size > 6Mo then rotate to another file. > > We have found some options in file destination like fifo buffer > control but nothing about files rotation. > > Is there anything for that kind of feature ? > > Regards > > WT. > > ---------------------- > > William Tambellini > > Lead Technical Organizer > > www.galaxysemi.com > > "Sometimes I think the surest sign that intelligent life exists > elsewhere in the universe is that none of it has tried to contact us." > > Bill Waterson (Calvin & Hobbes) > > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq >

1 0

log file size limit
by William Tambellini 25 Jul '11

25 Jul '11

Dear Syslog-ng team, We are currently evaluating syslog-ng in order to satisfy some requests from our customers regarding logs from our software. One of their requests is to limit log file size using some kind of rotation algo : if size > 6Mo then rotate to another file. We have found some options in file destination like fifo buffer control but nothing about files rotation. Is there anything for that kind of feature ? Regards WT. ---------------------- William Tambellini Lead Technical Organizer www.galaxysemi.com "Sometimes I think the surest sign that intelligent life exists elsewhere in the universe is that none of it has tried to contact us." Bill Waterson (Calvin & Hobbes)

1 0

can syslog-ng extract part of the hostname and make directory on the fly?
by fedora fedora 22 Jul '11

22 Jul '11

I have few hundreds of hosts, and say hosta has dns name groupa-hosta hostb has dns name groupbhostb hostc has dns name groupc.hostc ..etc..etc (note the first 6 characters defines the group) When a log comes in, can syslog-ng extracts the first 6 character from dns name, and drop the log file to the corresponding group? (So log from hostb will go under /log/groupb/groupahostb ) I cannot predefine the groups though, since i don't know how many groups will be there, the only thing i know is the first 6 character of the hostname will be the group name thanks

3 2

Anyone Going to Cisco Live/Networkers in Las Vegas next week?
by Clayton Dukes 22 Jul '11

22 Jul '11

Hey everyone, If any of you are attending the Cisco Live conference in Las Vegas next week and are interested in learning more about building scalable syslog management solutions, please feel free to sign up for my session: BRKNMS-2031. I will be covering step-by-step instructions on how to set up a system using syslog-ng. If you can't make the session, there is also a Lunch/Table topic you can sign up for just to pick my brain (what little there is, heh). See you there! ______________________________________________________________ Clayton Dukes ______________________________________________________________

2 5

Severity/Pri/Level
by Paul Muther 22 Jul '11

22 Jul '11

Hello all, Just joined the list. I was wondering if there was a method to rewrite the severity of an event based on the text of the entry? I know I can use regex to read the message but from what I have seen I can't set the severity level of an event. In reading posts it would appear it was a feature under consideration in the past. Thanks much, PM

3 4

3.3b1: "EOF occurred while idle"
by Nix 22 Jul '11

22 Jul '11

I'm seeing a lot of these: May 29 18:59:48 fold err: POLLERR occurred while idle; fd='25' [...] May 29 19:04:13 fold err: EOF occurred while idle; fd='25' (It also just coredumped, but this led to an autoreboot and the log was not preserved across the reboot :( ) -- NULL && (void)

2 7

cannot filter by program name
by Luis Pugoy 21 Jul '11

21 Jul '11

Hello. I am having trouble filtering by program name and was wondering if anyone could help. I am using syslog-ng v3.1.2 on Ubuntu 10.04 machines. I installed syslog-ng by installing the provided deb package. In Machine 1, the relevant lines in syslog-ng.conf are below: source s_tag_a { file('/home/ubuntu/a.tag' flags(no-parse) program_override('atag')); }; destination d_tag_a { tcp('machine2' port(40000)); }; log { source(s_tag_a); destination(d_tag_a); }; source s_tag_b { file('/home/ubuntu/b.tag' flags(no-parse) program_override('btag')); }; destination d_tag_b { tcp('machine2' port(40000)); }; log { source(s_tag_b); destination(d_tag_b); }; In Machine 2, the relevant lines in syslog-ng.conf are below: source s_tag { tcp(ip(0.0.0.0) port(40000)); }; destination d_tag_a { file('/home/ubuntu/a.tag'); }; filter f_tag_a { program('atag'); }; log { source(s_tag); destination(d_tag_a); filter(f_tag_a); }; destination d_tag_b { file('/home/ubuntu/b.tag'); }; filter f_tag_b { program('btag'); }; log { source(s_tag); destination(d_tag_b); filter(f_tag_b); }; Now when I execute the following commands in Machine 1: # echo 'atag' >> a.tag # echo 'btag' >> b.tag In Machine 2, the files a.tag and b.tag both contain the given lines. # cat a.tag atag btag # cat b.tag atag btag Is there something wrong in my configuration? Thank you.

1 1

Lot of log lost
by maxime.denier＠orange-ftgroup.com 20 Jul '11

20 Jul '11

Hello, I have recently installed syslog-ng OSE 3.1 as log collector and I face a problem. A great number of logs arrive on the server, but a little part of them arrive in the destination files, but all the destination files have some logs on it. I have enabled the verbose mode and I see this: Jul 20 07:52:04 sparte1 syslog-ng[2557]: Initializing destination file writer; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/zidane2/201107202400.csv' Jul 20 07:52:31 sparte1 syslog-ng[2557]: Reaping unused destination files; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Reaping unused destination files; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Destination timed out, reaping; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/peony2/201107202400.csv' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Closing log transport fd; fd='31' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Destination timed out, reaping; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/decca2/201107202400.csv' Jul 20 07:53:01 sparte1 syslog-ng[2557]: Closing log transport fd; fd='19' Jul 20 07:53:16 sparte1 syslog-ng[2557]: Initializing destination file writer; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/hyenne2/201107202400.csv' Jul 20 07:53:17 sparte1 syslog-ng[2557]: Initializing destination file writer; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/olive2/201107202400.csv' Jul 20 07:53:31 sparte1 syslog-ng[2557]: Reaping unused destination files; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv' Jul 20 07:53:31 sparte1 syslog-ng[2557]: Destination timed out, reaping; template='/var/logs/${NSM.DEVICE:-Unknown_device}/${NSM.RECEIVED_TIME.YE AR}${NSM.RECEIVED_TIME.MONTH}${NSM.RECEIVED_TIME.DAY}2400.csv', filename='/var/logs/zidane2/201107202400.csv' Jul 20 07:53:31 sparte1 syslog-ng[2557]: Closing log transport fd; fd='24' I haven't found information about the root cause of these timed out. This seems to be a writing problem. Before using syslog-ng, Logs were processed by a application owned by the firewall publisher on the same type of hardware without this great number of log lost. If any body have already faced this problem and have a solution, it would be wonderful. Regards, Maxime Denier ******************************************************************************** IMPORTANT.Les informations contenues dans ce message electronique y compris les fichiers attaches sont strictement confidentielles et peuvent etre protegees par la loi. Ce message electronique est destine exclusivement au(x) destinataire(s) mentionne(s) ci-dessus. Si vous avez recu ce message par erreur ou s il ne vous est pas destine, veuillez immediatement le signaler a l expediteur et effacer ce message et tous les fichiers eventuellement attaches. Toute lecture, exploitation ou transmission des informations contenues dans ce message est interdite. Tout message electronique est susceptible d alteration. A ce titre, le Groupe France Telecom decline toute responsabilite notamment s il a ete altere, deforme ou falsifie. De meme, il appartient au destinataire de s assurer de l absence de tout virus. IMPORTANT.This e-mail message and any attachments are strictly confidential and may be protected by law. This message is intended only for the named recipient(s) above. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender and delete this e-mail message. Any unauthorized view, usage or disclosure ofthis message is prohibited. Since e-mail messages may not be reliable, France Telecom Group shall not be liable for any message if modified, changed or falsified. Additionally the recipient should ensure they are actually virus free. ********************************************************************************

3 2