- syslog-ng - lists.balabit.hu

Re: [syslog-ng] queue management
by Fekete Róbert 08 Mar '10

08 Mar '10

Hi, AFAIK, unfortunately there is currently no way to manipulate the queue. Regards, Robert On Monday, March 08, 2010 18:12 CET, "Eric Snow" <esnow(a)verio.net> wrote: > If disk based buffering allows me to yank the message at the front of the queue, so that it doesn't hold up the ones behind it, then yes that is what I want. The statistics and ability to look at the queue are good, but I need to be able to manipulate the queue. > > OR > > I need to be able to have some form of exception handling in my syslog-ng configuration. Something that will catch a failure in a log and allow me to handle that failure intelligently. > > -eric > > -----Original Message----- > From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler > Sent: Saturday, March 06, 2010 2:09 AM > To: Syslog-ng users' and developers' mailing list > Subject: Re: [syslog-ng] queue management > > On Fri, 2010-03-05 at 17:49 -0500, Eric Snow wrote: > > Hi, > > > > > > > > I am looking to manage my output buffers. I have one destination that > > sometimes has trouble and I want to be able to manage the messages > > that got buffered while I fix my flaky destination. Is there any tool > > I can use to manage those buffered messages? If not, isn’t there any > > exception handling that I can apply to a log such that errors or other > > failures for that log can be redirected to another destination? I > > want to avoid clogging up my output queue and especially to avoid > > losing any messages. Thanks! > > you mean you have disk based buffering? in the upcoming 3.1 version (due > to be released within 2 weeks), there's a tool called dqtool to dump the > contents of the queue file, although I'm not sure that's what you meant. > there are further disk buffer related improvements in the queue for 3.2, > and also a way to send the messages to a failed destination to an > alternate server. > > also, syslog-ng is counting the number of messages in a given queue, > using its statistics framework. it is the stored counter you can see for > a given destination. this gets reported in the 'log statistics' message, > or via the syslog-ng control socket (/opt/syslog-ng/var/syslog-ng.ctl). > > > -- > Bazsi > > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > > > > This email message is intended for the use of the person to whom it has been sent, and may contain information that is confidential or legally protected. If you are not the intended recipient or have received this message in error, you are not authorized to copy, distribute, or otherwise use this message or its attachments. Please notify the sender immediately by return e-mail and permanently delete this message and any attachments. Verio, Inc. makes no warranty that this email is error or virus free. Thank you. > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html >

1 0

syslog-ng plugins
by Balazs Scheidler 06 Mar '10

06 Mar '10

Hi, This is a repost of my blog article which I've found important enough to post here as well. The original article is at: http://bazsi.blogs.balabit.com/2010/03/plugins-branch-updated.html It is about a git branch in the syslog-ng OSE 3.1 git repository, as such it is available at: git://git.balabit.hu/bazsi/syslog-ng-3.1.git Plugins branch updated ---------------------- Since the last post, I could hack a couple of hours on the plugins branch, which now compiles. The plugin framework is capable for supporting a quite important core functionality: all socket like sources/destinations are now found in an external plugin called "afsocket". The reason I've started with afsocket is to make syslog-ng a bit less dependant on OpenSSL. A couple of distributions didn't include syslog-ng 3.0 in their current releases, because it uses OpenSSL from /usr, while syslog-ng should remain in the root directory. By separating afsocket from the syslog-ng core, I can compile afsocket with and without TLS support, which can be put into separate packages. Thus syslog-ng can operate without OpenSSL. And the same plugin framework will enable me to create a wide variety of plugins. My ideas: * Plugins for all syslog-ng components (source, destination, filter, rewrite, parser) * Python scriptability (a simple correllation engine in Python?) * macro transformation functions, for example: $(stripslashes $macro), usable anywhere in templates and stripslashes a plugin that is invoked whenever such an expansion occurs * Hooks for transforming the log message as it enters syslog-ng (to fix parsing errors for example), Do you have other ideas? Please post them as comments or as emails to the mailing list. Again, this functionality is experimental, and I'm still going to rebase the current code and will probably be integrated to syslog-ng 3.2. I got to release 3.1 final first though. :) -- Bazsi

1 0

problem with filtering in syslog-ng
by Evan Baer 06 Mar '10

06 Mar '10

Hello, I'm trying to exclude ntpd update messages from /var/log/messages. I have created a filter like so: filter not_ntpd { not program("ntpd*"); }; And implemented it in my log lines: log { source(src); filter(not_ntpd); destination(messages); }; log { source(src); filter(f_notice); filter(f_not_authpriv); filter(not_ntpd); destination(messages); }; log { source(src); filter(f_kern); filter(f_debug); filter(not_ntpd); destination(messages); }; log { source(src); filter(f_lpr); filter(f_info); destination(messages); }; log { source(src); filter(f_mail); filter(f_crit); destination(messages); }; Yet the lines for ntpd still seem to pass through to the logfile. Feb 23 09:20:48 magpie-shn1e1324 ntpd[37397]: synchronized to 10.2.253.22, stratum 1 Feb 23 09:20:49 magpie-shn1f1318 ntpd[729]: kernel time sync status change 2001 Any thoughts? -- Evan

6 11

what is the 'supervising syslog-ng' process?
by Tim Edwards 06 Mar '10

06 Mar '10

We have syslog-ng-client-3.0.5-1.rhel5 installed on several machines. However on 2 of those Nagios has flagged that there's an extra process with the command 'supervising syslog-ng'. Both machine1 and machine2 are Scientific Linux 5.4 (a RHEL clone) x86_64 Xen virtual machines with the exact same syslog-ng.conf: [root@machine1 ~]# ps -ef | grep syslog-ng root 24891 1 0 09:26 ? 00:00:00 supervising syslog-ng root 24892 24891 0 09:26 ? 00:00:00 /opt/syslog-ng/sbin/syslog-ng --no-caps [root@machine2 ~]# ps -ef | grep syslog root 1718 1695 0 09:38 pts/0 00:00:00 grep syslog root 2137 1 0 Feb01 ? 00:00:09 /opt/syslog-ng/sbin/syslog-ng --no-caps Can someone explain what this 'supervising' process is? Thanks Tim Edwards

3 3

composing file names with system varible
by peceka 06 Mar '10

06 Mar '10

Hi, i've on my machine file with some name, for example: # cat /var/hostname smeagol now i want to have log files like this: /var/log/smeagol-all.log if i change content of /var/hostname and then restart syslog-ng, syslog-ng shoud start logging to files with new name. is it possible? mabye no directly but with some workaround? regards, p.

3 4

RFC3339 timestamp parsing
by Ian Wienand 06 Mar '10

06 Mar '10

Hi, I notice that in the syslog-ng from my Debian install (2.0.9) log_msg_parse_date has p = strptime(self->date.str, "%Y-%m-%dT%H:%M:%S", &tm); and in 3.1beta2 I see p = (guchar *) strptime((gchar *) date, "%Y-%m-%d T%H:%M:%S", &tm); Is the extra space intended? I'm also wondering about a seeming ambiguity in RFC3339 [1] In 5.6 it says NOTE: ISO 8601 defines date and time separated by "T". Applications using this syntax may choose, for the sake of readability, to specify a full-date and full-time separated by (say) a space character. However in Appendix A it says ISO 8601 states that the "T" may be omitted under some circumstances. This grammar requires the "T" to avoid ambiguity. So is the T required or not? Should syslog-ng be able to parse either format? -i [1] http://www.ietf.org/rfc/rfc3339.txt

2 1

pcre matches
by Patrick H. 06 Mar '10

06 Mar '10

How do you use pcre named pattern matches with store-matches? I've been trying to do things like message("IP: (?<ip>\S+)" flags(store-matches) type("pcre")) and then insert into a database with things like values("$ip") or values("$+{ip}") and nothing works. I cant seem to find any examples of usage through google either.

2 2

Using amazon s3 as logging backend
by Rory Toma 24 Feb '10

24 Feb '10

Has anyone used amazon s3 as logging back end yet?

1 0

Re: [syslog-ng] syslog-ng gets started twice when system reboots
by Xuan Zhong 23 Feb '10

23 Feb '10

I looked both /etc/rc.d/rc3.d and /etc/rc.d/rc5.d, and other rcx.d directories. Only one "S" script for syslog-ng and it's symbolic link to /etc/init.d/syslog-ng. All files under rcx.d directory are symbolic links. Do you know if the first occurance of "Starting syslog-ng:" or "Stopping syslog-ng:" comes from syslog-ng binary file? Thanks Xuan > Date: Thu, 18 Feb 2010 11:09:55 -0600 > From: wes.hardin(a)maxim-ic.com > To: syslog-ng(a)lists.balabit.hu > Subject: Re: [syslog-ng] syslog-ng gets started twice when system reboots > > On 02/17/2010 09:59 AM, Xuan Zhong wrote: > > > > I use the stable version syslog-ng-3.0.5-1.rhel5.x86_64.rpm downloaded > > from your website. It starts/stops twice only when the system reboots. > > The init script is only generated once. The second "Starting syslog-ng: > > [ok]" was generated by init script because I modified the init script > > text display and only the second start changed. Not sure where the first > > start comes from. > > > > Starting syslog-ng: Starting syslog-ng: [ok] > > [ok] > > > > > > When the system shuts down, I saw the following: > > Stopping syslog-ng: Stopping syslog-ng: [ok] > > [ok] > > > > > > Similarly, only the second "Stopping syslog-ng:" comes from the > > syslog-ng service script. > > Look in /etc/rc.d/rc{3,5}.d for multiple "S" scripts for syslog-ng. Everything > in the rc?.d directories is typically a symlink to a similarly named script in > /etc/rc.d/init.d (usually a relative link like ../init.d/script). > > If there is a file that isn't a symlink or doesn't link to an initscript in > /etc/rc.d/init.d, that should be considered suspicious. > -- > /* Wes Hardin */ _________________________________________________________________ Hotmail: Trusted email with Microsoft’s powerful SPAM protection. http://clk.atdmt.com/GBL/go/201469226/direct/01/

2 2

Problem with windows client
by Heinze, René 22 Feb '10

22 Feb '10

Good morning, I´m using syslog-ng on a sles 10 server acting as a syslog-ng server and a sles 10 server acting as a syslog-ng client. The clients logs successfully everything to the server. I added a Windows XP Professional SP2 client and installed the syslog-ng client for windows. But unfortunately the windows syslog-ng client is not able to connect to the server. The only thing i configured is the ip of the syslog-ng server. I attached the debug log to this email. What am I doing wrong? The is no firewall causing the problem. On the syslog-ng server i see incoming packages from the windows host using tcpdump. Best regards, René Heinze Besuchen Sie die Bundesdruckerei auf der CeBIT 2010 vom 2.-6.3.2010, Halle 9, Stand D80 Visit Bundesdruckerei at CeBIT, exhibition centre, hall 9 / stand D80 weitere Informationen unter: http://www.bundesdruckerei.de/de/unternehmen/untern_cebit2010/index.html find more information here http://www.bundesdruckerei.de/en/company/comp_cebit2010/index.html

2 3