April 2010 - syslog-ng - lists.balabit.hu

syslig-ng Agent for Windows supported OS
by John MacDonald 27 Apr '10

27 Apr '10

I am trying to configure the Windows Syslog-Ng agent on Windows 2003 R2 and Windows 2008 Standard. The installer will run and a Service is installed, but I do not have any of the features or configuration options. Can you tell me if the Operating Systems listed above are supported? I am using version syslog-ng-agent-setup-3.1.0-amd64.msi Thanks. John

1 0

filter after source or indifferent?
by Giovanni Mancuso 27 Apr '10

27 Apr '10

Hi, i have a question about syslog-ng configuration. I configure my log path in this way: log { source(s_funambol); destination(d_network); filter(f_funambol); }; And it doesn't work. If i configure the log path in this way: log { source(s_funambol); filter(f_funambol); destination(d_network); }; It works. Is it normal? Is required that the filter goes after the source? Thanks

2 1

Re: [syslog-ng] Process stored logs
by noel anderson 26 Apr '10

26 Apr '10

Thanks patrick. This raises another question, how can I Quantify processed logs. Like what is been processed/ un-processed/ lost. Thanks, Noel (hsxtrt) Date: Thu, 22 Apr 2010 11:26:59 -0600 From: "Patrick H." <syslogng(a)feystorm.net> Subject: Re: [syslog-ng] Process stored logs To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Message-ID: <4BD086E3.3030200(a)feystorm.net> Content-Type: text/plain; charset="iso-8859-1" The log_fifo_size variable controls how many messages the output buffer will hold. So if server Z is relaying to A, and A goes down, Z will start storing messages in this buffer. Unfortunately there is no way to say 'if destination A fails, log to destination A2 (which may be a file output or something) instead'. The premium version does support disk-based buffering though, so that if log_fifo_size fills up, it'll start writing out to a disk based file instead. Sent: Thursday, April 22, 2010 12:13:40 AM From: noel anderson <nascentcatalyst(a)yahoo.com> To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Process stored logs > I'm building an infra across the geo's to collect logs at a central repository. I have set up syslog-ng in 3 geo's (say for e.g X, Y, Z) to collect logs form servers in respective Geo. A forurth server (say for eg. A) where the logs are forwarded from the 3 log servers to aggregate all the logs from all GEO's. > > The problem where I fail to understand is, if my aggregator server (A) goes down, how do i process my stored logs on (X,) (Y), (Z), so that i do not loose any logs during my downtime. > > Is it possible to process backlog of logs on the server or do i have to change my infra so that i do not loose these logs? > > Thanks > Noel (hsxtrt)

2 1

Re: [syslog-ng] syslog-ng 3.1.1 "cuts" of the first charater of the Apache error log when forwarding
by filip.nollet＠bayercropscience.com 26 Apr '10

26 Apr '10

Hi again First of all, I already used syslog-ng before, but I had no idea it was doing parsing, so this is normal :p I changed my configuration to use the no_parse function in the source driver of the webserver on which the syslog-ng client forwards the messages to the syslog server. This does the trick. I don't even have to change the configuration on the target syslog server which greatly reduces configuration file changes. Thx for the answer! Filip > Hi, > > > I have a weird problem with syslog-ng. I am setting up a syslog-ng > client on an Apache server which forward both access and error log > to a central syslog-ng server. > > First of all please note that you're trying to deal with > non-standard-compliant messages. syslog-ng tries to parse the incoming > log lines trying various known syslog formats. > > > The access log on the source has entries like this: > > 10.3.154.20 - - [23/Apr/2010:13:55:25 +0200] "GET /images/userLo... > > and this becomes like this on the syslog server: > > servername 10.3.154.20: - - [23/Apr/2010:13:55:25 +0200] "GET /images/use > > The first syslog-ng (which receives the apache log) parses the line. > There is no syslog header found so it falls back to the legacy format. > There is neither priority code nor timestamp so the IP address becomes > the program name. That's why you see it logger on the second server. > > > All ok with this one, but with the error log I get this on the webserver: > > [Fri Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File doe... > > but this on the syslog server: > > servername Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File do... > > > > Where is the [ character at the beginning of the timestamp gone????? > > As above, this line isn't a syslog message at all. From syslog's > perspective it's more or less random junk. The trailing '[' causes > that there is no program name at all and the parser thinks the > following string up to the closing ']' is the pid like in normal logs > eg. programname[pid] > > I don't see how could this "pid" appear on the receiving side with > only the closing brace. Please show your configs and the captured > network packet too. > > In general when an app doesn't speak syslog formats and there is no > option to format its logs properly then using the no-parse option (on > all syslog-ng servers the log passes through) or writing a small app > which reformats the logs and feeds to syslog-ng is the way to go. > > Regards, > > Sandor > ________________________________________________________________________ The information contained in this e-mail is for the exclusive use of the intended recipient(s) and may be confidential, proprietary, and/or legally privileged. Inadvertent disclosure of this message does not constitute a waiver of any privilege. If you receive this message in error, please do not directly or indirectly use, print, copy, forward, or disclose any part of this message. Please also delete this e-mail and all copies and notify the sender. Thank you. For alternate languages please go to http://bayerdisclaimer.bayerweb.com ________________________________________________________________________

1 0

Re: [syslog-ng] syslog-ng Digest, Vol 60, Issue 31
by filip.nollet＠bayercropscience.com 26 Apr '10

26 Apr '10

Hi again First of all, I already used syslog-ng before, but I had no idea it was doing parsing, so this is normal :p I changed my configuration to use the no_parse function in the source driver of the webserver on which the syslog-ng client forwards the messages to the syslog server. This does the trick. I don't even have to change the configuration on the target syslog server which greatly reduces configuration file changes. Thx for the answer! Filip > Hi, > > > I have a weird problem with syslog-ng. I am setting up a syslog-ng > client on an Apache server which forward both access and error log > to a central syslog-ng server. > > First of all please note that you're trying to deal with > non-standard-compliant messages. syslog-ng tries to parse the incoming > log lines trying various known syslog formats. > > > The access log on the source has entries like this: > > 10.3.154.20 - - [23/Apr/2010:13:55:25 +0200] "GET /images/userLo... > > and this becomes like this on the syslog server: > > servername 10.3.154.20: - - [23/Apr/2010:13:55:25 +0200] "GET /images/use > > The first syslog-ng (which receives the apache log) parses the line. > There is no syslog header found so it falls back to the legacy format. > There is neither priority code nor timestamp so the IP address becomes > the program name. That's why you see it logger on the second server. > > > All ok with this one, but with the error log I get this on the webserver: > > [Fri Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File doe... > > but this on the syslog server: > > servername Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File do... > > > > Where is the [ character at the beginning of the timestamp gone????? > > As above, this line isn't a syslog message at all. From syslog's > perspective it's more or less random junk. The trailing '[' causes > that there is no program name at all and the parser thinks the > following string up to the closing ']' is the pid like in normal logs > eg. programname[pid] > > I don't see how could this "pid" appear on the receiving side with > only the closing brace. Please show your configs and the captured > network packet too. > > In general when an app doesn't speak syslog formats and there is no > option to format its logs properly then using the no-parse option (on > all syslog-ng servers the log passes through) or writing a small app > which reformats the logs and feeds to syslog-ng is the way to go. > > Regards, > > Sandor > ________________________________________________________________________ The information contained in this e-mail is for the exclusive use of the intended recipient(s) and may be confidential, proprietary, and/or legally privileged. Inadvertent disclosure of this message does not constitute a waiver of any privilege. If you receive this message in error, please do not directly or indirectly use, print, copy, forward, or disclose any part of this message. Please also delete this e-mail and all copies and notify the sender. Thank you. For alternate languages please go to http://bayerdisclaimer.bayerweb.com ________________________________________________________________________

1 0

syslog-ng 3.1.1 "cuts" of the first charater of the Apache error log when forwarding
by filip.nollet＠bayercropscience.com 23 Apr '10

23 Apr '10

Hi all I have a weird problem with syslog-ng. I am setting up a syslog-ng client on an Apache server which forward both access and error log to a central syslog-ng server. The access log on the source has entries like this: 10.3.154.20 - - [23/Apr/2010:13:55:25 +0200] "GET /images/userLo... and this becomes like this on the syslog server: servername 10.3.154.20: - - [23/Apr/2010:13:55:25 +0200] "GET /images/use All ok with this one, but with the error log I get this on the webserver: [Fri Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File doe... but this on the syslog server: servername Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File do... Where is the [ character at the beginning of the timestamp gone????? Both logs use the same type configuration; I have double checked everything, but I have no clue why syslog-ng(?) does this... In wireshark the receiving syslog server is receiving it like this, so it is send like this by syslog-ng on the source server. Any idea? Met vriendelijke groeten / Cordialement / Freundliche Grüße / Best Regards Filip Nollet Bio Information System Administrator _______________________ Bayer BioScience N.V. Bio Information Management - A&S Technologiepark 38 9052 Zwijnaarde Phone: +32-9.243.04.28 Fax: +32-9.224.06.94 E-mail: filip.nollet(a)bayercropscience.com ________________________________________________________________________ The information contained in this e-mail is for the exclusive use of the intended recipient(s) and may be confidential, proprietary, and/or legally privileged. Inadvertent disclosure of this message does not constitute a waiver of any privilege. If you receive this message in error, please do not directly or indirectly use, print, copy, forward, or disclose any part of this message. Please also delete this e-mail and all copies and notify the sender. Thank you. For alternate languages please go to http://bayerdisclaimer.bayerweb.com ________________________________________________________________________

2 1

When forwarding apache error_log the first character ([) of each message is removed
by filip.nollet＠bayercropscience.com 23 Apr '10

23 Apr '10

Hi all I have a weird problem with syslog-ng. I am setting up a syslog-ng client on an Apache server which forward both access and error log to a central syslog-ng server. The access log on the source has entries like this: 10.3.154.20 - - [23/Apr/2010:13:55:25 +0200] "GET /images/userLo... and this becomes like this on the syslog server: servername 10.3.154.20: - - [23/Apr/2010:13:55:25 +0200] "GET /images/use All ok with this one, but with the error log I get this on the webserver: [Fri Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File doe... but this on the syslog server: servername Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File do... Where is the [ character at the beginning of the timestamp gone????? Both logs use the same type configuration; I have double checked everything, but I have no clue why syslog-ng(?) does this... In wireshark the receiving syslog server is receiving it like this, so it is send like this by syslog-ng on the source server. Any idea? Met vriendelijke groeten / Cordialement / Freundliche Grüße / Best Regards Filip Nollet ________________________________________________________________________ The information contained in this e-mail is for the exclusive use of the intended recipient(s) and may be confidential, proprietary, and/or legally privileged. Inadvertent disclosure of this message does not constitute a waiver of any privilege. If you receive this message in error, please do not directly or indirectly use, print, copy, forward, or disclose any part of this message. Please also delete this e-mail and all copies and notify the sender. Thank you. For alternate languages please go to http://bayerdisclaimer.bayerweb.com ________________________________________________________________________

1 0

Process stored logs
by noel anderson 22 Apr '10

22 Apr '10

I'm building an infra across the geo's to collect logs at a central repository. I have set up syslog-ng in 3 geo's (say for e.g X, Y, Z) to collect logs form servers in respective Geo. A forurth server (say for eg. A) where the logs are forwarded from the 3 log servers to aggregate all the logs from all GEO's. The problem where I fail to understand is, if my aggregator server (A) goes down, how do i process my stored logs on (X,) (Y), (Z), so that i do not loose any logs during my downtime. Is it possible to process backlog of logs on the server or do i have to change my infra so that i do not loose these logs? Thanks Noel (hsxtrt)

2 1

Re: [syslog-ng] Rediecting logs
by Zoltán Pallagi 21 Apr '10

21 Apr '10

abhay singh wrote: > Hi, > > Yes, it worked when i changed underscore(_) to hyphen(-). > But again if you check > "destination d_stunnel{ > file("/data/extract/syslogs/$HOST/$HOST_syslog.log" " > in this line am creating directory of $HOST first then log file name, > this should work right? Yes, excepting that you are using "_" instead of "-" again. If you insist on using underscore, use with the following form: /data/extract/syslogs/${HOST}/${HOST}_syslog.log > > > On Tue, Apr 20, 2010 at 4:08 PM, Zoltán Pallagi <pzolee(a)balabit.hu > <mailto:pzolee@balabit.hu>> wrote: > > Hi, > > Replace $HOST_syslog.log to $HOST-syslog.log and it will work. > Syslog-ng seems to be handling wrong if underscore character is > after macroname. > Report a bug about it, please (https://bugzilla.balabit.com/) > > > abhay singh wrote: >> I have made that changes and restarted syslog services even >> though its not working/// >> Now its like this >> >> options { >> long_hostnames(off); >> sync(0); >> perm(0640); >> stats(3600); >> keep_hostname(yes); >> }; >> source s_external { >> udp(port(514)); >> #udp(ip("192.168.114.129") port(514)); >> #udp(ip("0.0.0.0") port(514)); >> }; # Specific >> source s_stunnel { >> tcp(ip("127.0.0.1") >> port(514) >> max-connections(1)); >> }; >> destination d_external { >> file("/data/extract/syslogs/$HOST_syslog.log" \ >> owner(cubot) group(users) perm(0600) dir_perm(0700) >> create_dirs(yes) >> template("$DATE $FULLHOST $PROGRAM $TAG [$FACILITY.$LEVEL] >> $MESSAGE \n") ); >> }; >> >> destination d_stunnel{ >> file("/data/extract/syslogs/$HOST/$HOST_syslog.log" \ >> owner(cubot) group(users) perm(0600) dir_perm(0700) >> create_dirs(yes) >> template("$DATE $FULLHOST $PROGRAM $TAG [$FACILITY.$LEVEL] >> $MESSAGE \n") ); >> }; >> >> log { source(s_external); destination(d_external); }; >> log { source(s_stunnel); destination(d_stunnel); }; >> >> >> >> On Tue, Apr 20, 2010 at 1:55 PM, Alan McKinnon >> <Alan.McKinnon(a)is.co.za <mailto:Alan.McKinnon@is.co.za>> wrote: >> >> On Tuesday 20 April 2010 08:38:51 abhay singh wrote: >> > Hi All, >> > >> > I have made changes in syslog-ng.conf to redirect logs >> coming from other >> > server to some specified log file and in some format >> > >> > source s_external { >> > udp(port(514)); >> > }; >> > source s_stunnel { >> > tcp(ip(127.0.0.1) port(514)); >> > }; >> > >> > destination s_external { >> file("/mydir/syslogs/$HOST_syslog.log" \ >> > owner(user) group(group) perm(0600) dir_perm(0700) >> create_dirs(yes)); >> > }; >> > >> > destination s_stunnel { >> file("/mydir/syslogs/$HOST/syslog.log" \ >> > owner(user) group(group) perm(0600) dir_perm(0700) >> create_dirs(yes)); >> > }; >> > >> > log { source(s_external); destination(s_external); }; >> > log { source(s_stunnel); destination(s_stunnel); }; >> > >> > But my log file is being created with name "syslog.log" and >> not as >> > "<hostnmae/ip>_syslog.log" neither $HOST directory made, >> what may be the >> > issue? >> > >> > Globle options are >> > >> > option{ >> > >> > long_hostname(off); >> > sync(0); >> > } >> >> >> Your sources and destinations have the same name. That is not >> allowed. >> >> Rename the destinations to d_<something> >> >> -- >> Alan McKinnon >> Systems Engineer^W Technician >> Infrastructure Services >> Internet Solutions >> >> +27 11 575 7585 >> >> Please note: This email and its content are subject to the >> disclaimer as displayed at the following link >> http://www.is.co.za/legal/E-mail+Confidentiality+Notice+and+Disclaimer.htm. >> Should you not have Web access, send a mail to >> disclaimers(a)is.co.za <mailto:disclaimers@is.co.za> and a copy >> will be emailed to you. >> ______________________________________________________________________________ >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >> Documentation: >> http://www.balabit.com/support/documentation/?product=syslog-ng >> FAQ: http://www.campin.net/syslog-ng/faq.html >> >> >> >> >> -- >> Regards.. >> Abhay Singh.. >> 91-98804-55587 >> Sr. Software Engineer. >> Robust Designs India Pvt Ltd. >> Bangalore >> ------------------------------------------------------------------------ >> ______________________________________________________________________________ >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >> Documentation: >> http://www.balabit.com/support/documentation/?product=syslog-ng >> FAQ: http://www.campin.net/syslog-ng/faq.html > > > -- > pzolee > > > > > > -- > Regards.. > Abhay Singh.. > 91-98804-55587 > Sr. Software Engineer. > Robust Designs India Pvt Ltd. > Bangalore -- pzolee

1 0

bugzilla items not forwarded to this list
by Evan Rempel 20 Apr '10

20 Apr '10

I have opened a couple of bugs via the bugzilla.balabit.com system and the bugs have not come through to the mailing list. Evan.

4 8