- syslog-ng - lists.balabit.hu

Re: [syslog-ng] syslog-ng 3.1.1 "cuts" of the first charater of the Apache error log when forwarding
by filip.nollet＠bayercropscience.com 26 Apr '10

26 Apr '10

Hi again First of all, I already used syslog-ng before, but I had no idea it was doing parsing, so this is normal :p I changed my configuration to use the no_parse function in the source driver of the webserver on which the syslog-ng client forwards the messages to the syslog server. This does the trick. I don't even have to change the configuration on the target syslog server which greatly reduces configuration file changes. Thx for the answer! Filip > Hi, > > > I have a weird problem with syslog-ng. I am setting up a syslog-ng > client on an Apache server which forward both access and error log > to a central syslog-ng server. > > First of all please note that you're trying to deal with > non-standard-compliant messages. syslog-ng tries to parse the incoming > log lines trying various known syslog formats. > > > The access log on the source has entries like this: > > 10.3.154.20 - - [23/Apr/2010:13:55:25 +0200] "GET /images/userLo... > > and this becomes like this on the syslog server: > > servername 10.3.154.20: - - [23/Apr/2010:13:55:25 +0200] "GET /images/use > > The first syslog-ng (which receives the apache log) parses the line. > There is no syslog header found so it falls back to the legacy format. > There is neither priority code nor timestamp so the IP address becomes > the program name. That's why you see it logger on the second server. > > > All ok with this one, but with the error log I get this on the webserver: > > [Fri Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File doe... > > but this on the syslog server: > > servername Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File do... > > > > Where is the [ character at the beginning of the timestamp gone????? > > As above, this line isn't a syslog message at all. From syslog's > perspective it's more or less random junk. The trailing '[' causes > that there is no program name at all and the parser thinks the > following string up to the closing ']' is the pid like in normal logs > eg. programname[pid] > > I don't see how could this "pid" appear on the receiving side with > only the closing brace. Please show your configs and the captured > network packet too. > > In general when an app doesn't speak syslog formats and there is no > option to format its logs properly then using the no-parse option (on > all syslog-ng servers the log passes through) or writing a small app > which reformats the logs and feeds to syslog-ng is the way to go. > > Regards, > > Sandor > ________________________________________________________________________ The information contained in this e-mail is for the exclusive use of the intended recipient(s) and may be confidential, proprietary, and/or legally privileged. Inadvertent disclosure of this message does not constitute a waiver of any privilege. If you receive this message in error, please do not directly or indirectly use, print, copy, forward, or disclose any part of this message. Please also delete this e-mail and all copies and notify the sender. Thank you. For alternate languages please go to http://bayerdisclaimer.bayerweb.com ________________________________________________________________________

1 0

Re: [syslog-ng] syslog-ng Digest, Vol 60, Issue 31
by filip.nollet＠bayercropscience.com 26 Apr '10

26 Apr '10

Hi again First of all, I already used syslog-ng before, but I had no idea it was doing parsing, so this is normal :p I changed my configuration to use the no_parse function in the source driver of the webserver on which the syslog-ng client forwards the messages to the syslog server. This does the trick. I don't even have to change the configuration on the target syslog server which greatly reduces configuration file changes. Thx for the answer! Filip > Hi, > > > I have a weird problem with syslog-ng. I am setting up a syslog-ng > client on an Apache server which forward both access and error log > to a central syslog-ng server. > > First of all please note that you're trying to deal with > non-standard-compliant messages. syslog-ng tries to parse the incoming > log lines trying various known syslog formats. > > > The access log on the source has entries like this: > > 10.3.154.20 - - [23/Apr/2010:13:55:25 +0200] "GET /images/userLo... > > and this becomes like this on the syslog server: > > servername 10.3.154.20: - - [23/Apr/2010:13:55:25 +0200] "GET /images/use > > The first syslog-ng (which receives the apache log) parses the line. > There is no syslog header found so it falls back to the legacy format. > There is neither priority code nor timestamp so the IP address becomes > the program name. That's why you see it logger on the second server. > > > All ok with this one, but with the error log I get this on the webserver: > > [Fri Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File doe... > > but this on the syslog server: > > servername Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File do... > > > > Where is the [ character at the beginning of the timestamp gone????? > > As above, this line isn't a syslog message at all. From syslog's > perspective it's more or less random junk. The trailing '[' causes > that there is no program name at all and the parser thinks the > following string up to the closing ']' is the pid like in normal logs > eg. programname[pid] > > I don't see how could this "pid" appear on the receiving side with > only the closing brace. Please show your configs and the captured > network packet too. > > In general when an app doesn't speak syslog formats and there is no > option to format its logs properly then using the no-parse option (on > all syslog-ng servers the log passes through) or writing a small app > which reformats the logs and feeds to syslog-ng is the way to go. > > Regards, > > Sandor > ________________________________________________________________________ The information contained in this e-mail is for the exclusive use of the intended recipient(s) and may be confidential, proprietary, and/or legally privileged. Inadvertent disclosure of this message does not constitute a waiver of any privilege. If you receive this message in error, please do not directly or indirectly use, print, copy, forward, or disclose any part of this message. Please also delete this e-mail and all copies and notify the sender. Thank you. For alternate languages please go to http://bayerdisclaimer.bayerweb.com ________________________________________________________________________

1 0

syslog-ng 3.1.1 "cuts" of the first charater of the Apache error log when forwarding
by filip.nollet＠bayercropscience.com 23 Apr '10

23 Apr '10

Hi all I have a weird problem with syslog-ng. I am setting up a syslog-ng client on an Apache server which forward both access and error log to a central syslog-ng server. The access log on the source has entries like this: 10.3.154.20 - - [23/Apr/2010:13:55:25 +0200] "GET /images/userLo... and this becomes like this on the syslog server: servername 10.3.154.20: - - [23/Apr/2010:13:55:25 +0200] "GET /images/use All ok with this one, but with the error log I get this on the webserver: [Fri Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File doe... but this on the syslog server: servername Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File do... Where is the [ character at the beginning of the timestamp gone????? Both logs use the same type configuration; I have double checked everything, but I have no clue why syslog-ng(?) does this... In wireshark the receiving syslog server is receiving it like this, so it is send like this by syslog-ng on the source server. Any idea? Met vriendelijke groeten / Cordialement / Freundliche Grüße / Best Regards Filip Nollet Bio Information System Administrator _______________________ Bayer BioScience N.V. Bio Information Management - A&S Technologiepark 38 9052 Zwijnaarde Phone: +32-9.243.04.28 Fax: +32-9.224.06.94 E-mail: filip.nollet(a)bayercropscience.com ________________________________________________________________________ The information contained in this e-mail is for the exclusive use of the intended recipient(s) and may be confidential, proprietary, and/or legally privileged. Inadvertent disclosure of this message does not constitute a waiver of any privilege. If you receive this message in error, please do not directly or indirectly use, print, copy, forward, or disclose any part of this message. Please also delete this e-mail and all copies and notify the sender. Thank you. For alternate languages please go to http://bayerdisclaimer.bayerweb.com ________________________________________________________________________

2 1

When forwarding apache error_log the first character ([) of each message is removed
by filip.nollet＠bayercropscience.com 23 Apr '10

23 Apr '10

Hi all I have a weird problem with syslog-ng. I am setting up a syslog-ng client on an Apache server which forward both access and error log to a central syslog-ng server. The access log on the source has entries like this: 10.3.154.20 - - [23/Apr/2010:13:55:25 +0200] "GET /images/userLo... and this becomes like this on the syslog server: servername 10.3.154.20: - - [23/Apr/2010:13:55:25 +0200] "GET /images/use All ok with this one, but with the error log I get this on the webserver: [Fri Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File doe... but this on the syslog server: servername Apr 23 13:55:25 2010] [error] [client 10.3.154.20] File do... Where is the [ character at the beginning of the timestamp gone????? Both logs use the same type configuration; I have double checked everything, but I have no clue why syslog-ng(?) does this... In wireshark the receiving syslog server is receiving it like this, so it is send like this by syslog-ng on the source server. Any idea? Met vriendelijke groeten / Cordialement / Freundliche Grüße / Best Regards Filip Nollet ________________________________________________________________________ The information contained in this e-mail is for the exclusive use of the intended recipient(s) and may be confidential, proprietary, and/or legally privileged. Inadvertent disclosure of this message does not constitute a waiver of any privilege. If you receive this message in error, please do not directly or indirectly use, print, copy, forward, or disclose any part of this message. Please also delete this e-mail and all copies and notify the sender. Thank you. For alternate languages please go to http://bayerdisclaimer.bayerweb.com ________________________________________________________________________

1 0

Process stored logs
by noel anderson 22 Apr '10

22 Apr '10

I'm building an infra across the geo's to collect logs at a central repository. I have set up syslog-ng in 3 geo's (say for e.g X, Y, Z) to collect logs form servers in respective Geo. A forurth server (say for eg. A) where the logs are forwarded from the 3 log servers to aggregate all the logs from all GEO's. The problem where I fail to understand is, if my aggregator server (A) goes down, how do i process my stored logs on (X,) (Y), (Z), so that i do not loose any logs during my downtime. Is it possible to process backlog of logs on the server or do i have to change my infra so that i do not loose these logs? Thanks Noel (hsxtrt)

2 1

Re: [syslog-ng] Rediecting logs
by Zoltán Pallagi 21 Apr '10

21 Apr '10

abhay singh wrote: > Hi, > > Yes, it worked when i changed underscore(_) to hyphen(-). > But again if you check > "destination d_stunnel{ > file("/data/extract/syslogs/$HOST/$HOST_syslog.log" " > in this line am creating directory of $HOST first then log file name, > this should work right? Yes, excepting that you are using "_" instead of "-" again. If you insist on using underscore, use with the following form: /data/extract/syslogs/${HOST}/${HOST}_syslog.log > > > On Tue, Apr 20, 2010 at 4:08 PM, Zoltán Pallagi <pzolee(a)balabit.hu > <mailto:pzolee@balabit.hu>> wrote: > > Hi, > > Replace $HOST_syslog.log to $HOST-syslog.log and it will work. > Syslog-ng seems to be handling wrong if underscore character is > after macroname. > Report a bug about it, please (https://bugzilla.balabit.com/) > > > abhay singh wrote: >> I have made that changes and restarted syslog services even >> though its not working/// >> Now its like this >> >> options { >> long_hostnames(off); >> sync(0); >> perm(0640); >> stats(3600); >> keep_hostname(yes); >> }; >> source s_external { >> udp(port(514)); >> #udp(ip("192.168.114.129") port(514)); >> #udp(ip("0.0.0.0") port(514)); >> }; # Specific >> source s_stunnel { >> tcp(ip("127.0.0.1") >> port(514) >> max-connections(1)); >> }; >> destination d_external { >> file("/data/extract/syslogs/$HOST_syslog.log" \ >> owner(cubot) group(users) perm(0600) dir_perm(0700) >> create_dirs(yes) >> template("$DATE $FULLHOST $PROGRAM $TAG [$FACILITY.$LEVEL] >> $MESSAGE \n") ); >> }; >> >> destination d_stunnel{ >> file("/data/extract/syslogs/$HOST/$HOST_syslog.log" \ >> owner(cubot) group(users) perm(0600) dir_perm(0700) >> create_dirs(yes) >> template("$DATE $FULLHOST $PROGRAM $TAG [$FACILITY.$LEVEL] >> $MESSAGE \n") ); >> }; >> >> log { source(s_external); destination(d_external); }; >> log { source(s_stunnel); destination(d_stunnel); }; >> >> >> >> On Tue, Apr 20, 2010 at 1:55 PM, Alan McKinnon >> <Alan.McKinnon(a)is.co.za <mailto:Alan.McKinnon@is.co.za>> wrote: >> >> On Tuesday 20 April 2010 08:38:51 abhay singh wrote: >> > Hi All, >> > >> > I have made changes in syslog-ng.conf to redirect logs >> coming from other >> > server to some specified log file and in some format >> > >> > source s_external { >> > udp(port(514)); >> > }; >> > source s_stunnel { >> > tcp(ip(127.0.0.1) port(514)); >> > }; >> > >> > destination s_external { >> file("/mydir/syslogs/$HOST_syslog.log" \ >> > owner(user) group(group) perm(0600) dir_perm(0700) >> create_dirs(yes)); >> > }; >> > >> > destination s_stunnel { >> file("/mydir/syslogs/$HOST/syslog.log" \ >> > owner(user) group(group) perm(0600) dir_perm(0700) >> create_dirs(yes)); >> > }; >> > >> > log { source(s_external); destination(s_external); }; >> > log { source(s_stunnel); destination(s_stunnel); }; >> > >> > But my log file is being created with name "syslog.log" and >> not as >> > "<hostnmae/ip>_syslog.log" neither $HOST directory made, >> what may be the >> > issue? >> > >> > Globle options are >> > >> > option{ >> > >> > long_hostname(off); >> > sync(0); >> > } >> >> >> Your sources and destinations have the same name. That is not >> allowed. >> >> Rename the destinations to d_<something> >> >> -- >> Alan McKinnon >> Systems Engineer^W Technician >> Infrastructure Services >> Internet Solutions >> >> +27 11 575 7585 >> >> Please note: This email and its content are subject to the >> disclaimer as displayed at the following link >> http://www.is.co.za/legal/E-mail+Confidentiality+Notice+and+Disclaimer.htm. >> Should you not have Web access, send a mail to >> disclaimers(a)is.co.za <mailto:disclaimers@is.co.za> and a copy >> will be emailed to you. >> ______________________________________________________________________________ >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >> Documentation: >> http://www.balabit.com/support/documentation/?product=syslog-ng >> FAQ: http://www.campin.net/syslog-ng/faq.html >> >> >> >> >> -- >> Regards.. >> Abhay Singh.. >> 91-98804-55587 >> Sr. Software Engineer. >> Robust Designs India Pvt Ltd. >> Bangalore >> ------------------------------------------------------------------------ >> ______________________________________________________________________________ >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >> Documentation: >> http://www.balabit.com/support/documentation/?product=syslog-ng >> FAQ: http://www.campin.net/syslog-ng/faq.html > > > -- > pzolee > > > > > > -- > Regards.. > Abhay Singh.. > 91-98804-55587 > Sr. Software Engineer. > Robust Designs India Pvt Ltd. > Bangalore -- pzolee

1 0

bugzilla items not forwarded to this list
by Evan Rempel 20 Apr '10

20 Apr '10

I have opened a couple of bugs via the bugzilla.balabit.com system and the bugs have not come through to the mailing list. Evan.

4 8

[Bug 80] New: Missing RPM dependency
by bugzilla＠bugzilla.balabit.com 20 Apr '10

20 Apr '10

https://bugzilla.balabit.com/show_bug.cgi?id=80 Summary: Missing RPM dependency Product: syslog-ng Version: 3.0.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: packaging AssignedTo: folti(a)balabit.hu ReportedBy: alexandra(a)walford.id.au Type of the Report: bug Estimated Hours: 0.0 Hi, Regarding the RHEL5 RPMs which you provide for syslog-ng-client 3.0.6, would you consider adding coreutils as a dependency: it looks like the %post script tries to run the syslog-ng initscript, which in turn calls uname. Background: we are attempting to install syslog-ng-client during a kickstart'd RHEL 5.5 build. Since syslog-ng-client doesn't explicitly depend upon coreutils, anaconda is attempting to install syslog-ng-client first: ... Installing syslog-ng-client-3.0.6-1.rhel5.x86_64 warning: syslog-ng-client-3.0.6-1.rhel5: Header V4 DSA signature: NOKEY, key ID 2aa28252 /var/tmp/rpm-tmp.82027: line 90: uname: command not found /var/tmp/rpm-tmp.82027: line 90: [: =: unary operator expected /var/tmp/rpm-tmp.82027: line 97: uname: command not found ... Installing coreutils-5.97-23.el5_4.1.x86_64 ... Thanks, Alexandra -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 0

Re: [syslog-ng] pattern matching against raw input?
by Robert Fekete 19 Apr '10

19 Apr '10

Hi Glen, Try setting the flags(no-parse) option on your source (http://www.balabit.com/dl/html/syslog-ng-ose-v3.1-guide-admin-en.html/ch06s…) That way syslog-ng will put all incoming data into the MESSAGE part without parsing, and add a syslog header (timestamp, etc.). I think that was what you meant. Regards, Robert On Thursday, April 15, 2010 21:09 CEST, Glen Johnson <gfjohnson(a)alaska.edu> wrote: > Is there a way to parse against raw input, with syslog-ng? > > I'm writing patterns for Cisco devices and the PROGRAM macro appears > to be gobbling (and then removing) the %FACILITY-SEVERITY-MNEMONIC > text, prior to the parser getting access [to the message], for Cisco > ASA devices. In other cases, PROGRAM gobbles the log sequence number > (from cisco ios) and leaves the date/time and FAC-SEV-MNEMONIC code > intact. > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.campin.net/syslog-ng/faq.html > >

1 0

pattern matching against raw input?
by Glen Johnson 17 Apr '10

17 Apr '10

Is there a way to parse against raw input, with syslog-ng? I'm writing patterns for Cisco devices and the PROGRAM macro appears to be gobbling (and then removing) the %FACILITY-SEVERITY-MNEMONIC text, prior to the parser getting access [to the message], for Cisco ASA devices. In other cases, PROGRAM gobbles the log sequence number (from cisco ios) and leaves the date/time and FAC-SEV-MNEMONIC code intact.

2 1