[syslog-ng] syslog-ng Digest, Vol 28, Issue 21
Wilson Lai
wilsonlai at macausjm.com
Fri Sep 7 11:26:02 CEST 2007
Dear all,
What happen if the log message is not a standard syslog message?
Thanks.
Regards,
Wilson Lai
System Engineer
IT Dept., SJM
Office ( : (853)2978585
Mobile ( : (853)66506709
Email +: : wilsonlai at macausjm.com
-----Original Message-----
From: syslog-ng-request at lists.balabit.hu
[mailto:syslog-ng-request at lists.balabit.hu]
Sent: Friday, August 17, 2007 3:45 PM
To: syslog-ng at lists.balabit.hu
Subject: syslog-ng Digest, Vol 28, Issue 21
Send syslog-ng mailing list submissions to
syslog-ng at lists.balabit.hu
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.balabit.hu/mailman/listinfo/syslog-ng
or, via email, send a message with subject or body 'help' to
syslog-ng-request at lists.balabit.hu
You can reach the person managing the list at
syslog-ng-owner at lists.balabit.hu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of syslog-ng digest..."
Today's Topics:
1. new syslog-ng white paper published (Balazs Scheidler)
2. Buffer Overflow : Insufficient buffer space for retrieving
STREAMS log message; res='2' ( Nicolas de Marqu? - Fromentin )
3. REGEX rewrites on packet body possible? (Eli Stair)
4. Re: Buffer Overflow : Insufficient buffer space for
retrieving STREAMS log message; res='2' (Russell Fulton)
5. Re: Logging Third party application logs to Syslog-NG
(Wilson Lai)
6. Re: Buffer Overflow : Insufficient buffer space for
retrieving STREAMS log message; res='2' (Balazs Scheidler)
7. Re: REGEX rewrites on packet body possible? (Balazs Scheidler)
8. Re: Logging Third party application logs to Syslog-NG
(Balazs Scheidler)
----------------------------------------------------------------------
Message: 1
Date: Thu, 16 Aug 2007 16:58:31 +0200
From: Balazs Scheidler <bazsi at balabit.hu>
Subject: [syslog-ng] new syslog-ng white paper published
To: syslog-ng at lists.balabit.hu
Message-ID: <1187276311.7565.52.camel at bzorp.balabit>
Content-Type: text/plain
Dear all,
We have published a new, syslog-ng related white paper on our website
titled: "Distributed syslog architectures with syslog-ng"
It basically contains the various deployment scenarios, their pros and
cons we've encountered in our experience.
It does contain information on syslog-ng Premium Edition, but still we
tried to concentrate on the technical content, and some of it equally
applies to syslog-ng OSE.
We'd appreciate feedback at documentation at balabit.com. Thanks.
--
Bazsi
------------------------------
Message: 2
Date: Thu, 16 Aug 2007 18:40:03 +0200
From: " Nicolas de Marqu? - Fromentin " <nicolas.demarque at gmail.com>
Subject: [syslog-ng] Buffer Overflow : Insufficient buffer space for
retrieving STREAMS log message; res='2'
To: "Syslog-ng users' and developers' mailing list"
<syslog-ng at lists.balabit.hu>
Message-ID:
<b778b3cc0708160940y50d0dfb4yc84f86dffa4efc6a at mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Hello,
During a big flow from a local server to a central server, we have a
"buffer
overflow" on the local server. After this message, syslog-ng is crash
and
don't be capable to send any messages.
The local server is a solaris 10 product, the central linux a redhat
EL4.
Do you know a rule to stop this problem
Best regard,
Nicolas
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.balabit.hu/pipermail/syslog-ng/attachments/20070816/b5a179d7/attachment-0001.htm
------------------------------
Message: 3
Date: Thu, 16 Aug 2007 10:07:03 -0700
From: Eli Stair <estair at ilm.com>
Subject: [syslog-ng] REGEX rewrites on packet body possible?
To: syslog-ng at lists.balabit.hu
Message-ID: <46C48437.90103 at ilm.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
I've got a problem with some network devices that is leading me to need
to find
some way to do regex rewriting of portions of the message body of syslog
messages.
Problem: I've got a bunch of Foundry devices that put their hostname
followed
by a comma in the body of the message (and some that do not). Some of
these
look like this:
{
2007-08-16:2007-08-16T09:50:16-07:00 hostname [hostname.local7.notice]
hostname, Linecard Module 13 temperature 50.0 C degrees is normal
}
# template("$R_ISODATE $HOST [$PROGRAM.$FACILITY.$PRIORITY] $MSG\n")
As you can see from the template, the second hostname reported with the
comma
is part of the MSG body. For reasons of properly searching/indexing
this data
I need to strip this out. I've seen mention of a tool called
'syslog-mailer'
that sounds like it would do the job somewhat. Additionally, I've seen
blog
chatter about potentially adding full regex rewrite capability to
syslog-ng in
the recent past. I can't find evidence of either of these methods
however.
My first try at solving this using an external program showed that when
passing
data OUT of syslog-ng to a defined program, only the message body is
sent and
before application of a template, the other information is dropped.
Thus it's
not possible to do processing of the whole payload externally, re-import
the
data via a socket and finish writing because the facility and HOST
information
is all gone!
I'm looking at writing a log proxy using Net::Dev::Tools::Syslog in perl
to
handle listening, rewriting if necessary, and forwarding full messages
on to
syslog-ng after. I'd just like to know if there are any better
suggestions, or
if this has been done before successfully in another way?
Cheers, and thanks for any insight.
/eli
------------------------------
Message: 4
Date: Fri, 17 Aug 2007 09:33:48 +1200
From: Russell Fulton <r.fulton at auckland.ac.nz>
Subject: Re: [syslog-ng] Buffer Overflow : Insufficient buffer space
for retrieving STREAMS log message; res='2'
To: Syslog-ng users' and developers' mailing list
<syslog-ng at lists.balabit.hu>
Message-ID: <46C4C2BC.9040808 at auckland.ac.nz>
Content-Type: text/plain; charset=ISO-8859-1
Hi Nicolas
I suggest you post the actual error messages to the list to give a few
more clues.
Russell
Nicolas de Marqu? - Fromentin wrote:
> Hello,
>
> During a big flow from a local server to a central server, we have a
> "buffer overflow" on the local server. After this message, syslog-ng
> is crash and don't be capable to send any messages.
>
> The local server is a solaris 10 product, the central linux a redhat
EL4.
>
> Do you know a rule to stop this problem
>
> Best regard,
>
> Nicolas
>
------------------------------------------------------------------------
>
> _______________________________________________
> syslog-ng maillist - syslog-ng at lists.balabit.hu
> https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
>
>
------------------------------
Message: 5
Date: Fri, 17 Aug 2007 11:37:36 +0800
From: "Wilson Lai" <wilsonlai at macausjm.com>
Subject: Re: [syslog-ng] Logging Third party application logs to
Syslog-NG
To: syslog-ng <syslog-ng at lists.balabit.hu>
Message-ID: <H000006e00732964.1187321856.mail.macausjm.com at MHS>
Content-Type: text/plain; charset="US-ASCII"
Dear Bill,
If there is a Scalix application which generate the log file in
"/var/opt/Scalix/ml/s/logs/fatal", where "fatal" is the log file. How
could
the source be defined to be listened by Syslog-NG?
Thanks.
Regards,
Wilson Lai
System Engineer
IT Dept., SJM
Office ( : (853)2978585
Mobile ( : (853)66506709
Email +: : wilsonlai at macausjm.com
-----Original Message-----
From: syslog-ng-request at lists.balabit.hu
[mailto:syslog-ng-request at lists.balabit.hu]
Sent: Thursday, August 16, 2007 6:00 PM
To: syslog-ng at lists.balabit.hu
Subject: syslog-ng Digest, Vol 28, Issue 20
Send syslog-ng mailing list submissions to
syslog-ng at lists.balabit.hu
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.balabit.hu/mailman/listinfo/syslog-ng
or, via email, send a message with subject or body 'help' to
syslog-ng-request at lists.balabit.hu
You can reach the person managing the list at
syslog-ng-owner at lists.balabit.hu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of syslog-ng digest..."
Today's Topics:
1. GLib version problem? (Burns Andrew)
2. Re: GLib version problem? (Geller, Sandor (IT))
3. Re: GLib version problem? (Valdis.Kletnieks at vt.edu)
4. Logging Third party application logs to Syslog-NG in Linux!
(Wilson Lai)
5. Re: Logging Third party application logs to Syslog-NG in
Linux! (Bill Nash)
6. Troubles with SE-Linux Syslog-ng and ntpd (Martin Voelker)
----------------------------------------------------------------------
Message: 1
Date: Wed, 15 Aug 2007 08:02:38 -0500
From: "Burns Andrew" <aburns at snyderdrug.com>
Subject: [syslog-ng] GLib version problem?
To: <syslog-ng at lists.balabit.hu>
Message-ID:
<699402D08697B94EAB9429B5083FAA1C042C2290 at exchange_2k.snyderdrug.co
m>
Content-Type: text/plain; charset="us-ascii"
Hello,
I'm trying to install syslog-NG onto a Cent OS 5 server, and running
into a problem. The problem appears to be more of a RedHat/Cent problem
(or even Glib problem) rather than Syslog-NG problem, but I thought I'd
check with the community before I harass Red Hat. I have the RPM verion
of glib2.12 installed, however the libraries seemed to be labled 2.0.0.
When I configure syslog-ng, I run across the following error:
checking pkg-config is at least version 0.9.0... yes
checking for GLIB... no
configure: error: Cannot find GLib library version >= 2.2: is
pkg-config in path?
[root at syslog syslog-ng-2.0.5]#
I have the rpm installed:
[root at syslog syslog-ng-2.0.5]# yum list installed glib2
Loading "installonlyn" plugin
Loading "fastestmirror" plugin
Installed Packages
glib2.i386 2.12.3-2.fc6
installed
[root at syslog syslog-ng-2.0.5]#
However the libraries seem to be mislabeled:
[root at syslog syslog-ng-2.0.5]# locate glib | grep "/lib/libg"
/lib/libglib-2.0.so.0
/lib/libglib-2.0.so.0.1200.3
[root at syslog syslog-ng-2.0.5]#
Am I just missing something simple, or is this a problem I should be
trying to speak to Cent/Red Hat about?
Thanks!
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.balabit.hu/pipermail/syslog-ng/attachments/20070815/d033c920/attachment.html
------------------------------
Message: 2
Date: Wed, 15 Aug 2007 14:13:49 +0100
From: "Geller, Sandor \(IT\)" <Sandor.Geller at morganstanley.com>
Subject: Re: [syslog-ng] GLib version problem?
To: "Syslog-ng users' and developers' mailing list"
<syslog-ng at lists.balabit.hu>
Message-ID:
<14F0A35F6E466D48BF11108F4E09E68C07CFDC9F at LNWEXMB58.msad.ms.com>
Content-Type: text/plain; charset="us-ascii"
Hi,
> Hello,
>
> I'm trying to install syslog-NG onto a Cent OS 5 server, and
> running into a problem. The problem appears to be more of a
> RedHat/Cent problem (or even Glib problem) rather than
> Syslog-NG problem, but I thought I'd check with the community
> before I harass Red Hat. I have the RPM verion of glib2.12
> installed, however the libraries seemed to be labled 2.0.0.
No, it is actually 2.12.3, the binary compatibility version is
2.0
> When I configure syslog-ng, I run across the following error:
> checking pkg-config is at least version 0.9.0... yes
> checking for GLIB... no
> configure: error: Cannot find GLib library version >=
> 2.2: is pkg-config in path?
> [root at syslog syslog-ng-2.0.5]#
>
> I have the rpm installed:
> [root at syslog syslog-ng-2.0.5]# yum list installed glib2
> Loading "installonlyn" plugin
> Loading "fastestmirror" plugin
> Installed Packages
> glib2.i386 2.12.3-2.fc6
> installed
> [root at syslog syslog-ng-2.0.5]#
>
> However the libraries seem to be mislabeled:
> [root at syslog syslog-ng-2.0.5]# locate glib | grep "/lib/libg"
> /lib/libglib-2.0.so.0
> /lib/libglib-2.0.so.0.1200.3
> [root at syslog syslog-ng-2.0.5]#
>
> Am I just missing something simple, or is this a problem I
> should be trying to speak to Cent/Red Hat about?
You need the development package too. yum install glib2-devel
Regards,
Sandor
--------------------------------------------------------
NOTICE: If received in error, please destroy and notify sender. Sender
does not intend to waive confidentiality or privilege. Use of this email
is prohibited when received in error.
------------------------------
Message: 3
Date: Wed, 15 Aug 2007 11:43:45 -0400
From: Valdis.Kletnieks at vt.edu
Subject: Re: [syslog-ng] GLib version problem?
To: Burns Andrew <aburns at snyderdrug.com>
Cc: Syslog-ng users' and developers' mailing list
<syslog-ng at lists.balabit.hu>
Message-ID: <5364.1187192625 at turing-police.cc.vt.edu>
Content-Type: text/plain; charset="us-ascii"
On Wed, 15 Aug 2007 08:02:38 CDT, Burns Andrew said:
> check with the community before I harass Red Hat. I have the RPM
verion
> of glib2.12 installed, however the libraries seemed to be labled
2.0.0.
Do you have glib2-devel RPM installed? Not having the -devel will cause
these sort of symptoms.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 226 bytes
Desc: not available
Url :
http://lists.balabit.hu/pipermail/syslog-ng/attachments/20070815/223d607b/attachment-0001.pgp
------------------------------
Message: 4
Date: Thu, 16 Aug 2007 12:19:35 +0800
From: "Wilson Lai" <wilsonlai at macausjm.com>
Subject: [syslog-ng] Logging Third party application logs to Syslog-NG
in Linux!
To: syslog-ng <syslog-ng at lists.balabit.hu>
Message-ID: <H000006e00726bb4.1187237975.mail.macausjm.com at MHS>
Content-Type: text/plain; charset="us-ascii"
Dear ALL,
I am now using the Syslog-NG OSE for centralized logging system.
How could I get the third party application (Linux Client) logs
logging to the Syslog-NG server? These third party application logs
are not managed by the Linux syslog daemon.
Would there be someone gives some information or suggestion!
Thanks a lot!!!
Regards,
Wilson Lai
System Engineer
IT Dept., SJM
Office ( : (853)2978585
Mobile ( : (853)66506709
Email +: : wilsonlai at macausjm.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.balabit.hu/pipermail/syslog-ng/attachments/20070816/42188c0b/attachment-0001.htm
------------------------------
Message: 5
Date: Wed, 15 Aug 2007 21:16:49 -0700 (MST)
From: Bill Nash <billn at billn.net>
Subject: Re: [syslog-ng] Logging Third party application logs to
Syslog-NG in Linux!
To: "Syslog-ng users' and developers' mailing list"
<syslog-ng at lists.balabit.hu>
Message-ID: <Pine.LNX.4.64.0708152115400.23309 at pegasus.billn.net>
Content-Type: TEXT/PLAIN; charset=US-ASCII
If the application itself is capable of generating a log file, you can
specify that log file as a 'source' within the syslog-ng config. Be sure
to add that source, once declared, to one of your log directives. It's
really that easy.
- billn
On Thu, 16 Aug 2007, Wilson Lai wrote:
> Dear ALL,
>
> I am now using the Syslog-NG OSE for centralized logging
system.
> How could I get the third party application (Linux Client) logs
>
> logging to the Syslog-NG server? These third party application
logs
> are not managed by the Linux syslog daemon.
>
> Would there be someone gives some information or suggestion!
>
> Thanks a lot!!!
>
>
>
> Regards,
>
> Wilson Lai
>
> System Engineer
>
> IT Dept., SJM
>
> Office ( : (853)2978585
>
> Mobile ( : (853)66506709
>
> Email +: : wilsonlai at macausjm.com
>
>
>
>
>
------------------------------
Message: 6
Date: Thu, 16 Aug 2007 10:16:54 +0200
From: "Martin Voelker" <martin.voelker at westlotto.com>
Subject: [syslog-ng] Troubles with SE-Linux Syslog-ng and ntpd
To: <syslog-ng at lists.balabit.hu>
Message-ID: <46C424160200008C0000855E at Mclp3_server.wl>
Content-Type: text/plain; charset="ISO-8859-1"
Hi,
I have some troubles using syslog-ng on RHEL4 with SE Linux. There comes
following message:
audit(1187252020.581:514): avc: denied { write } for pid=2646
comm="ntpd" name="log" dev=tmpfs ino=14840162
scontext=root:system_r:ntpd_t tcontext=root:object_r:device_t
tclass=sock_file
What can I do???
Thanks
Martin
Westdeutsche Lotterie GmbH & Co. OHG | Sitz: M?nster
Registergericht: Amtsgericht M?nster
Handelsregister: M?nster HRA 4379
Gesch?ftsf?hrer: Dr. Winfried Wortmann
Vorsitzender des Beirates: Ernst Gerlach
Gesellschafter:
Nordwestlotto in Nordrhein-Westfalen GmbH | Sitz: M?nster
Registergericht: Amtsgericht M?nster
Handelsregister: HRB 3840
Gesch?ftsf?hrer: Dr. Winfried Wortmann
NRW.BANK | Sitz: D?sseldorf und M?nster
Rechtsform: Anstalt des ?ffentlichen Rechts
Registergerichte: Amtsgerichte D?sseldorf/M?nster
Handelsregister: D?sseldorf HRA 15277/M?nster HRA 5300
------------------------------
_______________________________________________
syslog-ng maillist - syslog-ng at lists.balabit.hu
https://lists.balabit.hu/mailman/listinfo/syslog-ng
End of syslog-ng Digest, Vol 28, Issue 20
*****************************************
------------------------------
Message: 6
Date: Fri, 17 Aug 2007 09:38:36 +0200
From: Balazs Scheidler <bazsi at balabit.hu>
Subject: Re: [syslog-ng] Buffer Overflow : Insufficient buffer space
for retrieving STREAMS log message; res='2'
To: Syslog-ng users' and developers' mailing list
<syslog-ng at lists.balabit.hu>
Message-ID: <1187336316.6771.14.camel at bzorp.balabit>
Content-Type: text/plain
On Fri, 2007-08-17 at 09:33 +1200, Russell Fulton wrote:
> Hi Nicolas
>
> I suggest you post the actual error messages to the list to give a few
> more clues.
and syslog-ng version. 1.6.4 had a related fix.
--
Bazsi
------------------------------
Message: 7
Date: Fri, 17 Aug 2007 09:42:29 +0200
From: Balazs Scheidler <bazsi at balabit.hu>
Subject: Re: [syslog-ng] REGEX rewrites on packet body possible?
To: Syslog-ng users' and developers' mailing list
<syslog-ng at lists.balabit.hu>
Message-ID: <1187336549.6771.17.camel at bzorp.balabit>
Content-Type: text/plain
On Thu, 2007-08-16 at 10:07 -0700, Eli Stair wrote:
> I've got a problem with some network devices that is leading me to
need to find
> some way to do regex rewriting of portions of the message body of
syslog
> messages.
>
> Problem: I've got a bunch of Foundry devices that put their hostname
followed
> by a comma in the body of the message (and some that do not). Some of
these
> look like this:
>
> {
> 2007-08-16:2007-08-16T09:50:16-07:00 hostname [hostname.local7.notice]
> hostname, Linecard Module 13 temperature 50.0 C degrees is normal
> }
>
> # template("$R_ISODATE $HOST [$PROGRAM.$FACILITY.$PRIORITY] $MSG\n")
>
>
> As you can see from the template, the second hostname reported with
the comma
> is part of the MSG body. For reasons of properly searching/indexing
this data
> I need to strip this out. I've seen mention of a tool called
'syslog-mailer'
> that sounds like it would do the job somewhat. Additionally, I've
seen blog
> chatter about potentially adding full regex rewrite capability to
syslog-ng in
> the recent past. I can't find evidence of either of these methods
however.
>
> My first try at solving this using an external program showed that
when passing
> data OUT of syslog-ng to a defined program, only the message body is
sent and
> before application of a template, the other information is dropped.
Thus it's
> not possible to do processing of the whole payload externally,
re-import the
> data via a socket and finish writing because the facility and HOST
information
> is all gone!
>
> I'm looking at writing a log proxy using Net::Dev::Tools::Syslog in
perl to
> handle listening, rewriting if necessary, and forwarding full messages
on to
> syslog-ng after. I'd just like to know if there are any better
suggestions, or
> if this has been done before successfully in another way?
>
> Cheers, and thanks for any insight.
You could do something like this:
f_strip_hostname { match("^[\-a-zA-Z0-0]+,(.*)$") or match("^(.*)$"); };
destination d_out { file("/var/log/messages" template("$R_ISODATE $HOST
[$PROGRAM.$FACILITY.$PRIORITY] $1\n"); };
The filter will make $1 equal to the message part without a hostname,
either because
there was a hostname (first match), or because there wasn't.
Improving the hostname pattern would reduce ambiguity, as the pattern
will strip
everything till the first comma.
--
Bazsi
------------------------------
Message: 8
Date: Fri, 17 Aug 2007 09:45:09 +0200
From: Balazs Scheidler <bazsi at balabit.hu>
Subject: Re: [syslog-ng] Logging Third party application logs to
Syslog-NG
To: Syslog-ng users' and developers' mailing list
<syslog-ng at lists.balabit.hu>
Message-ID: <1187336709.6771.21.camel at bzorp.balabit>
Content-Type: text/plain
On Fri, 2007-08-17 at 11:37 +0800, Wilson Lai wrote:
> Dear Bill,
> If there is a Scalix application which generate the log file
in
> "/var/opt/Scalix/ml/s/logs/fatal", where "fatal" is the log file. How
> could
> the source be defined to be listened by Syslog-NG?
source s_file { file("/var/opt/Scalix/ml/s/logs/fatal" follow_freq(1));
};
This will read the log file, checking every second if there are new
entries. The current position is remembered accross restarts.
You need a recent syslog-ng version though (2.0.5 is fine)
Read
http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/ch08s01.html#id2550287
for more information.
--
Bazsi
------------------------------
_______________________________________________
syslog-ng maillist - syslog-ng at lists.balabit.hu
https://lists.balabit.hu/mailman/listinfo/syslog-ng
End of syslog-ng Digest, Vol 28, Issue 21
*****************************************
More information about the syslog-ng
mailing list