[syslog-ng] Re: syslog-ng Digest, Vol 24, Issue 33
Fabian Pucciarelli
fabiangp at gmail.com
Mon Apr 30 18:50:22 CEST 2007
I appreciate the help Bazsi, I receive around 600 of this messages daily, do
you know when this messages are generated? I there any way to change that
time to lets say one message/hour? TIA!
On Sun, 2007-04-29 at 10:25 -0600, Fabian Pucciarelli wrote:
> I have syslog ng writing to a mysql pipe and then a little script
> reading the pipe and writing to the db. I currently receive many
> messages like the following, I wonder if somenone can help me figure
> out what this message is saying..... thanks.
>
>
> | mailux | syslog | notice | notice | 2d | 2007-04-13 | 01:16:40
> | syslog-ng | syslog-ng[30548]: Log statistics;
> dropped='pipe(/tmp/mysql.pipe)=0', processed='center(queued)=40295',
> processed='center(received)=13206',
> processed='destination(d_mysql)=40295', processed='source(net_tcp)=0',
> processed='source(src)=8962', processed='source(net_udp)=4244' |
> 46674 |
This contains the various message counters that syslog-ng maintains
internally.
'processed' counters just count how much messages were processed at the
given point, 'dropped' counters indicate how much log messages were
dropped by syslog-ng itself.
The string in the parenthesis tell were the given counter is counting
inside syslog-ng.
'center' is the main log message dispatching mechanism, it received
13206 messages from various sources and then sent 40295 messages out,
e.g. you are probably sending individual messages to multiple
destinations.
You have two sources, one named 'src' generated 8962, the other called
'net_udp' 4244 messages.
--
Bazsi
On 4/30/07, syslog-ng-request at lists.balabit.hu <
syslog-ng-request at lists.balabit.hu> wrote:
>
> Send syslog-ng mailing list submissions to
> syslog-ng at lists.balabit.hu
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.balabit.hu/mailman/listinfo/syslog-ng
> or, via email, send a message with subject or body 'help' to
> syslog-ng-request at lists.balabit.hu
>
> You can reach the person managing the list at
> syslog-ng-owner at lists.balabit.hu
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of syslog-ng digest..."
>
>
> Today's Topics:
>
> 1. Re: syslog-ng Digest, Vol 24, Issue 32 (Fabian Pucciarelli)
> 2. Re: turn off case sensitivity for match regex filter
> (Balazs Scheidler)
> 3. Re: Re: syslog-ng Digest, Vol 24, Issue 32 (Balazs Scheidler)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 29 Apr 2007 10:25:18 -0600
> From: "Fabian Pucciarelli" <fabiangp at gmail.com>
> Subject: [syslog-ng] Re: syslog-ng Digest, Vol 24, Issue 32
> To: syslog-ng at lists.balabit.hu
> Message-ID:
> <1e6757090704290925n541deb73t13eace5731aced3b at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> I have syslog ng writing to a mysql pipe and then a little script reading
> the pipe and writing to the db. I currently receive many messages like the
> following, I wonder if somenone can help me figure out what this message
> is
> saying..... thanks.
>
>
> | mailux | syslog | notice | notice | 2d | 2007-04-13 | 01:16:40 |
> syslog-ng | syslog-ng[30548]: Log statistics;
> dropped='pipe(/tmp/mysql.pipe)=0', processed='center(queued)=40295',
> processed='center(received)=13206',
> processed='destination(d_mysql)=40295',
> processed='source(net_tcp)=0', processed='source(src)=8962',
> processed='source(net_udp)=4244' | 46674 |
>
> On 4/29/07, syslog-ng-request at lists.balabit.hu <
> syslog-ng-request at lists.balabit.hu> wrote:
> >
> > Send syslog-ng mailing list submissions to
> > syslog-ng at lists.balabit.hu
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> > https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > or, via email, send a message with subject or body 'help' to
> > syslog-ng-request at lists.balabit.hu
> >
> > You can reach the person managing the list at
> > syslog-ng-owner at lists.balabit.hu
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of syslog-ng digest..."
> >
> >
> > Today's Topics:
> >
> > 1. Re: turn off case sensitivity for match regex filter
> > (Balazs Scheidler)
> > 2. Re: turn off case sensitivity for match regex filter (stucky)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Sat, 28 Apr 2007 12:42:06 +0200
> > From: Balazs Scheidler <bazsi at balabit.hu>
> > Subject: Re: [syslog-ng] turn off case sensitivity for match regex
> > filter
> > To: Syslog-ng users' and developers' mailing list
> > <syslog-ng at lists.balabit.hu>
> > Message-ID: <1177756926.14925.17.camel at bzorp.balabit>
> > Content-Type: text/plain
> >
> > On Sat, 2007-04-28 at 01:52 -0700, stucky wrote:
> > > Guys
> > >
> > > Playing around with ng 2 and I started looking at the match filter
> > > again.
> > > Simple question that I cannot find an answer to anywhere on the net.
> > > How do I turn off case sensitivity for the match target ?
> > > I'd like the following line to match "error' or 'ERROR' or 'Error'
> > >
> > > filter logparse { match("error"); };
> > >
> > > but of course it only matches 'error' since by default regex is case
> > > sensitive.
> > > Basically I'm trying to emulate 'grep -i'
> > > I guess I could do this :
> > >
> > > filter logparse { match("[Ee][Rr][Rr][Oo][Rr]"); }; but it'd be soo
> > > much simpler to turn off case sensitivity.
> >
> > Yes, you are right. But it's not currently possible. It should be
> > however, I'll try to add it in the nearfuture.
> >
> > >
> > > And while we're talking regex. Shouldn't the above line actually read
> > > like this :
> > >
> > > filter logparse { match(".+error.+"); }; ?
> > >
> > > meaning "anything followed by 'error' followed by anything"
> > > Both appear to work so I assume the first line is interpreted by
> > > syslog-ng like the second line correct ?
> >
> > syslog-ng interprets "match" the same as grep, e.g. it does not care
> > where the pattern is found. if you want to match the beginning or the
> > end of line, you need to use explicit ^ and $ characters.
> >
> > --
> > Bazsi
> >
> >
> >
> > ------------------------------
> >
> > Message: 2
> > Date: Sat, 28 Apr 2007 12:44:04 -0700
> > From: stucky <stucky101 at gmail.com>
> > Subject: Re: [syslog-ng] turn off case sensitivity for match regex
> > filter
> > To: "Syslog-ng users' and developers' mailing list"
> > <syslog-ng at lists.balabit.hu>
> > Message-ID:
> > <30997e260704281244g6f1225bfwc7da2b4e05fdfd9f at mail.gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> >
> > Baszi
> >
> > Cool. I'm in the middle of building a new infrastructure and would like
> to
> > use this feature. I'm not a programmer but I assume adding this feature
> > shouldn't be very hard at all right ?
> > If you had a rough ETA that'd help me.
> >
> > thx
> >
> > On 4/28/07, Balazs Scheidler <bazsi at balabit.hu> wrote:
> > >
> > > On Sat, 2007-04-28 at 01:52 -0700, stucky wrote:
> > > > Guys
> > > >
> > > > Playing around with ng 2 and I started looking at the match filter
> > > > again.
> > > > Simple question that I cannot find an answer to anywhere on the net.
> > > > How do I turn off case sensitivity for the match target ?
> > > > I'd like the following line to match "error' or 'ERROR' or 'Error'
> > > >
> > > > filter logparse { match("error"); };
> > > >
> > > > but of course it only matches 'error' since by default regex is case
> > > > sensitive.
> > > > Basically I'm trying to emulate 'grep -i'
> > > > I guess I could do this :
> > > >
> > > > filter logparse { match("[Ee][Rr][Rr][Oo][Rr]"); }; but it'd be soo
> > > > much simpler to turn off case sensitivity.
> > >
> > > Yes, you are right. But it's not currently possible. It should be
> > > however, I'll try to add it in the nearfuture.
> > >
> > > >
> > > > And while we're talking regex. Shouldn't the above line actually
> read
> > > > like this :
> > > >
> > > > filter logparse { match(".+error.+"); }; ?
> > > >
> > > > meaning "anything followed by 'error' followed by anything"
> > > > Both appear to work so I assume the first line is interpreted by
> > > > syslog-ng like the second line correct ?
> > >
> > > syslog-ng interprets "match" the same as grep, e.g. it does not care
> > > where the pattern is found. if you want to match the beginning or the
> > > end of line, you need to use explicit ^ and $ characters.
> > >
> > > --
> > > Bazsi
> > >
> > > _______________________________________________
> > > syslog-ng maillist - syslog-ng at lists.balabit.hu
> > > https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
> > >
> > >
> >
> >
> > --
> > stucky
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL:
> >
> http://lists.balabit.hu/pipermail/syslog-ng/attachments/20070428/c3bdb0f9/attachment.html
> >
> > ------------------------------
> >
> > _______________________________________________
> > syslog-ng maillist - syslog-ng at lists.balabit.hu
> > https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >
> >
> > End of syslog-ng Digest, Vol 24, Issue 32
> > *****************************************
> >
>
>
>
> --
> Regards,
>
> Fabian Pucciarelli
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://lists.balabit.hu/pipermail/syslog-ng/attachments/20070429/92be486d/attachment-0001.html
>
> ------------------------------
>
> Message: 2
> Date: Sun, 29 Apr 2007 19:21:11 +0200
> From: Balazs Scheidler <bazsi at balabit.hu>
> Subject: Re: [syslog-ng] turn off case sensitivity for match regex
> filter
> To: Syslog-ng users' and developers' mailing list
> <syslog-ng at lists.balabit.hu>
> Message-ID: <1177867271.9878.11.camel at bzorp.balabit>
> Content-Type: text/plain
>
> On Sat, 2007-04-28 at 12:44 -0700, stucky wrote:
> > Baszi
> >
> > Cool. I'm in the middle of building a new infrastructure and would
> > like to use this feature. I'm not a programmer but I assume adding
> > this feature shouldn't be very hard at all right ?
> > If you had a rough ETA that'd help me.
>
> attached patch implements it using perl-like syntax, e.g.:
>
> filter f_case { match('(?i)regexp'); };
>
> It works in all filters that use regexps (e.g. match, host, program,
> etc) The regexp must begin with '(?' or otherwise the flag will not be
> recognized.
>
> Tomorrow's snapshot should contain it.
>
> >From 47f53555268efb72ab8db2d620d7669b8e5dc7a4 Mon Sep 17 00:00:00 2001
> From: Balazs Scheidler <bazsi at balabit.hu>
> Date: Sun, 29 Apr 2007 18:53:46 +0200
> Subject: [PATCH] added support for Perl-like regexp flags to support
> case-ignoring matches
>
> 2007-04-28 Balazs Scheidler <bazsi at balabit.hu>
>
> * src/filter.c (filter_re_compile): parse '(?i)' at the beginning
> of
> regexps as an ignore-case flag, just like Perl does
> ---
> src/filter.c | 34 +++++++++++++++++++++++++++++++---
> tests/unit/test_filters.c | 5 +++++
> 2 files changed, 36 insertions(+), 3 deletions(-)
>
> diff --git a/src/filter.c b/src/filter.c
> index 8e19440..cf7e3aa 100644
> --- a/src/filter.c
> +++ b/src/filter.c
> @@ -230,11 +230,39 @@ typedef struct _FilterRE
> } FilterRE;
>
> static gboolean
> -filter_re_compile(const char *re, regex_t *regex)
> +filter_re_compile(const gchar *re, regex_t *regex)
> {
> - int rc;
> + gint rc;
> + const gchar *re_comp = re;
> + gint flags = REG_EXTENDED;
>
> - rc = regcomp(regex, re, REG_EXTENDED);
> + if (re[0] == '(' && re[1] == '?')
> + {
> + gint i;
> +
> + for (i = 2; re[i] && re[i] != ')'; i++)
> + {
> + switch (re[i])
> + {
> + case 'i':
> + flags |= REG_ICASE;
> + break;
> + }
> + }
> + if (re[i])
> + {
> + re_comp = &re[i + 1];
> + }
> + else
> + {
> + msg_error("Invalid regexp flags",
> + evt_tag_str("re", re),
> + NULL);
> + return FALSE;
> + }
> + }
> +
> + rc = regcomp(regex, re_comp, flags);
> if (rc)
> {
> gchar buf[256];
> diff --git a/tests/unit/test_filters.c b/tests/unit/test_filters.c
> index 58236d7..187b572 100644
> --- a/tests/unit/test_filters.c
> +++ b/tests/unit/test_filters.c
> @@ -137,11 +137,16 @@ main(int argc G_GNUC_UNUSED, char *argv[]
> G_GNUC_UNUSED)
>
> testcase("<15>Oct 15 16:17:01 host openvpn[2499]: PTHREAD support
> initialized", 0, filter_host_new("^host$"), 1);
> testcase("<15>Oct 15 16:17:01 host openvpn[2499]: PTHREAD support
> initialized", 0, filter_host_new("^hos$"), 0);
> + testcase("<15>Oct 15 16:17:01 host openvpn[2499]: PTHREAD support
> initialized", 0, filter_host_new("pthread"), 0);
> fprintf(stderr, "One \"invalid regular expressions\" message is to be
> expected\n");
> TEST_ASSERT(filter_host_new("((") == NULL);
>
> + fprintf(stderr, "One \"invalid regular expressions\" message is to be
> expected\n");
> + TEST_ASSERT(filter_host_new("(?iana") == NULL);
> +
> testcase("<15>Oct 15 16:17:01 host openvpn[2499]: PTHREAD support
> initialized", 0, filter_match_new(" PTHREAD "), 1);
> testcase("<15>Oct 15 16:17:01 host openvpn[2499]: PTHREAD support
> initialized", 0, filter_match_new("^PTHREAD$"), 0);
> + testcase("<15>Oct 15 16:17:01 host openvpn[2499]: PTHREAD support
> initialized", 0, filter_match_new("(?i)pthread"), 1);
> fprintf(stderr, "One \"invalid regular expression\" message is to be
> expected\n");
> TEST_ASSERT(filter_match_new("((") == NULL);
>
>
>
> --
> Bazsi
>
>
>
> ------------------------------
>
> Message: 3
> Date: Sun, 29 Apr 2007 19:24:51 +0200
> From: Balazs Scheidler <bazsi at balabit.hu>
> Subject: Re: [syslog-ng] Re: syslog-ng Digest, Vol 24, Issue 32
> To: Syslog-ng users' and developers' mailing list
> <syslog-ng at lists.balabit.hu>
> Message-ID: <1177867491.9878.16.camel at bzorp.balabit>
> Content-Type: text/plain
>
> On Sun, 2007-04-29 at 10:25 -0600, Fabian Pucciarelli wrote:
> > I have syslog ng writing to a mysql pipe and then a little script
> > reading the pipe and writing to the db. I currently receive many
> > messages like the following, I wonder if somenone can help me figure
> > out what this message is saying..... thanks.
> >
> >
> > | mailux | syslog | notice | notice | 2d | 2007-04-13 | 01:16:40
> > | syslog-ng | syslog-ng[30548]: Log statistics;
> > dropped='pipe(/tmp/mysql.pipe)=0', processed='center(queued)=40295',
> > processed='center(received)=13206',
> > processed='destination(d_mysql)=40295', processed='source(net_tcp)=0',
> > processed='source(src)=8962', processed='source(net_udp)=4244' |
> > 46674 |
>
> This contains the various message counters that syslog-ng maintains
> internally.
>
> 'processed' counters just count how much messages were processed at the
> given point, 'dropped' counters indicate how much log messages were
> dropped by syslog-ng itself.
>
> The string in the parenthesis tell were the given counter is counting
> inside syslog-ng.
>
> 'center' is the main log message dispatching mechanism, it received
> 13206 messages from various sources and then sent 40295 messages out,
> e.g. you are probably sending individual messages to multiple
> destinations.
>
> You have two sources, one named 'src' generated 8962, the other called
> 'net_udp' 4244 messages.
>
> --
> Bazsi
>
>
>
> ------------------------------
>
> _______________________________________________
> syslog-ng maillist - syslog-ng at lists.balabit.hu
> https://lists.balabit.hu/mailman/listinfo/syslog-ng
>
>
> End of syslog-ng Digest, Vol 24, Issue 33
> *****************************************
>
--
Regards,
Fabian Pucciarelli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20070430/1e71e4bf/attachment.html
More information about the syslog-ng
mailing list