attachment is my current syslog-ng.conf.   and  esxi_pattern.xml.

my syslog-ng receive UDP log from esxi host and try to test the db-parse and log it.

i have change to <pattern>system</pattern>, but still can not get value from parse refer macro.

thanks.


2013/4/28 Evan Rempel <erempel@uvic.ca>
Sorry for not being more clear in my first response.

You have a template of

template("=== $PROGRAM,${.esxi_month} ${.esxi.date} ${.esxi.time} HOSTIP ${.esxi.host_ip},${.esxi.message}\n")

When syslog-ng receives a syslog message, it logged it as;

=== system,error,critical,   HOST IP ,

This means that $PROGRAM contains "system"

Now for the patterndb part.

The patterndb parser FIRST matches $PROGRAM To the <pattern>XXXX</pattern> in the <ruleset>

<?xml version="1.0" encoding="utf-8"?>
<patterndb version='3' pub_date='2009-04-17'>
    <ruleset name='esxi' id='123456678'>
        <pattern>XXXX</pattern>

In your case you have specified <pattern>ESXI</pattern> so the patterndb parser will NOT use any
of your patterndb because it does not match the $PROGRAM

You need to use

########   esxi_pattern.xml ############
<?xml version="1.0" encoding="utf-8"?>
<patterndb version='3' pub_date='2009-04-17'>
<ruleset name='esxi' id='123456678'>
<pattern>system</pattern>
<rules>
<rule provider='Fone Bro' id='182437592347598' class='esxi'>
<patterns>
<pattern>@STRING:.esxi.month:@ @STRING:.esxi.date:@
@STRING:.esxi.time::@@IPv4:.esxi.host_ip:@
@ESTRING:.esxi.program::@ @ANYSTRING:.esxi.message@</pattern>
</patterns>
</rule>
</rules>
</ruleset>
</patterndb>



You have not included a complete syslong-ng source line for me to see what you are trying to match against so I can
not tell if you pattern will actually match the lines that you are trying to match.
At my organization we run ESX as well, and none of our lines would match the pattern that you have, but
your environment might be different.

I hope this was more clear.

Evan.




________________________________________
From: ²»»µ°¢·å [onlydebian@gmail.com]
Sent: Sunday, April 28, 2013 8:24 AM
To: syslog-ng@lists.balabit.hu; Evan Rempel
Subject: Re:Can not get DBParse match macro result (syslog-ng 3.13 debian squeeze)

thanks to your reply.  i do not understand how to do now. it puzzle and trouble me some days.   i read the balabit syslog-ng OSE guide documents and only have simple information in there.

how to do on this
----->>>>
If you change the patterndb ruleset pattern to use a program of system rather than ESXI I think it would work.


2013/4/28 <syslog-ng-request@lists.balabit.hu<mailto:syslog-ng-request@lists.balabit.hu>>
Send syslog-ng mailing list submissions to
        syslog-ng@lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>

To subscribe or unsubscribe via the World Wide Web, visit
        https://lists.balabit.hu/mailman/listinfo/syslog-ng
or, via email, send a message with subject or body 'help' to
        syslog-ng-request@lists.balabit.hu<mailto:syslog-ng-request@lists.balabit.hu>

You can reach the person managing the list at
        syslog-ng-owner@lists.balabit.hu<mailto:syslog-ng-owner@lists.balabit.hu>

When replying, please edit your Subject line so it is more specific
than "Re: Contents of syslog-ng digest..."


Today's Topics:

   1.  Can not get DBParse match macro result (syslog-ng 3.13
      debian squeeze) (????)
   2. Re:  Can not get DBParse match macro result (syslog-ng 3.13
      debian squeeze) (Evan Rempel)


----------------------------------------------------------------------

Message: 1
Date: Sat, 27 Apr 2013 22:34:50 +0800
From: ???? <onlydebian@gmail.com<mailto:onlydebian@gmail.com>>
Subject: [syslog-ng] Can not get DBParse match macro result (syslog-ng
        3.13    debian squeeze)
To: syslog-ng@lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>
Message-ID:
        <CA+SSH2oBB2-WWvQksbchVVoyhfZbdVvDR=V7wJ1EJdvE6Zx9zg@mail.gmail.com<mailto:V7wJ1EJdvE6Zx9zg@mail.gmail.com>>
Content-Type: text/plain; charset="iso-8859-1"

when use pdbtool do match test, it is success. but from syslog-ng can not
return result of macro
i can not get macro result.  for example,   ${.esxi.month}  no value, same
as ${.esxi.host_ip} ${.esxi.time}

test log output ,just like this.
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,


do the pdbtool test, it's ok.  wish someone can give me some solution and
help. i have search some mail list but i can not get the right solution.
 thanks a lot.

root@debian:~# pdbtool match -D -c -p
/etc/syslog-ng/patterndb/esxi_pattern.xml -P ESXI -M "Apr 26 15:17:31
192.168.88.71 vmkernel: cpu11:8203)NMP: nmp_ThrottleLogForDevice:2319: Cmd
0x1a (0x4124444a6280, 0) to dev "mpx.vmhba0:C0:T0:L0" on path
"vmhba0:C0:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
Act:NONE"
Pattern matching part:
@STRING:.esxi.month=Apr@ @STRING:.esxi.date=26@
@STRING:.esxi.time=15:17:31@@IPv4:.esxi.host_ip=192.168.88.71@@ESTRING:.esxi.program=
vmkernel: cpu11:8203)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x1a
(0x4124444a6280, 0) to dev mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0
Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0.
Act:NONE@@ANYSTRING:.esxi.message=cpu11:8203)NMP:
nmp_ThrottleLogForDevice:2319: Cmd
0x1a (0x4124444a6280, 0) to dev mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0
Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE@
Matching part:
Apr 26 15:17:31 192.168.88.71 vmkernel: cpu11:8203)NMP:
nmp_ThrottleLogForDevice:2319: Cmd 0x1a (0x4124444a6280, 0) to dev
mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0 Failed: H:0x0 D:0x2 P:0x0 Valid
sense data: 0x5 0x20 0x0. Act:NONE
Values:
MESSAGE=Apr 26 15:17:31 192.168.88.71 vmkernel: cpu11:8203)NMP:
nmp_ThrottleLogForDevice:2319: Cmd 0x1a (0x4124444a6280, 0) to dev
mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0 Failed: H:0x0 D:0x2 P:0x0 Valid
sense data: 0x5 0x20 0x0. Act:NONE
PROGRAM=ESXI
.classifier.class=esxi
.classifier.rule_id=182437592347598
.esxi.month=Apr
.esxi.date=26
.esxi.time=15:17:31
.esxi.host_ip=192.168.88.71
.esxi.program= vmkernel
.esxi.message=cpu11:8203)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x1a
(0x4124444a6280, 0) to dev mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0
Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
root@debian:~#


my configuration like as below

########   esxi_pattern.xml ############
<?xml version="1.0" encoding="utf-8"?>
<patterndb version='3' pub_date='2009-04-17'>
<ruleset name='esxi' id='123456678'>
<pattern>ESXI</pattern>
<rules>
<rule provider='Fone Bro' id='182437592347598' class='esxi'>
<patterns>
<pattern>@STRING:.esxi.month:@ @STRING:.esxi.date:@
@STRING:.esxi.time::@@IPv4:.esxi.host_ip:@
@ESTRING:.esxi.program::@ @ANYSTRING:.esxi.message@</pattern>
</patterns>
</rule>
</rules>
</ruleset>
</patterndb>

######## syslog-ng.conf      ########

#####Parser#####
parser pattern_db {
        db_parser( file("/etc/syslog-ng/patterndb/esxi_pattern.xml"));
};

#Check pattern matching
destination udp_esxi_output {
   file("/var/log/pattern_output"
   template("=== $PROGRAM,${.esxi_month} ${.esxi.date} ${.esxi.time} HOST
IP ${.esxi.host_ip},${.esxi.message}\n")
template_escape(no));
};

#####Log#####
log {
        source(s_network);
        parser(pattern_db);
        destination(udp_esxi_output);
};
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20130427/20e80756/attachment.html

------------------------------

Message: 2
Date: Sat, 27 Apr 2013 16:10:02 +0000
From: Evan Rempel <erempel@uvic.ca<mailto:erempel@uvic.ca>>
Subject: Re: [syslog-ng] Can not get DBParse match macro result
        (syslog-ng 3.13 debian squeeze)
To: "Syslog-ng users' and developers' mailing list"
        <syslog-ng@lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>>
Message-ID: <q8vb966l4qe0219lsusm5ju4.1367078999747@email.android.com<mailto:q8vb966l4qe0219lsusm5ju4.1367078999747@email.android.com>>
Content-Type: text/plain; charset="iso-2022-jp"

It would appear that you have everything correct when the "PROGRAM" is ESXI but the log line as syslog-ng sees it has a PROGRAM of "system" according to your test log output.

If you change the patterndb ruleset pattern to use a program of system rather than ESXI I think it would work.


Evan Rempel   250.271.7691<tel:250.271.7691>
University Systems, University of Victoria

???? <onlydebian@gmail.com<mailto:onlydebian@gmail.com>> wrote:



when use pdbtool do match test, it is success. but from syslog-ng can not return result of macro
i can not get macro result.  for example,   ${.esxi.month}  no value, same as ${.esxi.host_ip} ${.esxi.time}

test log output ,just like this.
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,
=== system,error,critical,   HOST IP ,


do the pdbtool test, it's ok.  wish someone can give me some solution and help. i have search some mail list but i can not get the right solution.  thanks a lot.

root@debian:~# pdbtool match -D -c -p /etc/syslog-ng/patterndb/esxi_pattern.xml -P ESXI -M "Apr 26 15:17:31 192.168.88.71 vmkernel: cpu11:8203)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x1a (0x4124444a6280, 0) to dev "mpx.vmhba0:C0:T0:L0" on path "vmhba0:C0:T0:L0" Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE"
Pattern matching part:
@STRING:.esxi.month=Apr@ @STRING:.esxi.date=26@ @STRING:.esxi.time=15:17:31@ @IPv4:.esxi.host_ip=192.168.88.71@@ESTRING:.esxi.program= vmkernel: cpu11:8203)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x1a (0x4124444a6280, 0) to dev mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0 Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE@ @ANYSTRING:.esxi.message=cpu11:8203)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x1a (0x4124444a6280, 0) to dev mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0 Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE@
Matching part:
Apr 26 15:17:31 192.168.88.71 vmkernel: cpu11:8203)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x1a (0x4124444a6280, 0) to dev mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0 Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
Values:
MESSAGE=Apr 26 15:17:31 192.168.88.71 vmkernel: cpu11:8203)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x1a (0x4124444a6280, 0) to dev mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0 Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
PROGRAM=ESXI
.classifier.class=esxi
.classifier.rule_id=182437592347598
.esxi.month=Apr
.esxi.date=26
.esxi.time=15:17:31
.esxi.host_ip=192.168.88.71
.esxi.program= vmkernel
.esxi.message=cpu11:8203)NMP: nmp_ThrottleLogForDevice:2319: Cmd 0x1a (0x4124444a6280, 0) to dev mpx.vmhba0:C0:T0:L0 on path vmhba0:C0:T0:L0 Failed: H:0x0 D:0x2 P:0x0 Valid sense data: 0x5 0x20 0x0. Act:NONE
root@debian:~#


my configuration like as below

########   esxi_pattern.xml ############
<?xml version="1.0" encoding="utf-8"?>
<patterndb version='3' pub_date='2009-04-17'>
<ruleset name='esxi' id='123456678'>
<pattern>ESXI</pattern>
<rules>
<rule provider='Fone Bro' id='182437592347598' class='esxi'>
<patterns>
<pattern>@STRING:.esxi.month:@ @STRING:.esxi.date:@ @STRING:.esxi.time::@ @IPv4:.esxi.host_ip:@@ESTRING:.esxi.program::@ @ANYSTRING:.esxi.message@</pattern>
</patterns>
</rule>
</rules>
</ruleset>
</patterndb>

######## syslog-ng.conf      ########

#####Parser#####
parser pattern_db {
        db_parser( file("/etc/syslog-ng/patterndb/esxi_pattern.xml"));
};

#Check pattern matching
destination udp_esxi_output {
   file("/var/log/pattern_output"
   template("=== $PROGRAM,${.esxi_month} ${.esxi.date} ${.esxi.time} HOST IP ${.esxi.host_ip},${.esxi.message}\n")
template_escape(no));
};

#####Log#####
log {
        source(s_network);
        parser(pattern_db);
        destination(udp_esxi_output);
};

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20130427/2f67c039/attachment-0001.htm

------------------------------

_______________________________________________
syslog-ng maillist  -  syslog-ng@lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>
https://lists.balabit.hu/mailman/listinfo/syslog-ng


End of syslog-ng Digest, Vol 96, Issue 25
*****************************************