Thanks for the answer, but this will not solve the problem. I'll explain better the environment... we are collecting a high rate of messages for second, normalizing, storing in a db and doing quite real-time analysis and graphs. The rate of messages is really important, because it's not unusual to see sustained peak of about 3000 msg/sec for 10/20 minutes. To normalize and store them we used some parser but it's really impossible to use functions like regexec cause they will slow things down. (things are already slow down by the parser and the db wrapper) I was wondering if there where any other way, to get the $MSG - (less) $PROGRAM... I don't know the syslog-ng internals quite well, that's why I asked here if someone have done something similar patching syslog-ng. Also I cannot use match() cause all defined rules ( that are named in this environment ) have to be collected in the db, and I have no way to change this behaviour. Thanks again. Amodiovalerio Verde ----- Original Message ----- From: "Balazs Scheidler" <bazsi@balabit.hu> To: <syslog-ng@lists.balabit.hu> Sent: Thursday, June 05, 2003 4:47 PM Subject: Re: [syslog-ng]filter match()

On Mon, May 26, 2003 at 11:50:28AM +0200, Amodiovalerio Verde wrote:

...

Hi all,

during some extensive test on syslog-ng, I found something that really make me difficult to implement some filters.

I need to match same EXACT messages, and so I used something like match("^MATCHTHIS$") in a filter statement...

but I find that the match will consider the message starting from the program name...

Is there any way to match JUST the message part without the program one ??? ( the program() just work on the program part of the message, so i expected that match() will work on the other one... )

as the syslog message format varies from platform to platform it is quite difficult to parse which part is which. So while it is quite easy to recognize the whole message, it is more difficult to know which part is the programname. Therefore the message as a whole is stored, as it is used to construct forwarded messages. (think about the template "$DATE $HOST $MSG")

Apart from this it is also certainly possible to store the message part in itself, but the same can be accomplished by using regular expressions:

match("[a-zA-Z0-9]+(\[[0-9]+\])?: EXACTMATCH$"); ^^^^^^^^^^^^^^^^^^^^^^^^^ this matches the programname and its optional pid.

-- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1 _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

misunderstanding....my fault... I'm NOT doing syslog-ng match() but using some C written parser to parse the messages and fill the database. I meant I can't use regexec() and regcomp() cause are really slow functions... I have NOT to filter against program and messages but SPLIT them.... To summarize : A number of network appliance write to the syslog servers ( actually there are 3 in load balancing ) Filters are used in syslog-ng to let the user choose what to collect and where If the destination is the db, some parser are used as destination...the get the message, parse it, split fields also in the message part and fill the tables That way the user can have graphs and statistics and query the db. A fast example... a lot of messages cames from PIX....so are in the form...(if i remember...) %PIX-6-303002 Build TCP outbound connection : laddr 1.1.1.1/80 gaddr 2.2.2.2/100 faddr 3.3.3.3/110 what I actually do is let the user choose what kind of messages he wants ( using syslog-ng match() )... then get the message and parse it to doing specific content management...so I fill a table with fields : laddr,lport,gaddr,gport,faddr,fport,proto,outbound/inbound...and so on I got a big help from syslog-ng cause I pass the message to the parser using something like $TAG\t$R_DATE\t$SOURCEIP.....but the $MSG part give me not the message part alone but inclusive of the program ( and optional pid ) ... Because it exists a $PROGRAM macro that correctly match the program+pid part, I was wondering why the $MSG part have to include again it... Hope to be clear :P sorry but my english is not so good. Amodiovalerio Verde ----- Original Message ----- From: "Jonas Lundgren" <neonman@copyleft.mine.nu> To: <syslog-ng@lists.balabit.hu> Sent: Friday, June 06, 2003 11:34 AM Subject: Re: [syslog-ng]filter match()

...

Hi all,

during some extensive test on syslog-ng, I found something that really make me difficult to implement some filters.

I need to match same EXACT messages, and so I used something like match("^MATCHTHIS$") in a filter statement...

but I find that the match will consider the message starting from the program name...

Is there any way to match JUST the message part without the program one ??? ( the program() just work on the program part of the message, so i expected that match() will work on the other one... )

Thanks in advance

Amodiovalerio Verde

amodiovalerio.verde@ags-it.com amover@libero.it

_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html

A regexp won't slow things down.

_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html