syslog-ng 3.3.3 rewrite question regarding cisco IOS Messages
Hello List, I try to rewrite cisco IOS syslog messages with timestamps in the MESSAGE field. I want to remove the timestamp from the message and set the program to the so called mnemonic of the message.. I found the following example on the list: ... filter f_rewrite_cisco_program { match('%([^:]: ([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")); }; rewrite r_cisco_program { set("$1", value("PROGRAM") condition(filter(f_rewrite_cisco_program))); set("$2", value("MESSAGE") condition(filter(f_rewrite_cisco_program))); }; log { source(s_all); rewrite(r_cisco_program); destination(d_mydestination); }; But that does not work. I tried a lot of different rewrite syntaxes, none of them work for me. If I just employ the filter f_rewrite_cisco_program I` am able to filter out the cisco messages. Sample log line (written with template $R_ISODATE $HOST $MSG): 2011-11-30T18:23:50+01:00 192.168.1.1 217122: Nov 30 17:23:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down I`m using syslog-ng 3.3.3 debian package from madhouse. How I can rewrite my messages to filter the timestamp in the message field? any ideas? Any help is higly welcome, thanks in advance, Tom ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.
That was mine, and I think there's a couple mistakes in it because there appears to a be a missing parenthesis and a plus sign. Try this: match('%([^:]+):\s+([^\n]+)' value("MESSAGE") type("pcre") On Wed, Nov 30, 2011 at 11:26 AM, Thomas Wollner <tw@wollner-net.de> wrote:
Hello List,
I try to rewrite cisco IOS syslog messages with timestamps in the MESSAGE field. I want to remove the timestamp from the message and set the program to the so called mnemonic of the message..
I found the following example on the list:
... filter f_rewrite_cisco_program { match('%([^:]: ([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")); };
rewrite r_cisco_program { set("$1", value("PROGRAM") condition(filter(f_rewrite_cisco_program))); set("$2", value("MESSAGE") condition(filter(f_rewrite_cisco_program))); };
log { source(s_all); rewrite(r_cisco_program); destination(d_mydestination); };
But that does not work. I tried a lot of different rewrite syntaxes, none of them work for me. If I just employ the filter f_rewrite_cisco_program I` am able to filter out the cisco messages.
Sample log line (written with template $R_ISODATE $HOST $MSG):
2011-11-30T18:23:50+01:00 192.168.1.1 217122: Nov 30 17:23:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
I`m using syslog-ng 3.3.3 debian package from madhouse.
How I can rewrite my messages to filter the timestamp in the message field? any ideas?
Any help is higly welcome, thanks in advance,
Tom
---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Should probably throw something in there to make that match a little less ambiguous (so it doesnt match just anything with a % and : in it). Sent: Wed Nov 30 2011 12:20:03 GMT-0700 (MST) From: Martin Holste <mcholste@gmail.com> To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] syslog-ng 3.3.3 rewrite question regarding cisco IOS Messages
That was mine, and I think there's a couple mistakes in it because there appears to a be a missing parenthesis and a plus sign. Try this:
match('%([^:]+):\s+([^\n]+)' value("MESSAGE") type("pcre")
On Wed, Nov 30, 2011 at 11:26 AM, Thomas Wollner<tw@wollner-net.de> wrote:
Hello List,
I try to rewrite cisco IOS syslog messages with timestamps in the MESSAGE field. I want to remove the timestamp from the message and set the program to the so called mnemonic of the message..
I found the following example on the list:
... filter f_rewrite_cisco_program { match('%([^:]: ([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")); };
rewrite r_cisco_program { set("$1", value("PROGRAM") condition(filter(f_rewrite_cisco_program))); set("$2", value("MESSAGE") condition(filter(f_rewrite_cisco_program))); };
log { source(s_all); rewrite(r_cisco_program); destination(d_mydestination); };
But that does not work. I tried a lot of different rewrite syntaxes, none of them work for me. If I just employ the filter f_rewrite_cisco_program I` am able to filter out the cisco messages.
Sample log line (written with template $R_ISODATE $HOST $MSG):
2011-11-30T18:23:50+01:00 192.168.1.1 217122: Nov 30 17:23:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
I`m using syslog-ng 3.3.3 debian package from madhouse.
How I can rewrite my messages to filter the timestamp in the message field? any ideas?
Any help is higly welcome, thanks in advance,
Tom
---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Yeah, that's a good call. My original, more specific filter was this: filter f_rewrite_cisco_program { match('^(%(?:ASA|PIX|FWSM)\-\d\-\d{6}): ([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")); }; You're probably best off filtering by host IP, because adding in more to the regex to make it more specific could mean missing Cisco messages with non-standard timestamps, etc. On Wed, Nov 30, 2011 at 1:23 PM, Patrick H. <syslogng@feystorm.net> wrote:
Should probably throw something in there to make that match a little less ambiguous (so it doesnt match just anything with a % and : in it).
Sent: Wed Nov 30 2011 12:20:03 GMT-0700 (MST) From: Martin Holste <mcholste@gmail.com> To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] syslog-ng 3.3.3 rewrite question regarding cisco IOS Messages
That was mine, and I think there's a couple mistakes in it because there appears to a be a missing parenthesis and a plus sign. Try this:
match('%([^:]+):\s+([^\n]+)' value("MESSAGE") type("pcre")
On Wed, Nov 30, 2011 at 11:26 AM, Thomas Wollner <tw@wollner-net.de> wrote:
Hello List,
I try to rewrite cisco IOS syslog messages with timestamps in the MESSAGE field. I want to remove the timestamp from the message and set the program to the so called mnemonic of the message..
I found the following example on the list:
... filter f_rewrite_cisco_program { match('%([^:]: ([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")); };
rewrite r_cisco_program { set("$1", value("PROGRAM") condition(filter(f_rewrite_cisco_program))); set("$2", value("MESSAGE") condition(filter(f_rewrite_cisco_program))); };
log { source(s_all); rewrite(r_cisco_program); destination(d_mydestination); };
But that does not work. I tried a lot of different rewrite syntaxes, none of them work for me. If I just employ the filter f_rewrite_cisco_program I` am able to filter out the cisco messages.
Sample log line (written with template $R_ISODATE $HOST $MSG):
2011-11-30T18:23:50+01:00 192.168.1.1 217122: Nov 30 17:23:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
I`m using syslog-ng 3.3.3 debian package from madhouse.
How I can rewrite my messages to filter the timestamp in the message field? any ideas?
Any help is higly welcome, thanks in advance,
Tom
---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin, thanks for your suggestions. I just checked the new pattern, but the rewrite would not happen, too. I dont think that the pattern is the cause of the problem, because if I employ just the filter as an filter inside a logstatement, I receive cisco messages only in the destination. So the pattern matches. But no rewrite happens so far. My config is: destination d_mydestination_rewritten { file("/var/log/mylog-rewritten.log"); }; destination d_mydestination_raw { file("/var/log/mylog-raw.log"); }; destination d_mydestination_justcisco { file("/var/log/mylog-justcisco.log"); }; filter f_rewrite_cisco_program { match('%([^:]+):\s+([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")); }; rewrite r_cisco_program { set("$1", value("PROGRAM") condition(filter(f_rewrite_cisco_program))); set("$2", value("MESSAGE") condition(filter(f_rewrite_cisco_program))); }; log { source(s_src); rewrite(r_cisco_program); destination(d_mydestination_rewritten); }; log { source(s_src); filter(f_rewrite_cisco_program); destination(d_mydestination_justcisco); }; log { source(s_src); destination(d_mydestination_raw); }; The resulting logfiles: grep SYS-5-CONFIG /var/log/mylog-* /var/log/mylog-justcisco.log:Nov 30 19:59:10 192.168.111.10 2333: Nov 30 19:59:09.609: %SYS-5-CONFIG_I: Configured from console by tom on vty0 (192.168.1.2) /var/log/mylog-raw.log:Nov 30 19:59:10 192.168.111.10 2333: Nov 30 19:59:09.609: %SYS-5-CONFIG_I: Configured from console by tom on vty0 (192.168.1.2) /var/log/mylog-rewritten.log:Nov 30 19:59:10 192.168.111.10 2333: Nov 30 19:59:09.609: %SYS-5-CONFIG_I: Configured from console by tom on vty0 (192.168.1.2) So something must be wrong using the rewrite or the rewrite rule himself... Any ideas, further suggestions? Thanks in advance, Tom On 30.11.2011 20:20, Martin Holste wrote:
That was mine, and I think there's a couple mistakes in it because there appears to a be a missing parenthesis and a plus sign. Try this:
match('%([^:]+):\s+([^\n]+)' value("MESSAGE") type("pcre")
On Wed, Nov 30, 2011 at 11:26 AM, Thomas Wollner <tw@wollner-net.de> wrote:
Hello List,
I try to rewrite cisco IOS syslog messages with timestamps in the MESSAGE field. I want to remove the timestamp from the message and set the program to the so called mnemonic of the message..
I found the following example on the list:
... filter f_rewrite_cisco_program { match('%([^:]: ([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")); };
rewrite r_cisco_program { set("$1", value("PROGRAM") condition(filter(f_rewrite_cisco_program))); set("$2", value("MESSAGE") condition(filter(f_rewrite_cisco_program))); };
log { source(s_all); rewrite(r_cisco_program); destination(d_mydestination); };
But that does not work. I tried a lot of different rewrite syntaxes, none of them work for me. If I just employ the filter f_rewrite_cisco_program I` am able to filter out the cisco messages.
Sample log line (written with template $R_ISODATE $HOST $MSG):
2011-11-30T18:23:50+01:00 192.168.1.1 217122: Nov 30 17:23:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
I`m using syslog-ng 3.3.3 debian package from madhouse.
How I can rewrite my messages to filter the timestamp in the message field? any ideas?
Any help is higly welcome, thanks in advance,
Tom
---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFO1ozwTCCRT+dccOYRAhZgAKCfeYWPN0UFGrk+wQs9iR4AYV5MeACgxhRo ugsR6kHA/TbnemGBNEwxG70= =ZGi3 -----END PGP SIGNATURE-----
There was a bugfix recently to address conditional rewrite problems in the 3.3 branch, what revision are you on? On Wed, Nov 30, 2011 at 2:07 PM, Thomas Wollner <tw@wollner-net.de> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Martin,
thanks for your suggestions. I just checked the new pattern, but the rewrite would not happen, too. I dont think that the pattern is the cause of the problem, because if I employ just the filter as an filter inside a logstatement, I receive cisco messages only in the destination. So the pattern matches. But no rewrite happens so far.
My config is:
destination d_mydestination_rewritten { file("/var/log/mylog-rewritten.log"); };
destination d_mydestination_raw { file("/var/log/mylog-raw.log"); };
destination d_mydestination_justcisco { file("/var/log/mylog-justcisco.log"); };
filter f_rewrite_cisco_program { match('%([^:]+):\s+([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")); };
rewrite r_cisco_program { set("$1", value("PROGRAM") condition(filter(f_rewrite_cisco_program))); set("$2", value("MESSAGE") condition(filter(f_rewrite_cisco_program))); };
log { source(s_src); rewrite(r_cisco_program); destination(d_mydestination_rewritten); };
log { source(s_src); filter(f_rewrite_cisco_program); destination(d_mydestination_justcisco); };
log { source(s_src); destination(d_mydestination_raw); };
The resulting logfiles: grep SYS-5-CONFIG /var/log/mylog-* /var/log/mylog-justcisco.log:Nov 30 19:59:10 192.168.111.10 2333: Nov 30 19:59:09.609: %SYS-5-CONFIG_I: Configured from console by tom on vty0 (192.168.1.2) /var/log/mylog-raw.log:Nov 30 19:59:10 192.168.111.10 2333: Nov 30 19:59:09.609: %SYS-5-CONFIG_I: Configured from console by tom on vty0 (192.168.1.2) /var/log/mylog-rewritten.log:Nov 30 19:59:10 192.168.111.10 2333: Nov 30 19:59:09.609: %SYS-5-CONFIG_I: Configured from console by tom on vty0 (192.168.1.2)
So something must be wrong using the rewrite or the rewrite rule himself...
Any ideas, further suggestions?
Thanks in advance,
Tom
On 30.11.2011 20:20, Martin Holste wrote:
That was mine, and I think there's a couple mistakes in it because there appears to a be a missing parenthesis and a plus sign. Try this:
match('%([^:]+):\s+([^\n]+)' value("MESSAGE") type("pcre")
On Wed, Nov 30, 2011 at 11:26 AM, Thomas Wollner <tw@wollner-net.de> wrote:
Hello List,
I try to rewrite cisco IOS syslog messages with timestamps in the MESSAGE field. I want to remove the timestamp from the message and set the program to the so called mnemonic of the message..
I found the following example on the list:
... filter f_rewrite_cisco_program { match('%([^:]: ([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")); };
rewrite r_cisco_program { set("$1", value("PROGRAM") condition(filter(f_rewrite_cisco_program))); set("$2", value("MESSAGE") condition(filter(f_rewrite_cisco_program))); };
log { source(s_all); rewrite(r_cisco_program); destination(d_mydestination); };
But that does not work. I tried a lot of different rewrite syntaxes, none of them work for me. If I just employ the filter f_rewrite_cisco_program I` am able to filter out the cisco messages.
Sample log line (written with template $R_ISODATE $HOST $MSG):
2011-11-30T18:23:50+01:00 192.168.1.1 217122: Nov 30 17:23:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
I`m using syslog-ng 3.3.3 debian package from madhouse.
How I can rewrite my messages to filter the timestamp in the message field? any ideas?
Any help is higly welcome, thanks in advance,
Tom
---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iD8DBQFO1ozwTCCRT+dccOYRAhZgAKCfeYWPN0UFGrk+wQs9iR4AYV5MeACgxhRo ugsR6kHA/TbnemGBNEwxG70= =ZGi3 -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, my version is from madhouse and installed via apt-get... syslog-ng -V syslog-ng 3.3.3 Installer-Version: 3.3.3 Revision: Debian/3.3.3.dfsg-1~mhp0~squeeze Compile-Date: Nov 18 2011 15:37:57 thank you, regards, Tom On 30.11.2011 21:31, Martin Holste wrote:
There was a bugfix recently to address conditional rewrite problems in the 3.3 branch, what revision are you on?
On Wed, Nov 30, 2011 at 2:07 PM, Thomas Wollner <tw@wollner-net.de> wrote: Martin,
thanks for your suggestions. I just checked the new pattern, but the rewrite would not happen, too. I dont think that the pattern is the cause of the problem, because if I employ just the filter as an filter inside a logstatement, I receive cisco messages only in the destination. So the pattern matches. But no rewrite happens so far.
My config is:
destination d_mydestination_rewritten { file("/var/log/mylog-rewritten.log"); };
destination d_mydestination_raw { file("/var/log/mylog-raw.log"); };
destination d_mydestination_justcisco { file("/var/log/mylog-justcisco.log"); };
filter f_rewrite_cisco_program { match('%([^:]+):\s+([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")); };
rewrite r_cisco_program { set("$1", value("PROGRAM") condition(filter(f_rewrite_cisco_program))); set("$2", value("MESSAGE") condition(filter(f_rewrite_cisco_program))); };
log { source(s_src); rewrite(r_cisco_program); destination(d_mydestination_rewritten); };
log { source(s_src); filter(f_rewrite_cisco_program); destination(d_mydestination_justcisco); };
log { source(s_src); destination(d_mydestination_raw); };
The resulting logfiles: grep SYS-5-CONFIG /var/log/mylog-* /var/log/mylog-justcisco.log:Nov 30 19:59:10 192.168.111.10 2333: Nov 30 19:59:09.609: %SYS-5-CONFIG_I: Configured from console by tom on vty0 (192.168.1.2) /var/log/mylog-raw.log:Nov 30 19:59:10 192.168.111.10 2333: Nov 30 19:59:09.609: %SYS-5-CONFIG_I: Configured from console by tom on vty0 (192.168.1.2) /var/log/mylog-rewritten.log:Nov 30 19:59:10 192.168.111.10 2333: Nov 30 19:59:09.609: %SYS-5-CONFIG_I: Configured from console by tom on vty0 (192.168.1.2)
So something must be wrong using the rewrite or the rewrite rule himself...
Any ideas, further suggestions?
Thanks in advance,
Tom
On 30.11.2011 20:20, Martin Holste wrote:
That was mine, and I think there's a couple mistakes in it because there appears to a be a missing parenthesis and a plus sign. Try this:
match('%([^:]+):\s+([^\n]+)' value("MESSAGE") type("pcre")
On Wed, Nov 30, 2011 at 11:26 AM, Thomas Wollner <tw@wollner-net.de> wrote:
Hello List,
I try to rewrite cisco IOS syslog messages with timestamps in the MESSAGE field. I want to remove the timestamp from the message and set the program to the so called mnemonic of the message..
I found the following example on the list:
... filter f_rewrite_cisco_program { match('%([^:]: ([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")); };
rewrite r_cisco_program { set("$1", value("PROGRAM") condition(filter(f_rewrite_cisco_program))); set("$2", value("MESSAGE") condition(filter(f_rewrite_cisco_program))); };
log { source(s_all); rewrite(r_cisco_program); destination(d_mydestination); };
But that does not work. I tried a lot of different rewrite syntaxes, none of them work for me. If I just employ the filter f_rewrite_cisco_program I` am able to filter out the cisco messages.
Sample log line (written with template $R_ISODATE $HOST $MSG):
2011-11-30T18:23:50+01:00 192.168.1.1 217122: Nov 30 17:23:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down
I`m using syslog-ng 3.3.3 debian package from madhouse.
How I can rewrite my messages to filter the timestamp in the message field? any ideas?
Any help is higly welcome, thanks in advance,
Tom
----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFO1quATCCRT+dccOYRAvC/AJ0V7sPbuv8bLlJB0QSXonssP9EevgCg4cXB GXV4gVR2A2EqDMjdJRPh6pQ= =8x3r -----END PGP SIGNATURE-----
Thomas Wollner <tw@wollner-net.de> writes:
Hi,
my version is from madhouse and installed via apt-get...
syslog-ng -V syslog-ng 3.3.3 Installer-Version: 3.3.3 Revision: Debian/3.3.3.dfsg-1~mhp0~squeeze Compile-Date: Nov 18 2011 15:37:57
This version should have the fix Martin is referring to. Although, I applied a different version than what's in Bazsi's tree, I think, so maybe I missed something. I'll push out a new package sometime tomorrow, if all goes well. If the bug is still present there, then it's probably a new issue. In that case, I'll have a go at it this friday, unless something unexpected comes my way. -- |8]
Gergely Nagy <algernon@balabit.hu> writes:
Thomas Wollner <tw@wollner-net.de> writes:
Hi,
my version is from madhouse and installed via apt-get...
syslog-ng -V syslog-ng 3.3.3 Installer-Version: 3.3.3 Revision: Debian/3.3.3.dfsg-1~mhp0~squeeze Compile-Date: Nov 18 2011 15:37:57
This version should have the fix Martin is referring to. Although, I applied a different version than what's in Bazsi's tree, I think, so maybe I missed something.
I'll push out a new package sometime tomorrow, if all goes well. If the bug is still present there, then it's probably a new issue. In that case, I'll have a go at it this friday, unless something unexpected comes my way.
New package (3.3.3.dfsg-1~mhp1) pushed out, based on the latest 3.3 git snapshot. I double checked by hand, and it has the rewrite fix Martin referred to. Can you try if your problem still persists? -- |8]
Hi, thank you for your time and providing new packages. I updated to the new version, but the problem still exists. No conditional rewrite at all. During installation the system complained about a unresolved dependency. syslog-ng 3.3.3 is build against libpcre >= 8.10. Debian squeeze only provides libpcre3 8.02-1.1 So, do you think this is an issue? on which system the packages for debian squeeze are build? Any backports are in the game? In general: We need syslog-ng with mongodb, patterndb and conditional rewrite support. Do you - or others - have a recommendation which version to use? BTW: mojology is a very nice peace of software. I like it very much. Are there any plans to go on further with that? Things like Message search, etc? best regards, Tom Zitat von Gergely Nagy <algernon@balabit.hu>:
Gergely Nagy <algernon@balabit.hu> writes:
Thomas Wollner <tw@wollner-net.de> writes:
Hi,
my version is from madhouse and installed via apt-get...
syslog-ng -V syslog-ng 3.3.3 Installer-Version: 3.3.3 Revision: Debian/3.3.3.dfsg-1~mhp0~squeeze Compile-Date: Nov 18 2011 15:37:57
This version should have the fix Martin is referring to. Although, I applied a different version than what's in Bazsi's tree, I think, so maybe I missed something.
I'll push out a new package sometime tomorrow, if all goes well. If the bug is still present there, then it's probably a new issue. In that case, I'll have a go at it this friday, unless something unexpected comes my way.
New package (3.3.3.dfsg-1~mhp1) pushed out, based on the latest 3.3 git snapshot. I double checked by hand, and it has the rewrite fix Martin referred to.
Can you try if your problem still persists?
-- |8]
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.
Thomas Wollner <tw@wollner-net.de> writes:
I updated to the new version, but the problem still exists. No conditional rewrite at all. During installation the system complained about a unresolved dependency. syslog-ng 3.3.3 is build against libpcre >= 8.10. Debian squeeze only provides libpcre3 8.02-1.1
Hrm. Then that's a bug in my build environment, I'll try to fix it ASAP. I might have screwed up by creating a sid chroot instead of a squeeze one.. As soon as I figured that out, I'll push a new version.
So, do you think this is an issue? on which system the packages for debian squeeze are build? Any backports are in the game?
It looks like a bug on my end.
In general: We need syslog-ng with mongodb, patterndb and conditional rewrite support. Do you - or others - have a recommendation which version to use?
My packages are your best bet, I believe. As soon as the conditional rewrite is fixed, that is. This should happen soon, as that's the highest on my TODO list at the moment.
BTW: mojology is a very nice peace of software. I like it very much. Are there any plans to go on further with that? Things like Message search, etc?
With mojology, no, not really. Mojology is a kind of proof of concept, and it works fairly well for that, but adding anything more complex to is non-trivial to say the least. I do have something in the works that will hopefully be interesting (yes, it includes message search ;). But it's in a very early stage, and will take a good few months before it's ready to be made public. -- |8]
Thomas Wollner <tw@wollner-net.de> writes:
I updated to the new version, but the problem still exists. No conditional rewrite at all. During installation the system complained about a unresolved dependency. syslog-ng 3.3.3 is build against libpcre >= 8.10. Debian squeeze only provides libpcre3 8.02-1.1
Hrm, are you sure you're using the appropriate apt line? I just checked, and my squeeze packages have libpcre3 (>= 7.7) dependencies (the unstable ones do have >= 8.10). -- |8]
Hi, sorry, my fault. got the unstable line in addition in the apt.sources. commented out, reinstalled - no dependency conflicts this time - but conditional rewrite wont work. can you reproduce the error? or do you have a working example for conditional rewrites? best regards, Tom aptitude install syslog-ng The following NEW packages will be installed: libjson0{a} libmongo-client0{a} syslog-ng syslog-ng-mod-json{a} syslog-ng-mod-mongodb{a} syslog-ng-mod-sql{ab} The following packages will be upgraded: syslog-ng-core{b} 1 packages upgraded, 6 newly installed, 0 to remove and 1 not upgraded. Need to get 277 kB/356 kB of archives. After unpacking 463 kB will be used. The following packages have unmet dependencies: syslog-ng-core: Depends: libpcre3 (>= 8.10) but 8.02-1.1 is installed. Depends: libssl1.0.0 (>= 1.0.0) which is a virtual package. syslog-ng-mod-sql: Depends: libdbi1 (>= 0.8.4) which is a virtual package. Depends: libssl1.0.0 (>= 1.0.0) which is a virtual package. Zitat von Gergely Nagy <algernon@balabit.hu>:
Thomas Wollner <tw@wollner-net.de> writes:
I updated to the new version, but the problem still exists. No conditional rewrite at all. During installation the system complained about a unresolved dependency. syslog-ng 3.3.3 is build against libpcre >= 8.10. Debian squeeze only provides libpcre3 8.02-1.1
Hrm, are you sure you're using the appropriate apt line?
I just checked, and my squeeze packages have libpcre3 (>= 7.7) dependencies (the unstable ones do have >= 8.10).
-- |8]
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.
Thomas Wollner <tw@wollner-net.de> writes:
can you reproduce the error? or do you have a working example for conditional rewrites?
Didn't get that far yet, will see in about half an hour or so.
aptitude install syslog-ng The following NEW packages will be installed: libjson0{a} libmongo-client0{a} syslog-ng syslog-ng-mod-json{a} syslog-ng-mod-mongodb{a} syslog-ng-mod-sql{ab} The following packages will be upgraded: syslog-ng-core{b} 1 packages upgraded, 6 newly installed, 0 to remove and 1 not upgraded. Need to get 277 kB/356 kB of archives. After unpacking 463 kB will be used. The following packages have unmet dependencies: syslog-ng-core: Depends: libpcre3 (>= 8.10) but 8.02-1.1 is installed. Depends: libssl1.0.0 (>= 1.0.0) which is a virtual package. syslog-ng-mod-sql: Depends: libdbi1 (>= 0.8.4) which is a virtual package. Depends: libssl1.0.0 (>= 1.0.0) which is a virtual package.
That's still trying to pull syslog-ng-core from the unstable version, by the looks of it. -- |8]
Gergely Nagy <algernon@balabit.hu> writes:
Thomas Wollner <tw@wollner-net.de> writes:
can you reproduce the error? or do you have a working example for conditional rewrites?
Didn't get that far yet, will see in about half an hour or so.
Yep, reproduced. filter in itself catches it nicely, rewrite fails: Incoming log entry; line='217122: Nov 30 17:23:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down' # conditional rewrite rules Filter node evaluation result; filter_result='not-match', filter_type='filter(f_rewrite_cisco_program)' Rewrite condition unmatched, skipping rewrite; value='PROGRAM' Rewrite expression evaluation result; value='PROGRAM', new_value='' Filter node evaluation result; filter_result='not-match', filter_type='filter(f_rewrite_cisco_program)' Rewrite condition unmatched, skipping rewrite; value='MESSAGE' Rewrite expression evaluation result; value='MESSAGE', new_value='217122: Nov 30 17:23:49: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/1, changed state to down' # filter in itself Filter rule evaluation begins; filter_rule='f_rewrite_cisco_program' Filter node evaluation result; filter_result='match' Filter rule evaluation result; filter_result='match', filter_rule='f_rewrite_cisco_program' -- |8]
Gergely Nagy <algernon@balabit.hu> writes:
Gergely Nagy <algernon@balabit.hu> writes:
Thomas Wollner <tw@wollner-net.de> writes:
can you reproduce the error? or do you have a working example for conditional rewrites?
Didn't get that far yet, will see in about half an hour or so.
Yep, reproduced. filter in itself catches it nicely, rewrite fails:
And I have a suspicion where the problem lies. With a bit of luck, I'll have a solution by tomorrow. Thanks for the report! -- |8]
Gergely Nagy <algernon@balabit.hu> writes:
can you reproduce the error? or do you have a working example for conditional rewrites?
Didn't get that far yet, will see in about half an hour or so.
Yep, reproduced. filter in itself catches it nicely, rewrite fails:
And I have a suspicion where the problem lies. With a bit of luck, I'll have a solution by tomorrow.
While I don't yet have a solution, I know where the problem is, and am working on a fix. For the time being, I can offer a workaround: if you inline the condition, instead of using filter() inside the condition, that will work: rewrite r_cisco_program_inline { set("$1", value("PROGRAM"), condition( match('%([^:]+):\s+([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")) )); set("$2", value("MESSAGE"), condition( match('%([^:]+):\s+([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")) )); }; This is inconvenient, slow and ugly and in the long term, unmaintainable, but works until I prepare a correct fix for the condition(filter(foo)) case. -- |8]
Hi, yes, the workaround works for me. Thank you very much! Hope you get the cause of quickly. Thanks again for your time! best regards, Tom Zitat von Gergely Nagy <algernon@balabit.hu>:
Gergely Nagy <algernon@balabit.hu> writes:
can you reproduce the error? or do you have a working example for conditional rewrites?
Didn't get that far yet, will see in about half an hour or so.
Yep, reproduced. filter in itself catches it nicely, rewrite fails:
And I have a suspicion where the problem lies. With a bit of luck, I'll have a solution by tomorrow.
While I don't yet have a solution, I know where the problem is, and am working on a fix.
For the time being, I can offer a workaround: if you inline the condition, instead of using filter() inside the condition, that will work:
rewrite r_cisco_program_inline { set("$1", value("PROGRAM"), condition( match('%([^:]+):\s+([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")) )); set("$2", value("MESSAGE"), condition( match('%([^:]+):\s+([^\n]+)' value("MESSAGE") type("pcre") flags("store-matches" "nobackref")) )); };
This is inconvenient, slow and ugly and in the long term, unmaintainable, but works until I prepare a correct fix for the condition(filter(foo)) case.
-- |8]
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.
Thomas Wollner <tw@wollner-net.de> writes:
yes, the workaround works for me. Thank you very much! Hope you get the cause of quickly.
Took a few iterations, but I just sent a patch to the list that fixes the underlying issue. I won't apply it to the Debian packages yet, because I have to double check that it doesn't introduce a memory leak (I don't believe it does, but I want to be extra careful around those parts of the code that I touch for the first time). -- |8]
participants (4)
-
Gergely Nagy
-
Martin Holste
-
Patrick H.
-
Thomas Wollner