Syslog-ng/Rsyslog "message" differences (?)
Hello all! I've got a bit of a strange problem. I've been developing software (Shameless plug: http://sagan.softwink.com) that does log analysis. This software is mostly used with Syslog-ng and Rsyslog. We recently started testing some log normalization, and I'm seeing different results from syslog-ng's $MSG string and rsyslog %msg%. For example: template("$MSG\n") template-escape(no)); }; --------------- Syslog-ng: template("$MSG\n") template-escape(no)); }; Output: sshd[20657]: Invalid user champtest from 66.177.167.194 --------------- Rsyslog: $template sagan, "%msg%\n" Output: Invalid user champtest from 66.177.167.194 --------------- Whitespacing aside, with syslog-ng I get the program information within the message field. I should point out that most of the systems in the network are Syslog-ng and reporting to Rsyslog (which I can switch out with Syslog-ng for testing). I guess my questions are: 1. Does this sound like a Syslog-ng/Rsyslog interoperability issue? 2. Or do the two just see the "message" formats differently? 3. Maybe it's just my setup (syslog-ng/rsyslog versions)? I have a "work around" with Rsyslog, but was wondering if there where any thoughts on this issue? Thanks -- Champ Clark III | Softwink, Inc | 800-538-9357 x 101 http://www.softwink.com GPG Key ID: 58A2A58F Key fingerprint = 7734 2A1C 007D 581E BDF7 6AD5 0F1F 655F 58A2 A58F If it wasn't for C, we'd be using BASI, PASAL and OBOL.
On Thu, 2011-01-13 at 12:54 -0500, Champ Clark III [Softwink] wrote:
Hello all!
I've got a bit of a strange problem. I've been developing software (Shameless plug: http://sagan.softwink.com) that does log analysis. This software is mostly used with Syslog-ng and Rsyslog. We recently started testing some log normalization, and I'm seeing different results from syslog-ng's $MSG string and rsyslog %msg%. For example:
template("$MSG\n") template-escape(no)); };
--------------- Syslog-ng: template("$MSG\n") template-escape(no)); };
Output: sshd[20657]: Invalid user champtest from 66.177.167.194
--------------- Rsyslog:
$template sagan, "%msg%\n"
Output: Invalid user champtest from 66.177.167.194 ---------------
You seem to want only the message part of the log entry. With syslog-ng, $MSG contains the program too, and if you want the message part only, you should try $MSGONLY in the template. That should do just what you want. By the way, there's excellent documentation about syslog-ng macros, which explains the difference between $MSG and $MSGONLY at http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.2-guid... -- |8]
You seem to want only the message part of the log entry. With syslog-ng, $MSG contains the program too, and if you want the message part only, you should try $MSGONLY in the template. That should do just what you want.
By the way, there's excellent documentation about syslog-ng macros, which explains the difference between $MSG and $MSGONLY at http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.2-guid...
Thank you for the reply. I thought I'd RTFM'ed, but obviously not enough. It sounds like $MSGONLY is exactly what I need. -- Champ Clark III | Softwink, Inc | 800-538-9357 x 101 http://www.softwink.com GPG Key ID: 58A2A58F Key fingerprint = 7734 2A1C 007D 581E BDF7 6AD5 0F1F 655F 58A2 A58F If it wasn't for C, we'd be using BASI, PASAL and OBOL.
On Thu, 2011-01-13 at 13:06 -0500, Champ Clark III [Softwink] wrote:
You seem to want only the message part of the log entry. With syslog-ng, $MSG contains the program too, and if you want the message part only, you should try $MSGONLY in the template. That should do just what you want.
By the way, there's excellent documentation about syslog-ng macros, which explains the difference between $MSG and $MSGONLY at http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.2-guid...
Thank you for the reply. I thought I'd RTFM'ed, but obviously not enough. It sounds like $MSGONLY is exactly what I need.
On second read, $MSG's behaviour seems to be dependent upon what version of syslog-ng you're using. According to the documentation, syslog-ng prior to 3.0 had the program name and the pid in $MSG, in 3.0 and past that, $MSG is synonymous to $MSGONLY. I'm not sure whether <3.0 contains $MSGONLY, but if they do, then all is well, and one can use the same macro for all versions. -- |8]
On second read, $MSG's behaviour seems to be dependent upon what version of syslog-ng you're using. According to the documentation, syslog-ng prior to 3.0 had the program name and the pid in $MSG, in 3.0 and past that, $MSG is synonymous to $MSGONLY.
I'm not sure whether <3.0 contains $MSGONLY, but if they do, then all is well, and one can use the same macro for all versions.
Hopefully $MSGONLY is backward compatible with in < 3.0. I'll have to test that. I'll also make a notation in the documentation about this as well. Thanks for the input. -- Champ Clark III | Softwink, Inc | 800-538-9357 x 101 http://www.softwink.com GPG Key ID: 58A2A58F Key fingerprint = 7734 2A1C 007D 581E BDF7 6AD5 0F1F 655F 58A2 A58F If it wasn't for C, we'd be using BASI, PASAL and OBOL.
On Thu, 2011-01-13 at 13:22 -0500, Champ Clark III [Softwink] wrote:
On second read, $MSG's behaviour seems to be dependent upon what version of syslog-ng you're using. According to the documentation, syslog-ng prior to 3.0 had the program name and the pid in $MSG, in 3.0 and past that, $MSG is synonymous to $MSGONLY.
I'm not sure whether <3.0 contains $MSGONLY, but if they do, then all is well, and one can use the same macro for all versions.
Hopefully $MSGONLY is backward compatible with in < 3.0. I'll have to test that. I'll also make a notation in the documentation about this as well. Thanks for the input.
yes, MSGONLY has been present in syslog-ng for a while, definitely introduced earlier than 3.0. -- Bazsi
participants (3)
-
Balazs Scheidler
-
Champ Clark III [Softwink]
-
Gergely Nagy