assigning structured data from IETF syslog messages to macros
hi all, I have run into an issue when trying to process structured data from IETF syslog messages. According to syslog-ng administrators guide (I am using syslog-ng 3.0.7), the SDATA.SDID.SDNAME macro can be used for referring structured data. However, when I tried to test the example from the administrator guide, I found that only the $SDATA macro works fine. The example is somewhat confusing, since the SDATA.EXAMPLE.EVENTSOURCE macro allegedly points to the SD ID "exampleSDID@0" (which contains the name eventSource). In order to test syslog-ng with a clearer case, I created the following messages with nc: echo -n '<165>1 2010-06-09T18:00:00.000000+02:00 192.168.1.1 myproc - - [timeQuality tzKnown="0" isSynced="0"] This is a test' | nc -w 1 -u localhost 8514 The syslog-ng configuration looks like this: template format { template("$DATE $HOST [$FACILITY.$LEVEL] $MSG StructuredData = ${SDATA.TIMEQUALITY.TZKNOWN}\n"); }; #template format { template("$DATE $HOST [$FACILITY.$LEVEL] $MSG StructuredData = ${SDATA}\n"); }; source ietfmsg { syslog(ip(0.0.0.0) port(8514) transport(udp)); }; destination netlog { file("/var/log/syslog-ng.log" template(format)); }; log { source(ietfmsg); destination(netlog); }; If the second template definition is active (i.e., the first definition is commented out), everything works fine and I have the following messages in the log: Jun 9 18:00:00 localhost [local4.notice] This is a test StructuredData = [timeQuality tzKnown="0" isSynced="0"] However, when the first template definition is activated, I will get the following output from syslog-ng: Incoming log entry; line='<165>1 2010-06-09T18:00:00.000000+02:00 192.168.1.1 myproc - - [timeQuality tzKnown="0" isSynced="0"] This is a test' No such value known; value='SDATA.TIMEQUALITY.TZKNOWN' Also, the log messages look like this: Jun 9 18:00:00 localhost [local4.notice] This is a test StructuredData = What am I doing wrong? The example from the manual does not seem to work, neither does my simple test case. kind regards, risto
Hi, The SDATA parameters are case sensitives and they start with a dot ".", like this: .SDATA.timeQuality.tzKnown=0 .SDATA.timeQuality.isSynced=0 template format { template("$DATE $HOST [$FACILITY.$LEVEL] $MSG StructuredData = ${.SDATA.timeQuality.tzKnown}\n"); }; Though it might make sense to make them case insesitive. BR, M PS: I have also attached a patch which adds an $ALLVAL macro which you can use to debug such value related problems. The patch is dirty, the macro name is also, but works. :) On Thu, 2010-06-10 at 11:14 +0300, Risto Vaarandi wrote:
hi all,
I have run into an issue when trying to process structured data from IETF syslog messages. According to syslog-ng administrators guide (I am using syslog-ng 3.0.7), the SDATA.SDID.SDNAME macro can be used for referring structured data.
However, when I tried to test the example from the administrator guide, I found that only the $SDATA macro works fine. The example is somewhat confusing, since the SDATA.EXAMPLE.EVENTSOURCE macro allegedly points to the SD ID "exampleSDID@0" (which contains the name eventSource).
In order to test syslog-ng with a clearer case, I created the following messages with nc:
echo -n '<165>1 2010-06-09T18:00:00.000000+02:00 192.168.1.1 myproc - - [timeQuality tzKnown="0" isSynced="0"] This is a test' | nc -w 1 -u localhost 8514
The syslog-ng configuration looks like this:
template format { template("$DATE $HOST [$FACILITY.$LEVEL] $MSG StructuredData = ${SDATA.TIMEQUALITY.TZKNOWN}\n"); };
#template format { template("$DATE $HOST [$FACILITY.$LEVEL] $MSG StructuredData = ${SDATA}\n"); };
source ietfmsg { syslog(ip(0.0.0.0) port(8514) transport(udp)); }; destination netlog { file("/var/log/syslog-ng.log" template(format)); }; log { source(ietfmsg); destination(netlog); };
If the second template definition is active (i.e., the first definition is commented out), everything works fine and I have the following messages in the log: Jun 9 18:00:00 localhost [local4.notice] This is a test StructuredData = [timeQuality tzKnown="0" isSynced="0"]
However, when the first template definition is activated, I will get the following output from syslog-ng: Incoming log entry; line='<165>1 2010-06-09T18:00:00.000000+02:00 192.168.1.1 myproc - - [timeQuality tzKnown="0" isSynced="0"] This is a test' No such value known; value='SDATA.TIMEQUALITY.TZKNOWN'
Also, the log messages look like this: Jun 9 18:00:00 localhost [local4.notice] This is a test StructuredData =
What am I doing wrong? The example from the manual does not seem to work, neither does my simple test case.
kind regards, risto ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
-- Key fingerprint = F78C 25CA 5F88 6FAF EA21 779D 3279 9F9E 1155 670D
Marton, thanks for the info, everything is working fine now. BTW, can the syslog-ng admin guide be updated with the correct information? regards, risto On 06/11/2010 05:43 PM, ILLES, Marton wrote:
Hi,
The SDATA parameters are case sensitives and they start with a dot ".", like this:
.SDATA.timeQuality.tzKnown=0 .SDATA.timeQuality.isSynced=0
template format { template("$DATE $HOST [$FACILITY.$LEVEL] $MSG StructuredData = ${.SDATA.timeQuality.tzKnown}\n"); };
Though it might make sense to make them case insesitive.
BR,
M
PS: I have also attached a patch which adds an $ALLVAL macro which you can use to debug such value related problems. The patch is dirty, the macro name is also, but works. :)
On Thu, 2010-06-10 at 11:14 +0300, Risto Vaarandi wrote:
hi all,
I have run into an issue when trying to process structured data from IETF syslog messages. According to syslog-ng administrators guide (I am using syslog-ng 3.0.7), the SDATA.SDID.SDNAME macro can be used for referring structured data.
However, when I tried to test the example from the administrator guide, I found that only the $SDATA macro works fine. The example is somewhat confusing, since the SDATA.EXAMPLE.EVENTSOURCE macro allegedly points to the SD ID "exampleSDID@0" (which contains the name eventSource).
In order to test syslog-ng with a clearer case, I created the following messages with nc:
echo -n '<165>1 2010-06-09T18:00:00.000000+02:00 192.168.1.1 myproc - - [timeQuality tzKnown="0" isSynced="0"] This is a test' | nc -w 1 -u localhost 8514
The syslog-ng configuration looks like this:
template format { template("$DATE $HOST [$FACILITY.$LEVEL] $MSG StructuredData = ${SDATA.TIMEQUALITY.TZKNOWN}\n"); };
#template format { template("$DATE $HOST [$FACILITY.$LEVEL] $MSG StructuredData = ${SDATA}\n"); };
source ietfmsg { syslog(ip(0.0.0.0) port(8514) transport(udp)); }; destination netlog { file("/var/log/syslog-ng.log" template(format)); }; log { source(ietfmsg); destination(netlog); };
If the second template definition is active (i.e., the first definition is commented out), everything works fine and I have the following messages in the log: Jun 9 18:00:00 localhost [local4.notice] This is a test StructuredData = [timeQuality tzKnown="0" isSynced="0"]
However, when the first template definition is activated, I will get the following output from syslog-ng: Incoming log entry; line='<165>1 2010-06-09T18:00:00.000000+02:00 192.168.1.1 myproc - - [timeQuality tzKnown="0" isSynced="0"] This is a test' No such value known; value='SDATA.TIMEQUALITY.TZKNOWN'
Also, the log messages look like this: Jun 9 18:00:00 localhost [local4.notice] This is a test StructuredData =
What am I doing wrong? The example from the manual does not seem to work, neither does my simple test case.
kind regards, risto ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
participants (2)
-
ILLES, Marton
-
Risto Vaarandi