[syslog-ng] Can TAG field be terminated by space?

Evan Rempel erempel at uvic.ca
Sat Dec 29 19:00:09 CET 2007


> On Tue, 2007-10-23 at 18:50 +0200, Balazs Scheidler wrote:
>> On Mon, 2007-10-22 at 18:26 +0900, Tsurusawa Takeshi wrote:
>>> Hi,
>>>
>>> I'm using syslog-ng-2.0.5 and trying to store syslog messages into a DB table with
>>>  a TAG field column.
>>>
>>> But when TAG fileld is terminated with space character(' ') in syslog message,
>>> syslog-ng parser returns as TAG field not only TAG field but also some extra strings
>>> end with a colon(':') in CONTENT field.
>>>
>>> For example,  if MSG part is "program abc: message...", syslog-ng returns
>>> "program abc" as a TAG field.
>>>
>>> According to the section 4.1.3 of RFC3164, a space character can also terminate a TAG field.
>>>
>>>    The TAG is a string of
>>>    ABNF alphanumeric characters that MUST NOT exceed 32 characters.  Any
>>>    non-alphanumeric character will terminate the TAG field and will be
>>>    assumed to be the starting character of the CONTENT field.  Most
>>>    commonly, the first character of the CONTENT field that signifies the
>>>    conclusion of the TAG field has been seen to be the left square
>>>    bracket character ("["), a colon character (":"), or a space
>>>    character.
>>>
>>> Is it possible to change this behavior of syslog-ng?
>>
>> I'm usually reluctant to make such changes as it is very easy to create
>> regressions when changing the log parsing code.
>>
>> However there were two similar cases in the message parsing code that
>> used different TAG terminator characters, thus I unified the two. (one
>> was using space as separator).
>>
>> Here's the patch:
>>
>> http://git.balabit.hu/?p=bazsi/syslog-ng-2.0.git;a=commit;h=4a84d904fe0fc5b3627773bafe6966ad9037386d
>>
>> tomorrow's snapshot should also contain the change.
>>
>
> As I feared somewhat this change caused a regression for programs that
> intentionally use "/" in the name of the program. Such an example is
> "postfix"
>
> I've changed this patch, a program name is terminated by either of the
> following characters: space, '[' and ':'. This still achieves what you
> originally wanted, but still does not match the RFC as that causes
> trouble.
>
> The two code paths using different characters were also unified to use
> this three characters.

Wow, this is a can of worms.

If I use a template of

$PRI $DATE $HOST $FACILITY.$PRIORITY $PROGRAM: $MSGONLY

it should recreate the entire syslog message, but it will not.
The information inside of the [xxx] of the program will be
dropped (or will it be part of the MSGONLY?

If you want to exclude the [xxx] from the PROGRAM macro then I think
that a new macro is required that will contain the [xxx] component.
Perhaps INSTANCE or IDENTIFIER or UNIQUE. I don't have the RFC
in front of me, but using the terminology that the RFC uses
would be good. The I can write a template of

$PRI $DATE $HOST $FACILITY.$PRIORITY $PROGRAM[$INSTANCE]: $MSGONLY

to recreate the syslog record.

An what about if the INSTANCE is not present in the record....
Perhaps we need a conditional template? Currently two
destinations with the same endpoint, that use different templates
and different filters can be used to accomplish this, but it
gets convoluted very quickly.

Evan.


More information about the syslog-ng mailing list