[syslog-ng] Messages starting with tab not processed cleanly

[Gergely Nagy]

Jon Nathan <jon+dated+1316554556.86f3d9 at rupture.net> writes:

> Client syslog:
>
> Sep 15 20:30:21 h.example.com scsi: [ID 107833 kern.warning] WARNING: /pci at 7b,0/pci1022,7458 at 11/pci1000,3060 at 2/sd at 0,0 (sd2):
> Sep 15 20:30:21 h.example.com         Error for Command: read                    Error Level: Fatal

This has a hostname because the message is coming from /dev/log (or
whatever its Solaris equivalent is), where only the message part
appears, the date and host is appended by the syslog daemon.

> This shows up in my centralized syslog-ng server as:
>
> Sep 15 20:30:21 h.example.com scsi: [ID 107833 kern.warning] WARNING: /pci at 7b,0/pci1022,7458 at 11/pci1000,3060 at 2/sd at 0,0 (sd2):
> Sep 15 20:30:21         Error for Command: read                    Error Level: Fatal

This does not have a host header (it actually does, see below!),
because, if you look at the message as it comes through the net:

> 20:30:21.454941 IP (tos 0x0, ttl 255, id 43671, offset 0, flags [DF], proto UDP (17), length 109)
>     10.1.1.1.59299 > 10.1.1.2.514: SYSLOG, length: 81
>         Facility kernel (0), Severity warning (4)
>         Msg: Sep 15 20:30:21 \0x09Error for Command: read                    Error Level: Fatal

The first word after the date in a BSD syslog message will be the host,
thus, syslog-ng will treat \0x09Error as the hostname, as that's what's
coming through the wire. For added fun, $PROGRAM will be set to "for"
aswell.

If you turn off keep-hostname(), which I assume you have on at your
source (otherwise the \0x09Error would get replaced), you'll see that it
will disappear, and get replaced by a hostname.

The best course of action I can think of, is using a filter + rewrite
combination, something like the following:

filter f_tabhost { host("\t*" type(glob)); };

rewrite r_rewrite_taberror {
  set("$HOST $PROGRAM $MSG", value("MSG"));
  set("$HOST_FROM", value("HOST"));
  set("-", value("PROGRAM"));
};

log {
 source(...);
 filter(f_tabhost);
 rewrite(r_rewrite_taberror);
 destination(d_syslog);
 flags(final);
};

log {
 source(...);
 destination(d_syslog);
};

This will set the message part appropriately, and set $HOST to
$HOST_FROM (which is the originating host's name, as looked up via DNS),
and $PROGRAM to a default value of "-".

Provided that the sending hosts' reverse dns is the same as the host it
sends in syslog messages, the above blocks should do the right thing.

(I'm not sure whether all of this is supported in 3.1, I suppose it
is. I only tried with 3.3, as that's what I have at hand, but according
to the documentation, this should work with 3.2 aswell)

-- 
|8]