[syslog-ng] Transparent TCP forward: what am I doing wrong?

[JP Vossen]

On 06/30/2011 04:47 AM, Sandor Geller wrote:
<snip>
> syslog-ng could beoverkill for such a purpose, writing a few lines of
> perl code could be easier...

True, but I'm already using syslog-ng and the fewer things I have to 
build, deploy and maintain the better.


>> If I send the messages into syslog-ng or rsyslog, I get the correct data
>> with no "extra" fields in the log file.  But unfortunately, I need to
>> send messages into another proprietary syslog listener, and in that one
>> I'm getting the extra fields.  I'd blame the whole mess on that, except
>> that when I do a packet dump I do see the 2 extra fields in there.  So I
>> *think* that syslog-ng and rsyslog are smart enough to handle them and
>> my other thing isn't.
>
> syslog-ng adds the timestamp and hostname fields by default. Many
> people think that syslog daemons should accept random junk, but the
> syslog daemon shouldn't be treated like a plain transport mechanism.
> There are RFCs describing the syslog formats and the daemons should
> enforce the standards.

Very true!!!  Unfortunately, the environments I have to handle do have 
quite a lot of random junk that pretends to be "syslog" and I have to 
deal with it.  That's partly why I was using that cruddy netcat command, 
it's actuality quite a good test case for my needs.  :-/

But if *everything* was RFC-compliant my (and I suspect a lot of folks 
on this list) life would be a lot easier. :-)


>> As I said, my goal is to receive TCP/514 and **transparently** forward
>> logs with no changes, as if they came in via UDP, to the localhost via
>> UDP/514.  In other words, I'm using syslog-ng as a shim to feed syslog
>> over TCP to a listener which only listens on UDP.
>>
>> Why doesn't it work?  What totally obvious thing am I missing?  Am I
>> doing anything else dumb?
>
> Your incoming logs aren't properly formatted syslog messages so
> syslog-ng has to guess which fields are present and which are missing
> and adds the required fields. If you don't want this then tell
> syslog-ng to don't parse the logs and then just use a custom template,
> something like this:
>
> source s_tcpincoming {
>      tcp(
>          ip(0.0.0.0) port(514) max_connections(1000)
>          keep_timestamp(no)
>          keep_hostname(no)
>          flags(no-parse)
>      );
> };
> destination d_fe_tcp {
>      udp(
>          "127.0.0.2" port(514)
>          spoof_source(yes)
>          template("${MESSAGE}\n")
>      );
> };
<snip>
> hth,
> Sandor

Yes!  Thank you!  The no-parse and template lines did the trick.

Thanks also to Gergely Nagy for the same solution.  This is part of why 
I love open source, random people all over the world are just so 
helpful.  Thanks also to Jose Pedro Oliveira and others on the list for 
lots of help with other (build on CentOS-5) issues.

Later,
JP
----------------------------|:::======|-------------------------------
JP Vossen, CISSP            |:::======|      http://bashcookbook.com/
My Account, My Opinions     |=========|      http://www.jpsdomain.org/
----------------------------|=========|-------------------------------
"Microsoft Tax" = the additional hardware & yearly fees for the add-on
software required to protect Windows from its own poorly designed and
implemented self, while the overhead incidentally flattens Moore's Law.