[syslog-ng] snmptrapd and HOST macro mapping

[joël Winteregg]

Hello Balazs,

Thank you very much for you quick reply !
The csv-xxx functionality looks really great. You're doing a really good
job ! Thanks ! Indeed, today, I was looking for such a functionality
into the doc ;-)

I think I will try your proposal as soon as possible and I will let you
know how it works in my use case.

Juste to know, does syslog-ng only use relay config statements
(keep_hostname, etc.) when the log source is defined as udp() or tcp() ?
I'm asking this, because I'm wondering if I forward my SNMP trap to
syslogd and then to syslog-ng through udp (@SYSLOG-COLLECTOR defined in
syslog.conf), syslog-ng will maybe see the SNMP trap as a compliant RFC
3164 forwarded message ?

Thanks again and best regards,


Joël

On Wed, 2008-11-12 at 22:48 +0100, Balazs Scheidler wrote:
> On Wed, 2008-11-12 at 17:51 +0100, joël Winteregg wrote:
> > Dear Syslog-ng users,
> > 
> > 
> > I'm trying to understand (using syslog-ng community source code and some
> > testing) if it is possible to map a given hostname located into MSGONLY
> > macro to the FULLHOST or HOST macro. I'm trying to achieve this in order
> > to fetch the hostname (SNMP trap source) provided by my snmptrapd
> > collector (which is located on the same server as syslog-ng).
> > 
> > The snmptrapd collector is logging its trap using syslog API as follow
> > (snmptrapd source code):
> > 
> > int log_handler_syslog(  netsnmp_log_handler* logh, int pri, const char
> > *str){
> >  /*
> >   * XXX
> >   * We've got three items of information to work with:
> >   *     Is the syslog currently open?
> >   *     What ident string to use?
> >   *     What facility to log to?
> >   *
> >   * We've got two "magic" locations (imagic & magic) plus the token
> >   */
> >      if (!(logh->imagic)) {
> >         const char *ident    = logh->token;
> >         int   facility = (int)(intptr_t)logh->magic;
> >         if (!ident)
> >             ident = netsnmp_ds_get_string(NETSNMP_DS_LIBRARY_ID, NETSNMP_DS_LIB_APPTYPE);
> >         openlog(ident, LOG_CONS | LOG_PID, facility);
> >         logh->imagic = 1;
> >     }
> >     syslog( pri, "%s", str );
> >     return 1;
> > }
> > 
> > As provided by syslog.h, there is no way to set the hostname used into syslog message (it is automatically set by the API).
> > So my logs, provided by snmptrapd, look like the following (where HOST macro is related to the server hosting syslog-ng and snmptrapd):
> > 
> > 12/11/2008 16:57:14 SYSLOG-COLLECTOR LEVEL=info snmptrapd[29592]: wlc02.mydomain.com Cold Start
> > 
> > In order to be able to get wlc02.mydomain.com as $HOST or $FULLHOST macro, I set:
> > chain_hostnames(yes);
> > keep_hostname(yes);
> > 
> > and I formated my snmptrapd logs MSGONLY content (using snmptrad config statements) as RFC 3164 need it:
> > 
> > 12/11/2008 16:58:00 SYSLOG-COLLECTOR LEVEL=info snmptrapd[29592]: Nov 12 16:57:59 wlc02.mydomain.com Cold Start
> > 
> > I thought that the above log would be seen as a relayed message by syslog-ng allowing me to fetch wlc02.mydomain.com into $HOST or $FULLHOST.
> > Unfortunately this config provide me the following string into $FULLHOST macro: s_source at SYSLOG-COLLECTOR 
> > 
> > Does syslog-ng only use relay config statements (keep_hostname, etc.)  when the log source is defined as udp() or tcp() ?
> > 
> > I saw another hint about this here: http://osdir.com/ml/syslog-ng/2004-10/msg00062.html It says that:
> > "If you still have your snmptrapd hostname, then you have to modify snmptrapd to use the original hostname when it sends the message about the trap"
> > 
> > But as given above (syslog API usage into snmptrapd) how can I set the hostname of the syslog message ?
> > 
> > 
> > Any hint is welcome !
> 
> Well, one way is to patch snmptrapd to avoid using the syslog() API and
> go to syslog-ng directly _OR_ you can try the latest 3.0 devel snapshot,
> which has support for rewrite rules.
> 
> You could do something like this with 3.0:
> 
> filter f_snmptrapd { program("snmptrapd"); };
> 
> # this parser assumes this snmptrapd format
> # <host> "<message payload>"
> parser p_snmptrapd { 
> 	csv-parser(columns("SNMPTRAP.HOST", "SNMPTRAP.MSG") flags(escape-backslash, strip-whitespace));
> };
> 
> rewrite r_snmptrapd {
> 	set("${SNMPTRAP.HOST}" value("HOST"));
> 	set("${SNMPTRAP.MSG}" value("MESSAGE"));
> }
> 
> log { source(...); filter(f_snmptrapd); parser(p_snmptrapd); rewrite(r_snmptrapd); destination(...); };
> 
> As it seems it would be useful to add a new flag to csv-parser(): last-column-greedy, in which case 
> any non-processed input would be added to the last column.
> 
> If you are willing to try if this works in your setup, I'm willing to invest the time to 
> implement that csv parser option. If that would be done, it would be easier to process
> even the original input.
> 
> -- 
> Bazsi
> 
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>