It works like a charm. Thank you Matt! Andika Daud  |  Sr. Web Technologist  |  Adobe  |  p. 408.536.4713  | adaud@adobe.com -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Matt Zagrabelny Sent: Tuesday, February 21, 2012 10:18 AM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Extract portion of SDATA file name? On Tue, Feb 21, 2012 at 11:45 AM, Andika Daud <adaud@adobe.com> wrote:

Hi,

I'm a noobie and just about beginning to try out syslog-ng at my company.  I have the following configuration on the client:

source s_apache {

        file("/apps/log/apache/store-dev03/*"

        program_override("apache")

        flags(no-parse)

        );

};

destination d_remote {

    syslog("10.5.76.125" transport("tcp") port(514)

#    template("${MSGONLY}")

    );

};

log {

        source(s_apache);

        destination(d_remote);

};

And something like this on the server side:

source s_network {

   syslog(ip(0.0.0.0)

        port(514) transport("tcp") );

};

destination d_local {

    file("/apps/log/syslong-ng/$PROGRAM/$HOST/$YEAR$MONTH$DAY.log"

    # ignore the use of template below, I just wanted to prove the server side could

    # see the value of the filepath from the client

    template("${.SDATA.file@18372.4.name} - ${MSGONLY}\n")

    );

};

log {

   source(s_network);

   destination(d_local);

};

My ("${.SDATA.file@18372.4.name} value can be something like this: '/apps/log/apache/store-dev03/access.log'. What I want to be able to do is to parse that path.  That is to discard the begining of the path, /apps/log/apache.  Get the rest of the value, especially the file name (access.log) to be used to construct destination path.

Is this possible?  Thank you for your kind help.

Hi Andika, I'm not a syslog-ng expert, but I believe I've done what you are seeking to do. Here is a chunk of syslog-ng configs that I use for what you describe: ---{begin}--- # Send the following apache log files to bulldog so that people with shell access can examine # log messages. source s_apache_logs { file("/var/log/apache2/access.log" bulldog flags(no-parse)); file("/var/log/apache2/error.log" flags(no-parse)); file("/var/log/apache2/other_vhosts_access.log" flags(no-parse)); file("/var/log/apache2/ssl_access.log" flags(no-parse)); }; destination d_bulldog { syslog( "bulldog.d.umn.edu" transport("tls") port(6514) tls( peer-verify(required-trusted) ca_dir('/etc/syslog-ng/ssl/ca.d') key_file('/etc/syslog-ng/ssl/server.key') cert_file('/etc/syslog-ng/ssl/server.crt') ) ); }; # The funny .SDATA.file@18372.4.name is for structured data which, I believe, is part of the (new) # syslog protocol - RFC5424-formatted (IETF-syslog). # We need to set the filename value in the SDATA field. $FILE_NAME is a macro which returns the # full filename path. ie '/var/log/apache2/access.log'. That will then get assigned to the # structured data value, .SDATA.file@18372.4.name . rewrite r_setfilename { set( "$FILE_NAME", value(".SDATA.file@18372.4.name") ); }; # After getting the value set for the filename, truncate the directory portion and only use the # basename. Use a simple string substitution. rewrite r_use_basename { subst( "/var/log/apache2/", "", value(".SDATA.file@18372.4.name") type("string") flags("prefix") ); }; log { source(s_apache_logs); rewrite(r_setfilename); rewrite(r_use_basename); destination(d_bulldog); }; ---{end}--- Hope that helps, -mz ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq