[syslog-ng] preserving tags across the network

[Matt Zagrabelny]

On Fri, Nov 11, 2011 at 1:23 PM, Fekete Róbert <frobert at balabit.hu> wrote:
>
> On Friday, November 11, 2011 18:04 CET, Matt Zagrabelny <mzagrabe at d.umn.edu> wrote:
>
>> Hi,
>>
>> I am trying to ship a bunch of apache log files across the network and
>> on the syslog-ng server side then break them into their individual
>> files again.
>>
>> I am not sure the best way to do this, but it looks like tags might be helpful.
>
> Hi, tags are not part of the syslog message unless you add them to the message using a template on the client, and then somehow extract them from the messages on the server. But it seems that you are actually trying to separate logs from different files, and you are using the syslog() drivers on your server and clients. If you use the file source and the syslog destination, syslog-ng adds the filename and some other metadata to the SDATA part of the message. You can extract this on the server side, and use it as a macro in the filename template on your server.
> Like:
> destination d_test {
>   file( "/var/log/apache2/${.SDATA.file at 18372.4.name}"
>          create_dirs(yes)
>        );
> };
>
> For details on other metadata added to SDATA, see http://www.balabit.com/sites/default/files/documents/syslog-ng-pe-4.0-guides/syslog-ng-pe-v4.0-guide-admin-en.html/file-source-and-syslog-destination.html
>
> Note to myself: this section is missing from the OSE guide for some reason, even though I believe the feature is available in OSE. Should check with Bazsi and update the docs.

Hi Róbert,

Thanks for the hints.

I am using syslog-ng from Debian Squeeze:

ii  syslog-ng                                3.1.3-3
               Next generation logging daemon

I have the following configured now on the server:

destination d_web2_access {
  file(
    "/tmp/apache2/${.SDATA.file at 18372.4.name}"
    create_dirs(yes)
  );
};

log {
  source(s_tls);
  destination(d_web2_access);
};

But all I see is the (newly created) /tmp/apache2 directory.

I can verify that I am getting apache logs sent over because they are
making it into the "user" facility file for my generic logging. The
relevant config snippet is:

destination d_remote_clients {
  file(
       "/var/log/syslog-ng/remote_clients/$HOST_FROM/$YEAR/$MONTH/$DAY/$FACILITY"
       owner(root)
       group(root)
       perm(0644)
       dir_perm(0755)
       create_dirs(yes)
      );
};

Any other ideas why the SDATA macro is not working?

Thanks again for the help!

-matt zagrabelny