[syslog-ng] syslog-ng not following symlinks correctly on UBUNTU, works fine in RHEL
Antal Nemes
antal.nemes at balabit.com
Sun Jul 1 06:28:27 UTC 2018
> the real question is why does it work in RHEL and fail in UBUNTU?
The difference might come from that you are using different versions of
syslog-ng on the two systems:
> UBUNTU -
> syslog-ng 3.5.6
> RHEL
> syslog-ng 3.3.5
> The constant name is needed because I obviously cannot keep changing the
syslog-ng configuration to match the day's localhost log file name.
You can avoid such hack. Please check the recent addition to syslog-ng:
wildcard filesource. You need to upgrade to 3.16 though.
https://syslog-ng.com/documents/html/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/reference-source-wildcard-file.html
On Sun, Jul 1, 2018 at 5:26 AM Evan Rempel <erempel at uvic.ca> wrote:
> I suspect it is more related to the filesystem and options.
>
> 1. Are you using a different fileystem (xfs vs ext3 or NFS)
> 2. Are there different mount options for the filesystem?
>
> Evan.
>
> On 06/30/2018 08:21 PM, Donatello D wrote:
> > @Jim - this is what i use as a workaround already, but it is
> > sub-optimal, as there will be other files that are rolled over in
> > different intervals, so i end up reloading config multiple times.
> >
> > the real question is why does it work in RHEL and fail in UBUNTU?
> >
> >
> >> Date: Sat, 30 Jun 2018 20:31:59 -0400
> >> From: Jim Hendrick <james.r.hendrick at gmail.com>
> >> To: "Syslog-ng users' and developers' mailing list"
> >> <syslog-ng at lists.balabit.hu>
> >> Subject: Re: [syslog-ng] syslog-ng not following symlinks correctly on
> >> UBUNTU, works fine in RHEL
> >> Message-ID:
> >> <CANEn2idABV25G1vFa4B=
> WhOyuHjd3HwLMKFBHgqydH6zvH0H9w at mail.gmail.com>
> >> Content-Type: text/plain; charset="utf-8"
> >>
> >> hardlinks are simply additional pointers to the same inode.
> >> symlinks contain the name of the referenced object
> >>
> >> If you want a hack - maybe have syslog-ng reload after the file changes?
> >> (SIGHUP)
> >>
> >>
> >>
> >> On Sat, Jun 30, 2018 at 12:56 PM, Ankit Agarwal <
> ankit at travelmyheart.org>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> I ran into a similar problem on Ubuntu as well.
> >>>
> >>> In my case, I was tracking the Tomcat localhost log file in syslog-ng
> but
> >>> Tomcat creates a new log file everyday by default, and the filename
> changes
> >>> (since it includes the date).
> >>>
> >>> Therefore, I periodically created a softlink to the localhost log file
> >>> where the link had a constant name. The constant name is needed
> because I
> >>> obviously cannot keep changing the syslog-ng configuration to match the
> >>> day's localhost log file name.
> >>>
> >>> I found that the softlink did not work.
> >>>
> >>> Instead I had to create a hardlink.
> >>>
> >>> This is because the softlink's modified date does not change when the
> >>> underlying file changes. The hardlink's modified date does change
> since it
> >>> is pointing to the actual data. We need the modified date to change
> for the
> >>> syslog-ng client to pick up new log entries.
> >>>
> >>> In my case, I periodically ran the following command via CRON in the
> >>> Tomcat logs directory:
> >>>
> >>> sudo ln -f $(ls -t localhost.* | head -1) tomcat_localhost.log
> >>>
> >>> This is to get the latest localhost log file and create the hardlink
> for
> >>> it (overwriting the older hardlink that may have been pointing to the
> >>> previous day's localhost log file).
> >>>
> >>> I ran this every hour just to be safe.
> >>>
> >>>
> >>> So in your case, I think you would just need to recreate the hardlink
> as
> >>> soon as your log file is rotated.
> >>>
> >>>
> >>> Hope this helps.
> >>>
> >>> Ankit
> >>>
> >>>
> >>>
> >>> ---- On Sat, 30 Jun 2018 01:13:44 -0700 *Donatello D
> >>> <bluray.vik at gmail.com <bluray.vik at gmail.com>>* wrote ----
> >>>
> >>> syslog-ng is configured to read a symlink pointing to logs generated
> from
> >>> my application which rotates the file using log4j2 rollingfile
> appender.
> >>> Everything works fine till the rotation happens. after the file get
> rotated
> >>> syslog-ng still seems to hold on to the older inode (which is not
> moved)
> >>> and doesn't change to follow the new logs. this however does not
> happen in
> >>> RHEL where syslog-ng recognizes the file is now rotated and moves to
> the
> >>> new file. In both cases the sym link is always configured to point to
> the
> >>> latest file. version details and logs from both OSs below.
> >>>
> >>> What am i missing here?
> >>>
> >>> UBUNTU -
> >>> syslog-ng 3.5.6
> >>> Installer-Version: 3.5.6
> >>> Revision: 3.5.6-2.1 [@416d315] (Ubuntu/16.04)
> >>> Compile-Date: Oct 24 2015 03:49:19
> >>> Available-Modules: afsocket,afuser,tfgeoip,confgen,csvparser,
> >>> syslogformat,afamqp,redis,afsql,affile,afsmtp,linux-
> >>> kmsg-format,dbparser,system-source,cryptofuncs,basicfuncs,
> >>> json-plugin,afprog,afsocket-tls,afstomp,afsocket-notls,afmongodb
> >>> Enable-Debug: off
> >>> Enable-GProf: off
> >>> Enable-Memtrace: off
> >>> Enable-IPv6: on
> >>> Enable-Spoof-Source: on
> >>> Enable-TCP-Wrapper: on
> >>> Enable-Linux-Caps: on
> >>> Enable-Pcre: on
> >>>
> >>> symlink is pointing to the file that gets the logs. prior to rotation
> the
> >>> process watches correctly for the file (same inodes held by my app and
> >>> syslog-ng)
> >>>
> >>> lrwxrwxrwx 1 root root 56 Jun 29 08:44 node1-access.log ->
> >>> /x/logs/vik-test_access.log
> >>>
> >>> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
> >>> java 11032 vikram 53w REG 8,1 1101 1542626
> >>> vik-test_access.log
> >>> syslog-ng 21661 root 9r REG 8,1 1101 1542626
> >>> vik-test_access.log
> >>>
> >>>
> >>> Post rotation, syslog-ng holds on to the older file (now rotated).
> >>>
> >>> COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
> >>> java 11032 vikram 53w REG 8,1 876 1542631
> >>> e/elasticsearch-6.2.3/logs/vik-test_access.log
> >>> syslog-ng 21661 root 9r REG 8,1 1101 1542626
> >>> e/elasticsearch-6.2.3/logs/vik-test_access-2018-06-30.log
> >>>
> >>> The same setup works perfectly fine in RHEL (version details below)
> where
> >>> syslog-ng follows the new file correctly.
> >>>
> >>> RHEL
> >>> syslog-ng 3.3.5
> >>> Installer-Version: 3.3.5
> >>> Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-
> >>> ng-ose--mainline--3.3--master#d5d607c05251b38e821efe27bc46ac8db78dd722
> >>> Compile-Date: Oct 18 2012 15:17:09
> >>> Default-Modules: affile,afprog,afsocket,afuser,
> >>> basicfuncs,csvparser,dbparser,syslogformat
> >>> Available-Modules: afprog,afsocket-tls,dbparser,confgen,convertfuncs,
> >>>
> basicfuncs,afsocket,afmongodb,csvparser,affile,dummy,syslogformat,afuser
> >>> Enable-Debug: off
> >>> Enable-GProf: off
> >>> Enable-Memtrace: off
> >>> Enable-IPv6: on
> >>> Enable-Spoof-Source: off
> >>> Enable-TCP-Wrapper: on
> >>> Enable-Linux-Caps: off
> >>> Enable-Pcre: on
> >>>
> >>>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180701/56b3c81b/attachment-0001.html>
More information about the syslog-ng
mailing list