[syslog-ng] syslog-ng not following symlinks correctly on UBUNTU, works fine in RHEL

Sun Jul 1 13:54:39 UTC 2018

No clue. Maybe look at are the files on the same or different partitions?
Are the filesystems the same type? Mount options?

Weird...

On Sat, Jun 30, 2018, 11:21 PM Donatello D <bluray.vik at gmail.com> wrote:

> @Jim - this is what i use as a  workaround already, but it is
> sub-optimal, as there will be other files that are rolled over in
> different intervals, so i end up reloading config multiple times.
>
> the real question is why does it work in RHEL and fail in UBUNTU?
>
>
> > Date: Sat, 30 Jun 2018 20:31:59 -0400
> > From: Jim Hendrick <james.r.hendrick at gmail.com>
> > To: "Syslog-ng users' and developers' mailing list"
> >         <syslog-ng at lists.balabit.hu>
> > Subject: Re: [syslog-ng] syslog-ng not following symlinks correctly on
> >         UBUNTU, works fine in RHEL
> > Message-ID:
> >         <CANEn2idABV25G1vFa4B=
> WhOyuHjd3HwLMKFBHgqydH6zvH0H9w at mail.gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> >
> > hardlinks are simply additional pointers to the same inode.
> > symlinks contain the name of the referenced object
> >
> > If you want a hack - maybe have syslog-ng reload after the file changes?
> > (SIGHUP)
> >
> >
> >
> > On Sat, Jun 30, 2018 at 12:56 PM, Ankit Agarwal <ankit at travelmyheart.org
> >
> > wrote:
> >
> > > Hi,
> > >
> > > I ran into a similar problem on Ubuntu as well.
> > >
> > > In my case, I was tracking the Tomcat localhost log file in syslog-ng
> but
> > > Tomcat creates a new log file everyday by default, and the filename
> changes
> > > (since it includes the date).
> > >
> > > Therefore, I periodically created a softlink to the localhost log file
> > > where the link had a constant name. The constant name is needed
> because I
> > > obviously cannot keep changing the syslog-ng configuration to match the
> > > day's localhost log file name.
> > >
> > > I found that the softlink did not work.
> > >
> > > Instead I had to create a hardlink.
> > >
> > > This is because the softlink's modified date does not change when the
> > > underlying file changes. The hardlink's modified date does change
> since it
> > > is pointing to the actual data. We need the modified date to change
> for the
> > > syslog-ng client to pick up new log entries.
> > >
> > > In my case, I periodically ran the following command via CRON in the
> > > Tomcat logs directory:
> > >
> > > sudo ln -f $(ls -t localhost.* | head -1) tomcat_localhost.log
> > >
> > > This is to get the latest localhost log file and create the hardlink
> for
> > > it (overwriting the older hardlink that may have been pointing to the
> > > previous day's localhost log file).
> > >
> > > I ran this every hour just to be safe.
> > >
> > >
> > > So in your case, I think you would just need to recreate the hardlink
> as
> > > soon as your log file is rotated.
> > >
> > >
> > > Hope this helps.
> > >
> > > Ankit
> > >
> > >
> > >
> > > ---- On Sat, 30 Jun 2018 01:13:44 -0700 *Donatello D
> > > <bluray.vik at gmail.com <bluray.vik at gmail.com>>* wrote ----
> > >
> > > syslog-ng is configured to read a symlink pointing to logs generated
> from
> > > my application which rotates the file using log4j2 rollingfile
> appender.
> > > Everything works fine till the rotation happens. after the file get
> rotated
> > > syslog-ng still seems to hold on to the older inode (which is not
> moved)
> > > and doesn't change to follow the new logs. this however does not
> happen in
> > > RHEL where syslog-ng recognizes the file is now rotated and moves to
> the
> > > new file. In both cases the sym link is always configured to point to
> the
> > > latest file. version details and logs from both OSs below.
> > >
> > > What am i missing here?
> > >
> > > UBUNTU -
> > > syslog-ng 3.5.6
> > > Installer-Version: 3.5.6
> > > Revision: 3.5.6-2.1 [@416d315] (Ubuntu/16.04)
> > > Compile-Date: Oct 24 2015 03:49:19
> > > Available-Modules: afsocket,afuser,tfgeoip,confgen,csvparser,
> > > syslogformat,afamqp,redis,afsql,affile,afsmtp,linux-
> > > kmsg-format,dbparser,system-source,cryptofuncs,basicfuncs,
> > > json-plugin,afprog,afsocket-tls,afstomp,afsocket-notls,afmongodb
> > > Enable-Debug: off
> > > Enable-GProf: off
> > > Enable-Memtrace: off
> > > Enable-IPv6: on
> > > Enable-Spoof-Source: on
> > > Enable-TCP-Wrapper: on
> > > Enable-Linux-Caps: on
> > > Enable-Pcre: on
> > >
> > > symlink is pointing to the file that gets the logs. prior to rotation
> the
> > > process watches correctly for the file (same inodes held by my app and
> > > syslog-ng)
> > >
> > > lrwxrwxrwx 1 root root 56 Jun 29 08:44 node1-access.log ->
> > > /x/logs/vik-test_access.log
> > >
> > > COMMAND     PID       USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
> > > java      11032       vikram 53w   REG    8,1     1101 1542626
> > > vik-test_access.log
> > > syslog-ng 21661       root    9r   REG    8,1     1101 1542626
> > > vik-test_access.log
> > >
> > >
> > > Post rotation, syslog-ng holds on to the older file (now rotated).
> > >
> > > COMMAND     PID       USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
> > > java      11032       vikram  53w   REG    8,1      876 1542631
> > > e/elasticsearch-6.2.3/logs/vik-test_access.log
> > > syslog-ng 21661       root    9r   REG    8,1     1101 1542626
> > > e/elasticsearch-6.2.3/logs/vik-test_access-2018-06-30.log
> > >
> > > The same setup works perfectly fine in RHEL (version details below)
> where
> > > syslog-ng follows the new file correctly.
> > >
> > > RHEL
> > > syslog-ng 3.3.5
> > > Installer-Version: 3.3.5
> > > Revision: ssh+git://bazsi@git.balabit//var/scm/git/syslog-ng/syslog-
> > > ng-ose--mainline--3.3--master#d5d607c05251b38e821efe27bc46ac8db78dd722
> > > Compile-Date: Oct 18 2012 15:17:09
> > > Default-Modules: affile,afprog,afsocket,afuser,
> > > basicfuncs,csvparser,dbparser,syslogformat
> > > Available-Modules: afprog,afsocket-tls,dbparser,confgen,convertfuncs,
> > >
> basicfuncs,afsocket,afmongodb,csvparser,affile,dummy,syslogformat,afuser
> > > Enable-Debug: off
> > > Enable-GProf: off
> > > Enable-Memtrace: off
> > > Enable-IPv6: on
> > > Enable-Spoof-Source: off
> > > Enable-TCP-Wrapper: on
> > > Enable-Linux-Caps: off
> > > Enable-Pcre: on
> > >
> > > ____________________________________________________________
> > > __________________
> > > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > > Documentation: http://www.balabit.com/support/documentation/?
> > > product=syslog-ng
> > > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> > >
> > >
> > >
> > >
> > > ____________________________________________________________
> > > __________________
> > > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > > Documentation: http://www.balabit.com/support/documentation/?
> > > product=syslog-ng
> > > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> > >
> > >
> > >
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: <
> http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180630/db683a38/attachment.html
> >
> >
> > ------------------------------
> >
> > Subject: Digest Footer
> >
> > _______________________________________________
> > syslog-ng maillist  -  syslog-ng at lists.balabit.hu
> > https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >
> >
> > ------------------------------
> >
> > End of syslog-ng Digest, Vol 159, Issue 1
> > *****************************************
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180701/675137ef/attachment.html>