<div dir="ltr">hardlinks are simply additional pointers to the same inode.<div>symlinks contain the name of the referenced object</div><div><br></div><div>If you want a hack - maybe have syslog-ng reload after the file changes? (SIGHUP)</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Jun 30, 2018 at 12:56 PM, Ankit Agarwal <span dir="ltr"><<a href="mailto:ankit@travelmyheart.org" target="_blank">ankit@travelmyheart.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u></u><div><div style="font-size:10pt;font-family:Verdana,Arial,Helvetica,sans-serif"><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">Hi,<br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">I ran into a similar problem on Ubuntu as well.<br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">In my case, I was tracking the Tomcat localhost log file in syslog-ng but Tomcat creates a new log file everyday by default, and the filename changes (since it includes the date).<br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">Therefore, I periodically created a softlink to the localhost log file where the link had a constant name. The constant name is needed because I obviously cannot keep changing the syslog-ng configuration to match the day's localhost log file name.<br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">I found that the softlink did not work.<br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">Instead I had to create a hardlink.<br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">This is because the softlink's modified date does not change when the underlying file changes. The hardlink's modified date does change since it is pointing to the actual data. We need the modified date to change for the syslog-ng client to pick up new log entries.<br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">In my case, I periodically ran the following command via CRON in the Tomcat logs directory:<br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">sudo ln -f $(ls -t localhost.* | head -1) tomcat_localhost.log<br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">This is to get the latest localhost log file and create the hardlink for it (overwriting the older hardlink that may have been pointing to the previous day's localhost log file).<br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">I ran this every hour just to be safe.<br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">So in your case, I think you would just need to recreate the hardlink as soon as your log file is rotated.<br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">Hope this helps.<br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">Ankit<br></div><div style="color:rgb(0,0,0);font-family:Verdana,Arial,Helvetica,sans-serif;font-size:13.3333px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial"><br></div><div><br></div><div class="m_3791569637936762633zmail_extra"><div id="m_3791569637936762633Zm-_Id_-Sgn1"><div><br></div><div>---- On Sat, 30 Jun 2018 01:13:44 -0700 <b>Donatello D <<a href="mailto:bluray.vik@gmail.com" target="_blank">bluray.vik@gmail.com</a>></b> wrote ----<br></div></div><div><br></div><blockquote style="border-left:1px solid #cccccc;padding-left:6px;margin:0 0 0 5px"><div><div dir="ltr"><div>syslog-ng is configured to read a symlink pointing to logs generated from my application which rotates the file using log4j2 rollingfile appender. Everything works fine till the rotation happens. after the file get rotated syslog-ng still seems to hold on to the older inode (which is not moved) and doesn't change to follow the new logs. this however does not happen in RHEL where syslog-ng recognizes the file is now rotated and moves to the new file. In both cases the sym link is always configured to point to the latest file. version details and logs from both OSs below.<br></div><div><br></div><div>What am i missing here?<br></div><div><br></div><div>UBUNTU - <br></div><div>syslog-ng 3.5.6<br></div><div>Installer-Version: 3.5.6<br></div><div>Revision: 3.5.6-2.1 [@416d315] (Ubuntu/16.04)<br></div><div>Compile-Date: Oct 24 2015 03:49:19<br></div><div>Available-Modules: afsocket,afuser,tfgeoip,<wbr>confgen,csvparser,<wbr>syslogformat,afamqp,redis,<wbr>afsql,affile,afsmtp,linux-<wbr>kmsg-format,dbparser,system-<wbr>source,cryptofuncs,basicfuncs,<wbr>json-plugin,afprog,afsocket-<wbr>tls,afstomp,afsocket-notls,<wbr>afmongodb<br></div><div>Enable-Debug: off<br></div><div>Enable-GProf: off<br></div><div>Enable-Memtrace: off<br></div><div>Enable-IPv6: on<br></div><div>Enable-Spoof-Source: on<br></div><div>Enable-TCP-Wrapper: on<br></div><div>Enable-Linux-Caps: on<br></div><div>Enable-Pcre: on<br></div><div><br></div><div>symlink is pointing to the file that gets the logs. prior to rotation the process watches correctly for the file (same inodes held by my app and syslog-ng)<br></div><div><br></div><div>lrwxrwxrwx 1 root root 56 Jun 29 08:44 node1-access.log -> /x/logs/vik-test_access.log<br></div><div><br></div><div>COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME<br></div><div>java 11032 vikram 53w REG 8,1 1101 1542626 vik-test_access.log<br></div><div>syslog-ng 21661 root 9r REG 8,1 1101 1542626 vik-test_access.log<br></div><div><br></div><div><br></div><div>Post rotation, syslog-ng holds on to the older file (now rotated).<br></div><div><br></div><div>COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME<br></div><div>java 11032 vikram 53w REG 8,1 876 1542631 e/elasticsearch-6.2.3/logs/<wbr>vik-test_access.log<br></div><div>syslog-ng 21661 root 9r REG 8,1 1101 1542626 e/elasticsearch-6.2.3/logs/<wbr>vik-test_access-2018-06-30.log<br></div><div><br></div><div>The same setup works perfectly fine in RHEL (version details below) where syslog-ng follows the new file correctly.<br></div><div><br></div><div>RHEL<br></div><div>syslog-ng 3.3.5<br></div><div>Installer-Version: 3.3.5<br></div><div>Revision: ssh+git://<a href="mailto:bazsi@git.balabit" target="_blank">bazsi@git.balabit</a>//<wbr>var/scm/git/syslog-ng/syslog-<wbr>ng-ose--mainline--3.3--master#<wbr>d5d607c05251b38e821efe27bc46ac<wbr>8db78dd722<br></div><div>Compile-Date: Oct 18 2012 15:17:09<br></div><div>Default-Modules: affile,afprog,afsocket,afuser,<wbr>basicfuncs,csvparser,dbparser,<wbr>syslogformat<br></div><div>Available-Modules: afprog,afsocket-tls,dbparser,<wbr>confgen,convertfuncs,<wbr>basicfuncs,afsocket,afmongodb,<wbr>csvparser,affile,dummy,<wbr>syslogformat,afuser<br></div><div>Enable-Debug: off<br></div><div>Enable-GProf: off<br></div><div>Enable-Memtrace: off<br></div><div>Enable-IPv6: on<br></div><div>Enable-Spoof-Source: off<br></div><div>Enable-TCP-Wrapper: on<br></div><div>Enable-Linux-Caps: off<br></div><div>Enable-Pcre: on<br></div><div><br></div></div><div>______________________________<wbr>______________________________<wbr>__________________<br></div><div>Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/<wbr>mailman/listinfo/syslog-ng</a><br></div><div>Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/<wbr>support/documentation/?<wbr>product=syslog-ng</a><br></div><div>FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" target="_blank">http://www.balabit.com/wiki/<wbr>syslog-ng-faq</a><br></div></div></blockquote></div><div><br></div></div><br></div><br>______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/<wbr>mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/<wbr>support/documentation/?<wbr>product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/<wbr>syslog-ng-faq</a><br>
<br>
<br></blockquote></div><br></div>