Re: [syslog-ng] (U) [Non-DoD Source] Re: Rotate syslog-ng log files
CLASSIFICATION: UNCLASSIFIED Hello, So the file rotates now successfully – but what I have noticed is that after the file rotates it collects data for first few minute or so and then it stops collecting data (basically the file size never grows and timestamp never changes to the most latest when I check the file). I do see that syslog process/service is running. If I restart the service/process, it starts collecting data until the file rotation happens. Can you please let me know what would be causing this behavior? Thanks Jitesh Amin CLASSIFICATION: UNCLASSIFIED From: Amin, Jitesh CTR DISA JSP (US) Sent: Tuesday, June 5, 2018 9:59 AM To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: RE: (U) [syslog-ng] [Non-DoD Source] Re: Rotate syslog-ng log files CLASSIFICATION: UNCLASSIFIED OK skipping the {} made it work and I now see a syslog file with timestamp (year-month-day). Does this mean it should rotate to new log file name (tomorrows timestamp) at midnight tonight? OR I need to add syntax so it rolls everyday with new timestamp. Just wanted to confirm. Thanks Jitesh Amin CLASSIFICATION: UNCLASSIFIED From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Scheidler, Balázs Sent: Thursday, May 31, 2018 5:48 AM To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] (U) [Non-DoD Source] Re: Rotate syslog-ng log files All active links contained in this email were disabled. Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser. _____ I mean syslog-ng 1.6.8 On May 30, 2018 22:54, "Balazs Scheidler" <bazsi77@gmail.com < Caution-mailto:bazsi77@gmail.com > > wrote: syslog-ng does have template support, it just doesnt support braces, which came later. Just write $YEAR instead of ${YEAR} On May 30, 2018 09:41, "Gergely Nagy" <algernon@balabit.com < Caution-mailto:algernon@balabit.com > > wrote:
"Amin" == Amin, Jitesh CTR DISA JSP (US) <jitesh.amin.ctr@mail.mil < Caution-mailto:jitesh.amin.ctr@mail.mil > > writes:
Amin> Let me ask this, with the following config = destination Amin> syslog { file("/var/log/syslog-${YEAR}-${MONTH}-${DAY}.log"); Amin> }; Amin> It created new file and started writing to it (versus creating Amin> new syslog.log). Question, if we plan to accept this for now, Amin> with above config, would it create a new file ever day with Amin> following file names or no it would not work with v1.6.8 With syslog-ng 1.6.8, it would not create a new file every day, and would continue writing to syslog-{YEAR}-{MONTH}-{DAY}.log. With newer versions, it would create files like `syslog-2018-05-30.log`. No `.0`, `.1` or the like would be appended. That's a convention of logrotate. With syslog-ng, you get filenames that match the template, they will have nothing appended or prepended that is not in the filename template. -- |8] ______________________________________________________________________________ Member info: Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng < Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: Caution-http://www.balabit.com/support/documentation/?product=syslog-ng < Caution-http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: Caution-http://www.balabit.com/wiki/syslog-ng-faq < Caution-http://www.balabit.com/wiki/syslog-ng-faq > ______________________________________________________________________________ Member info: Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng < Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: Caution-http://www.balabit.com/support/documentation/?product=syslog-ng < Caution-http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: Caution-http://www.balabit.com/wiki/syslog-ng-faq < Caution-http://www.balabit.com/wiki/syslog-ng-faq >
Hi, Honestly I don't know, it seems like a bug. Even the 1.6 branch had bugfixes up until 1.6.11, but it's been more than a decade ago. :) Any chance of this being upgrade to a newer version? I am sure Balabit can also offer services, but I am not trying to be pushy. -- Bazsi On Thu, Jun 7, 2018 at 1:06 PM, Amin, Jitesh CTR DISA JSP (US) < jitesh.amin.ctr@mail.mil> wrote:
CLASSIFICATION: UNCLASSIFIED
Hello,
So the file rotates now successfully – but what I have noticed is that after the file rotates it collects data for first few minute or so and then it stops collecting data (basically the file size never grows and timestamp never changes to the most latest when I check the file).
I do see that syslog process/service is running. If I restart the service/process, it starts collecting data until the file rotation happens.
Can you please let me know what would be causing this behavior?
Thanks
Jitesh Amin
CLASSIFICATION: UNCLASSIFIED
*From:* Amin, Jitesh CTR DISA JSP (US) *Sent:* Tuesday, June 5, 2018 9:59 AM *To:* Syslog-ng users' and developers' mailing list < syslog-ng@lists.balabit.hu> *Subject:* RE: (U) [syslog-ng] [Non-DoD Source] Re: Rotate syslog-ng log files
CLASSIFICATION: UNCLASSIFIED
OK skipping the {} made it work and I now see a syslog file with timestamp (year-month-day). Does this mean it should rotate to new log file name (tomorrows timestamp) at midnight tonight? OR I need to add syntax so it rolls everyday with new timestamp. Just wanted to confirm.
Thanks
Jitesh Amin
CLASSIFICATION: UNCLASSIFIED
*From:* syslog-ng <syslog-ng-bounces@lists.balabit.hu> *On Behalf Of *Scheidler, Balázs *Sent:* Thursday, May 31, 2018 5:48 AM *To:* Syslog-ng users' and developers' mailing list < syslog-ng@lists.balabit.hu> *Subject:* Re: [syslog-ng] (U) [Non-DoD Source] Re: Rotate syslog-ng log files
All active links contained in this email were disabled. Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser. ------------------------------
I mean syslog-ng 1.6.8
On May 30, 2018 22:54, "Balazs Scheidler" <bazsi77@gmail.com < Caution- mailto:bazsi77@gmail.com > > wrote:
syslog-ng does have template support, it just doesnt support braces, which came later.
Just write $YEAR instead of ${YEAR}
On May 30, 2018 09:41, "Gergely Nagy" <algernon@balabit.com < Caution-mailto:algernon@balabit.com > > wrote:
"Amin" == Amin, Jitesh CTR DISA JSP (US) <jitesh.amin.ctr@mail.mil < Caution-mailto:jitesh.amin.ctr@mail.mil > > writes:
Amin> Let me ask this, with the following config = destination Amin> syslog { file("/var/log/syslog-${YEAR}-${MONTH}-${DAY}.log"); Amin> };
Amin> It created new file and started writing to it (versus creating Amin> new syslog.log). Question, if we plan to accept this for now, Amin> with above config, would it create a new file ever day with Amin> following file names or no it would not work with v1.6.8
With syslog-ng 1.6.8, it would not create a new file every day, and would continue writing to syslog-{YEAR}-{MONTH}-{DAY}.log. With newer versions, it would create files like `syslog-2018-05-30.log`. No `.0`, `.1` or the like would be appended. That's a convention of logrotate. With syslog-ng, you get filenames that match the template, they will have nothing appended or prepended that is not in the filename template.
-- |8] ____________________________________________________________ __________________ Member info: Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng < Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: Caution-http://www.balabit.com/support/documentation/? product=syslog-ng < Caution-http://www.balabit.com/support/ documentation/?product=syslog-ng > FAQ: Caution-http://www.balabit.com/wiki/syslog-ng-faq < Caution- http://www.balabit.com/wiki/syslog-ng-faq >
____________________________________________________________ __________________ Member info: Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng < Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: Caution-http://www.balabit.com/support/documentation/? product=syslog-ng < Caution-http://www.balabit.com/support/ documentation/?product=syslog-ng > FAQ: Caution-http://www.balabit.com/wiki/syslog-ng-faq < Caution- http://www.balabit.com/wiki/syslog-ng-faq >
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hello, Syslog-ng doesn't do any form of rotation - you're using macros in the destination filenames instead. Contents of macros like $YEAR get parsed from the timestamp of the incoming messages so as long as messages contain older timestamps syslog-ng will (re)open files reflecting these older timestamps and append logs there. There is no need to restart syslog-ng as not the restart changes where logs will get written but the metadata associated with the logs. IIRC the antique 1.6.8 version you're using also has support for other datetime macros (prefixed with C_ or R_) which reflect the 'C'urrent or 'R'eception timestamp so you can alter the current behaviour - although I wouldn't recommend switching for example to $C_YEAR-$C_MONTH-$C_DAY as it could be confusing to see the last few logs of a given day written to another file than people would expect. Does this make sense or did I misunderstood your observation? Regards, Sandor On 06/07/2018 01:06 PM, Amin, Jitesh CTR DISA JSP (US) wrote:
CLASSIFICATION: UNCLASSIFIED
Hello,
So the file rotates now successfully – but what I have noticed is that after the file rotates it collects data for first few minute or so and then it stops collecting data (basically the file size never grows and timestamp never changes to the most latest when I check the file).
I do see that syslog process/service is running. If I restart the service/process, it starts collecting data until the file rotation happens.
Can you please let me know what would be causing this behavior?
Thanks
Jitesh Amin
CLASSIFICATION: UNCLASSIFIED
*From:*Amin, Jitesh CTR DISA JSP (US) *Sent:* Tuesday, June 5, 2018 9:59 AM *To:* Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> *Subject:* RE: (U) [syslog-ng] [Non-DoD Source] Re: Rotate syslog-ng log files
CLASSIFICATION: UNCLASSIFIED
OK skipping the {} made it work and I now see a syslog file with timestamp (year-month-day). Does this mean it should rotate to new log file name (tomorrows timestamp) at midnight tonight? OR I need to add syntax so it rolls everyday with new timestamp. Just wanted to confirm.
Thanks
Jitesh Amin
CLASSIFICATION: UNCLASSIFIED
*From:*syslog-ng <syslog-ng-bounces@lists.balabit.hu> *On Behalf Of *Scheidler, Balázs *Sent:* Thursday, May 31, 2018 5:48 AM *To:* Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> *Subject:* Re: [syslog-ng] (U) [Non-DoD Source] Re: Rotate syslog-ng log files
All active links contained in this email were disabled. Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser.
------------------------------------------------------------------------
I mean syslog-ng 1.6.8
On May 30, 2018 22:54, "Balazs Scheidler" <bazsi77@gmail.com < Caution-mailto:bazsi77@gmail.com > > wrote:
syslog-ng does have template support, it just doesnt support braces, which came later.
Just write $YEAR instead of ${YEAR}
On May 30, 2018 09:41, "Gergely Nagy" <algernon@balabit.com < Caution-mailto:algernon@balabit.com > > wrote:
>>>>> "Amin" == Amin, Jitesh CTR DISA JSP (US) <jitesh.amin.ctr@mail.mil < Caution-mailto:jitesh.amin.ctr@mail.mil > > writes:
Amin> Let me ask this, with the following config = destination Amin> syslog { file("/var/log/syslog-${YEAR}-${MONTH}-${DAY}.log"); Amin> };
Amin> It created new file and started writing to it (versus creating Amin> new syslog.log). Question, if we plan to accept this for now, Amin> with above config, would it create a new file ever day with Amin> following file names or no it would not work with v1.6.8
With syslog-ng 1.6.8, it would not create a new file every day, and would continue writing to syslog-{YEAR}-{MONTH}-{DAY}.log. With newer versions, it would create files like `syslog-2018-05-30.log`. No `.0`, `.1` or the like would be appended. That's a convention of logrotate. With syslog-ng, you get filenames that match the template, they will have nothing appended or prepended that is not in the filename template.
-- |8] ______________________________________________________________________________ Member info: Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng < Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng >
Documentation: Caution-http://www.balabit.com/support/documentation/?product=syslog-ng < Caution-http://www.balabit.com/support/documentation/?product=syslog-ng >
FAQ: Caution-http://www.balabit.com/wiki/syslog-ng-faq < Caution-http://www.balabit.com/wiki/syslog-ng-faq >
______________________________________________________________________________ Member info: Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng < Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng >
Documentation: Caution-http://www.balabit.com/support/documentation/?product=syslog-ng < Caution-http://www.balabit.com/support/documentation/?product=syslog-ng >
FAQ: Caution-http://www.balabit.com/wiki/syslog-ng-faq < Caution-http://www.balabit.com/wiki/syslog-ng-faq >
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
C_ macros are much newer than 1.6, S_ and R_ probably work. On Thu, Jun 7, 2018 at 2:18 PM, Sandor Geller <sandor.geller@ericsson.com> wrote:
Hello,
Syslog-ng doesn't do any form of rotation - you're using macros in the destination filenames instead. Contents of macros like $YEAR get parsed from the timestamp of the incoming messages so as long as messages contain older timestamps syslog-ng will (re)open files reflecting these older timestamps and append logs there. There is no need to restart syslog-ng as not the restart changes where logs will get written but the metadata associated with the logs.
IIRC the antique 1.6.8 version you're using also has support for other datetime macros (prefixed with C_ or R_) which reflect the 'C'urrent or 'R'eception timestamp so you can alter the current behaviour - although I wouldn't recommend switching for example to $C_YEAR-$C_MONTH-$C_DAY as it could be confusing to see the last few logs of a given day written to another file than people would expect.
Does this make sense or did I misunderstood your observation?
Regards, Sandor
On 06/07/2018 01:06 PM, Amin, Jitesh CTR DISA JSP (US) wrote:
CLASSIFICATION: UNCLASSIFIED
Hello,
So the file rotates now successfully – but what I have noticed is that after the file rotates it collects data for first few minute or so and then it stops collecting data (basically the file size never grows and timestamp never changes to the most latest when I check the file).
I do see that syslog process/service is running. If I restart the service/process, it starts collecting data until the file rotation happens.
Can you please let me know what would be causing this behavior?
Thanks
Jitesh Amin
CLASSIFICATION: UNCLASSIFIED
*From:* Amin, Jitesh CTR DISA JSP (US) *Sent:* Tuesday, June 5, 2018 9:59 AM *To:* Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> <syslog-ng@lists.balabit.hu> *Subject:* RE: (U) [syslog-ng] [Non-DoD Source] Re: Rotate syslog-ng log files
CLASSIFICATION: UNCLASSIFIED
OK skipping the {} made it work and I now see a syslog file with timestamp (year-month-day). Does this mean it should rotate to new log file name (tomorrows timestamp) at midnight tonight? OR I need to add syntax so it rolls everyday with new timestamp. Just wanted to confirm.
Thanks
Jitesh Amin
CLASSIFICATION: UNCLASSIFIED
*From:* syslog-ng <syslog-ng-bounces@lists.balabit.hu> <syslog-ng-bounces@lists.balabit.hu> *On Behalf Of *Scheidler, Balázs *Sent:* Thursday, May 31, 2018 5:48 AM *To:* Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> <syslog-ng@lists.balabit.hu> *Subject:* Re: [syslog-ng] (U) [Non-DoD Source] Re: Rotate syslog-ng log files
All active links contained in this email were disabled. Please verify the identity of the sender, and confirm the authenticity of all links contained within the message prior to copying and pasting the address to a Web browser. ------------------------------
I mean syslog-ng 1.6.8
On May 30, 2018 22:54, "Balazs Scheidler" <bazsi77@gmail.com < Caution- mailto:bazsi77@gmail.com > <%C2%A0Caution-mailto:bazsi77@gmail.com> > wrote:
syslog-ng does have template support, it just doesnt support braces, which came later.
Just write $YEAR instead of ${YEAR}
On May 30, 2018 09:41, "Gergely Nagy" <algernon@balabit.com < Caution-mailto:algernon@balabit.com > <%C2%A0Caution-mailto:algernon@balabit.com> > wrote:
"Amin" == Amin, Jitesh CTR DISA JSP (US) <jitesh.amin.ctr@mail.mil < Caution-mailto:jitesh.amin.ctr@mail.mil > <%C2%A0Caution-mailto:jitesh.amin.ctr@mail.mil> > writes:
Amin> Let me ask this, with the following config = destination Amin> syslog { file("/var/log/syslog-${YEAR}-${MONTH}-${DAY}.log"); Amin> };
Amin> It created new file and started writing to it (versus creating Amin> new syslog.log). Question, if we plan to accept this for now, Amin> with above config, would it create a new file ever day with Amin> following file names or no it would not work with v1.6.8
With syslog-ng 1.6.8, it would not create a new file every day, and would continue writing to syslog-{YEAR}-{MONTH}-{DAY}.log. With newer versions, it would create files like `syslog-2018-05-30.log`. No `.0`, `.1` or the like would be appended. That's a convention of logrotate. With syslog-ng, you get filenames that match the template, they will have nothing appended or prepended that is not in the filename template.
-- |8] ____________________________________________________________ __________________ Member info: Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng < Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: Caution-http://www.balabit.com/support/documentation/? product=syslog-ng < Caution-http://www.balabit.com/support/ documentation/?product=syslog-ng > FAQ: Caution-http://www.balabit.com/wiki/syslog-ng-faq < Caution- http://www.balabit.com/wiki/syslog-ng-faq >
____________________________________________________________ __________________ Member info: Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng < Caution-https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: Caution-http://www.balabit.com/support/documentation/? product=syslog-ng < Caution-http://www.balabit.com/support/ documentation/?product=syslog-ng > FAQ: Caution-http://www.balabit.com/wiki/syslog-ng-faq < Caution- http://www.balabit.com/wiki/syslog-ng-faq >
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (4)
-
Amin, Jitesh CTR DISA JSP (US)
-
Fekete, Róbert
-
Sandor Geller
-
Scheidler, Balázs