Parser not parsing :-)
I can only assume I am not implementing this correctly. :-) But I have a parser I am trying to use so I can take a subset of the information of a message and send that subset to another receiver. This is the whole message: <13>Feb 4 18:40:17 myhost syslogng: 2012-02-04T18:40:17-08:00
myhostserver-http /tmp/logs/access_log Hi Mom
What I want to do is send out the message as : <13>Feb 4 18:40:17 myhost syslogng: Hi Mom Notice how I dropped the middle part out.
From what I have read, the parser acts on the message body alone. Is this correct? So I set it up to look for four(4) columns of data and to be "greedy" on the last column.
I have played around with the number of columns and even used a rewrite function instead. But the Parser continues to produce empty variables. And my template just echos out my default value. Any thoughts?
parser p_et_logmessage { csv-parser( #columns("ETMSG") #columns("ETMSG.ISODATE") columns("ETMSG.ISODATE", "ETMSG.EASI", "ETMSG.SOURCE", "ETMSG.BODY") delimiters(" ") #template("${MSG}") flags(greedy) ); };
rewrite r_rewrite_set{set('${ETMSG.BODY:-nothing}', value("MESSAGE"));};
template t_et_basic_logmessage { template("${ETMSG.BODY:-nothing}\n"); template_escape(no); };
destination destination_info { tcp("host2" port(8080) template(t_et_basic_logmessage) log_disk_fifo_size(32212254720) ); };
log { source(INTAKE); parser(p_et_logmessage); destination(destination_info); };
My latest Post: Givenchy Fall/Winter 2012 Collection – Runway | Highsnobiety.com<http://www.webeclubbin.com/blog/2012/01/givenchy-fallwinter-2012-collection-runway-highsnobiety-com/> Get a signature like this. <http://r1.wisestamp.com/r/landing?promo=19&dest=http%3A%2F%2Fwww.wisestamp.com%2Femail-install%3Futm_source%3Dextension%26utm_medium%3Demail%26utm_campaign%3Dpromo_19> CLICK HERE.<http://r1.wisestamp.com/r/landing?promo=19&dest=http%3A%2F%2Fwww.wisestamp.com%2Femail-install%3Futm_source%3Dextension%26utm_medium%3Demail%26utm_campaign%3Dpromo_19>
Looks like my last email was chopped off . Here it is again. ################ I can only assume I am not implementing this correctly. :-) But I have a parser I am trying to use so I can take a subset of the information of a message and send that subset to another receiver. This is the whole message: <13>Feb 4 18:40:17 myhost syslogng: 2012-02-04T18:40:17-08:00 myhostserver-http /tmp/logs/access_log Hi Mom What I want to do is send out the message as : <13>Feb 4 18:40:17 myhost syslogng: Hi Mom Notice how I dropped the middle part out.
From what I have read, the parser acts on the message body alone. Is this correct? So I set it up to look for four(4) columns of data and to be "greedy" on the last column.
I have played around with the number of columns and even used a rewrite function instead. But the Parser continues to produce empty variables. And my template just echos out my default value. Any thoughts? parser p_et_logmessage { csv-parser( #columns("ETMSG") #columns("ETMSG.ISODATE") columns("ETMSG.ISODATE", "ETMSG.EASI", "ETMSG.SOURCE", "ETMSG.BODY") delimiters(" ") #template("${MSG}") flags(greedy) ); }; rewrite r_rewrite_set{set('${ETMSG.BODY:-nothing}', value("MESSAGE"));}; template t_et_basic_logmessage { template("${ETMSG.BODY:-nothing}\n"); template_escape(no); }; destination destination_info { tcp("host2" port(8080) template(t_et_basic_logmessage) log_disk_fifo_size(32212254720) ); }; log { source(INTAKE); parser(p_et_logmessage); destination(destination_info); }; On Mon, Feb 6, 2012 at 11:07 AM, T. A. Smooth <catdaaaady@gmail.com> wrote:
I can only assume I am not implementing this correctly. :-)
But I have a parser I am trying to use so I can take a subset of the information of a message and send that subset to another receiver. This is the whole message:
<13>Feb 4 18:40:17 myhost syslogng: 2012-02-04T18:40:17-08:00 myhostserver-http /tmp/logs/access_log Hi Mom
What I want to do is send out the message as :
<13>Feb 4 18:40:17 myhost syslogng: Hi Mom
Notice how I dropped the middle part out.
From what I have read, the parser acts on the message body alone. Is this correct? So I set it up to look for four(4) columns of data and to be "greedy" on the last column.
I have played around with the number of columns and even used a rewrite function instead. But the Parser continues to produce empty variables. And my template just echos out my default value.
Any thoughts?
parser p_et_logmessage { csv-parser( #columns("ETMSG") #columns("ETMSG.ISODATE") columns("ETMSG.ISODATE", "ETMSG.EASI", "ETMSG.SOURCE", "ETMSG.BODY") delimiters(" ") #template("${MSG}") flags(greedy) ); };
rewrite r_rewrite_set{set('${ETMSG.BODY:-nothing}', value("MESSAGE"));};
template t_et_basic_logmessage { template("${ETMSG.BODY:-nothing}\n"); template_escape(no); };
destination destination_info { tcp("host2" port(8080) template(t_et_basic_logmessage) log_disk_fifo_size(32212254720) ); };
log { source(INTAKE); parser(p_et_logmessage); destination(destination_info); };
My latest Post: Givenchy Fall/Winter 2012 Collection – Runway | Highsnobiety.com Get a signature like this. CLICK HERE.
Okay this is really weird. Sorry again. I pasted contents of the email here with the configuration . Hopefully this is not too much of a inconvenience. http://pastebin.com/YQUD5TrA On Mon, Feb 6, 2012 at 11:27 AM, T. A. Smooth <catdaaaady@gmail.com> wrote:
Looks like my last email was chopped off . Here it is again. ################
I can only assume I am not implementing this correctly. :-)
But I have a parser I am trying to use so I can take a subset of the information of a message and send that subset to another receiver. This is the whole message:
<13>Feb 4 18:40:17 myhost syslogng: 2012-02-04T18:40:17-08:00 myhostserver-http /tmp/logs/access_log Hi Mom
What I want to do is send out the message as :
<13>Feb 4 18:40:17 myhost syslogng: Hi Mom
Notice how I dropped the middle part out.
From what I have read, the parser acts on the message body alone. Is this correct? So I set it up to look for four(4) columns of data and to be "greedy" on the last column.
I have played around with the number of columns and even used a rewrite function instead. But the Parser continues to produce empty variables. And my template just echos out my default value.
Any thoughts?
parser p_et_logmessage { csv-parser( #columns("ETMSG") #columns("ETMSG.ISODATE") columns("ETMSG.ISODATE", "ETMSG.EASI", "ETMSG.SOURCE", "ETMSG.BODY") delimiters(" ") #template("${MSG}") flags(greedy) ); };
rewrite r_rewrite_set{set('${ETMSG.BODY:-nothing}', value("MESSAGE"));};
template t_et_basic_logmessage { template("${ETMSG.BODY:-nothing}\n"); template_escape(no); };
destination destination_info { tcp("host2" port(8080) template(t_et_basic_logmessage) log_disk_fifo_size(32212254720) ); };
log { source(INTAKE); parser(p_et_logmessage); destination(destination_info); };
On Mon, Feb 6, 2012 at 11:07 AM, T. A. Smooth <catdaaaady@gmail.com> wrote:
I can only assume I am not implementing this correctly. :-)
But I have a parser I am trying to use so I can take a subset of the information of a message and send that subset to another receiver. This is the whole message:
<13>Feb 4 18:40:17 myhost syslogng: 2012-02-04T18:40:17-08:00 myhostserver-http /tmp/logs/access_log Hi Mom
What I want to do is send out the message as :
<13>Feb 4 18:40:17 myhost syslogng: Hi Mom
Notice how I dropped the middle part out.
From what I have read, the parser acts on the message body alone. Is this correct? So I set it up to look for four(4) columns of data and to be "greedy" on the last column.
I have played around with the number of columns and even used a rewrite function instead. But the Parser continues to produce empty variables. And my template just echos out my default value.
Any thoughts?
parser p_et_logmessage { csv-parser( #columns("ETMSG") #columns("ETMSG.ISODATE") columns("ETMSG.ISODATE", "ETMSG.EASI", "ETMSG.SOURCE", "ETMSG.BODY") delimiters(" ") #template("${MSG}") flags(greedy) ); };
rewrite r_rewrite_set{set('${ETMSG.BODY:-nothing}', value("MESSAGE"));};
template t_et_basic_logmessage { template("${ETMSG.BODY:-nothing}\n"); template_escape(no); };
destination destination_info { tcp("host2" port(8080) template(t_et_basic_logmessage) log_disk_fifo_size(32212254720) ); };
log { source(INTAKE); parser(p_et_logmessage); destination(destination_info); };
My latest Post: Givenchy Fall/Winter 2012 Collection – Runway | Highsnobiety.com Get a signature like this. CLICK HERE.
I think I found the issue. I define the parser once, but i use it in two different log statments. When i use it twice the parser places blanks in the custom columns. If i only use it once everything works and the custom columns have the right values. Expect behavior? On Feb 6, 2012 11:37 AM, "T. A. Smooth" <catdaaaady@gmail.com> wrote:
Okay this is really weird. Sorry again. I pasted contents of the email here with the configuration . Hopefully this is not too much of a inconvenience.
On Mon, Feb 6, 2012 at 11:27 AM, T. A. Smooth <catdaaaady@gmail.com> wrote:
Looks like my last email was chopped off . Here it is again. ################
I can only assume I am not implementing this correctly. :-)
But I have a parser I am trying to use so I can take a subset of the information of a message and send that subset to another receiver. This is the whole message:
<13>Feb 4 18:40:17 myhost syslogng: 2012-02-04T18:40:17-08:00 myhostserver-http /tmp/logs/access_log Hi Mom
What I want to do is send out the message as :
<13>Feb 4 18:40:17 myhost syslogng: Hi Mom
Notice how I dropped the middle part out.
From what I have read, the parser acts on the message body alone. Is this correct? So I set it up to look for four(4) columns of data and to be "greedy" on the last column.
I have played around with the number of columns and even used a rewrite function instead. But the Parser continues to produce empty variables. And my template just echos out my default value.
Any thoughts?
parser p_et_logmessage { csv-parser( #columns("ETMSG") #columns("ETMSG.ISODATE") columns("ETMSG.ISODATE", "ETMSG.EASI", "ETMSG.SOURCE", "ETMSG.BODY") delimiters(" ") #template("${MSG}") flags(greedy) ); };
rewrite r_rewrite_set{set('${ETMSG.BODY:-nothing}', value("MESSAGE"));};
template t_et_basic_logmessage { template("${ETMSG.BODY:-nothing}\n"); template_escape(no); };
destination destination_info { tcp("host2" port(8080) template(t_et_basic_logmessage) log_disk_fifo_size(32212254720) ); };
log { source(INTAKE); parser(p_et_logmessage); destination(destination_info); };
On Mon, Feb 6, 2012 at 11:07 AM, T. A. Smooth <catdaaaady@gmail.com> wrote:
I can only assume I am not implementing this correctly. :-)
But I have a parser I am trying to use so I can take a subset of the
information of a message and send that subset to another receiver.
This is the whole message:
<13>Feb 4 18:40:17 myhost syslogng: 2012-02-04T18:40:17-08:00 myhostserver-http /tmp/logs/access_log Hi Mom
What I want to do is send out the message as :
<13>Feb 4 18:40:17 myhost syslogng: Hi Mom
Notice how I dropped the middle part out.
From what I have read, the parser acts on the message body alone. Is this correct? So I set it up to look for four(4) columns of data and to be "greedy" on the last column.
I have played around with the number of columns and even used a rewrite function instead. But the Parser continues to produce empty variables. And my template just echos out my default value.
Any thoughts?
parser p_et_logmessage { csv-parser( #columns("ETMSG") #columns("ETMSG.ISODATE") columns("ETMSG.ISODATE", "ETMSG.EASI", "ETMSG.SOURCE",
"ETMSG.BODY")
delimiters(" ") #template("${MSG}") flags(greedy) ); };
rewrite r_rewrite_set{set('${ETMSG.BODY:-nothing}',
value("MESSAGE"));};
template t_et_basic_logmessage { template("${ETMSG.BODY:-nothing}\n");
template_escape(no); };
destination destination_info { tcp("host2" port(8080) template(t_et_basic_logmessage) log_disk_fifo_size(32212254720) ); };
log { source(INTAKE); parser(p_et_logmessage); destination(destination_info); };
My latest Post: Givenchy Fall/Winter 2012 Collection – Runway | Highsnobiety.com Get a signature like this. CLICK HERE.
I think this bug has been corrected in recent versions, but check the changelog/git tree for details. Robert On 02/08/2012 02:40 AM, T. A. Smooth wrote:
I think I found the issue. I define the parser once, but i use it in two different log statments.
When i use it twice the parser places blanks in the custom columns. If i only use it once everything works and the custom columns have the right values.
Expect behavior?
On Feb 6, 2012 11:37 AM, "T. A. Smooth" <catdaaaady@gmail.com <mailto:catdaaaady@gmail.com>> wrote:
Okay this is really weird. Sorry again. I pasted contents of the email here with the configuration . Hopefully this is not too much of a inconvenience.
On Mon, Feb 6, 2012 at 11:27 AM, T. A. Smooth <catdaaaady@gmail.com <mailto:catdaaaady@gmail.com>> wrote: > Looks like my last email was chopped off . > Here it is again. > ################ > > I can only assume I am not implementing this correctly. :-) > > But I have a parser I am trying to use so I can take a subset of the > information of a message and send that subset to another receiver. > This is the whole message: > > <13>Feb 4 18:40:17 myhost syslogng: 2012-02-04T18:40:17-08:00 > myhostserver-http /tmp/logs/access_log Hi Mom > > What I want to do is send out the message as : > > <13>Feb 4 18:40:17 myhost syslogng: Hi Mom > > Notice how I dropped the middle part out. > > From what I have read, the parser acts on the message body alone. Is > this correct? > So I set it up to look for four(4) columns of data and to be "greedy" > on the last column. > > I have played around with the number of columns and even used a > rewrite function instead. But the Parser continues to produce empty > variables. And my template just echos out my default value. > > Any thoughts? > > > > > > parser p_et_logmessage { > csv-parser( > #columns("ETMSG") > #columns("ETMSG.ISODATE") > columns("ETMSG.ISODATE", "ETMSG.EASI", "ETMSG.SOURCE", > "ETMSG.BODY") > delimiters(" ") > #template("${MSG}") > flags(greedy) > ); > }; > > rewrite r_rewrite_set{set('${ETMSG.BODY:-nothing}', value("MESSAGE"));}; > > template t_et_basic_logmessage { > template("${ETMSG.BODY:-nothing}\n"); template_escape(no); }; > > > destination destination_info { > tcp("host2" port(8080) > template(t_et_basic_logmessage) > log_disk_fifo_size(32212254720) > ); > }; > > log { > source(INTAKE); > parser(p_et_logmessage); > destination(destination_info); > }; > > > > > > > > > On Mon, Feb 6, 2012 at 11:07 AM, T. A. Smooth <catdaaaady@gmail.com <mailto:catdaaaady@gmail.com>> wrote: >> >> I can only assume I am not implementing this correctly. :-) >> >> But I have a parser I am trying to use so I can take a subset of the information of a message and send that subset to another receiver. >> This is the whole message: >> >>> <13>Feb 4 18:40:17 myhost syslogng: 2012-02-04T18:40:17-08:00 myhostserver-http /tmp/logs/access_log Hi Mom >> >> >> What I want to do is send out the message as : >> >>> <13>Feb 4 18:40:17 myhost syslogng: Hi Mom >> >> >> Notice how I dropped the middle part out. >> >> From what I have read, the parser acts on the message body alone. Is this correct? >> So I set it up to look for four(4) columns of data and to be "greedy" on the last column. >> >> I have played around with the number of columns and even used a rewrite function instead. But the Parser continues to produce empty variables. And my template just echos out my default value. >> >> Any thoughts? >> >> >>> >>> >>> >>> parser p_et_logmessage { >>> csv-parser( >>> #columns("ETMSG") >>> #columns("ETMSG.ISODATE") >>> columns("ETMSG.ISODATE", "ETMSG.EASI", "ETMSG.SOURCE", "ETMSG.BODY") >>> delimiters(" ") >>> #template("${MSG}") >>> flags(greedy) >>> ); >>> }; >> >> >>> >>> rewrite r_rewrite_set{set('${ETMSG.BODY:-nothing}', value("MESSAGE"));}; >>> >>> template t_et_basic_logmessage { >>> template("${ETMSG.BODY:-nothing}\n"); template_escape(no); }; >>> >>> >>> destination destination_info { >>> tcp("host2" port(8080) >>> template(t_et_basic_logmessage) >>> log_disk_fifo_size(32212254720) >>> ); >>> }; >>> >>> log { >>> source(INTAKE); >>> parser(p_et_logmessage); >>> destination(destination_info); >>> }; >> >> >> >> >> My latest Post: Givenchy Fall/Winter 2012 Collection – Runway | Highsnobiety.com >> Get a signature like this. CLICK HERE.
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
"T. A. Smooth" <catdaaaady@gmail.com> writes:
I think I found the issue. I define the parser once, but i use it in two different log statments.
As Robert said, this is a known, and corrected bug. syslog-ng 3.3.4 should have it fixed. -- |8]
Hmm. I am running my tests in 4.1.3-1.rhel5 Similar bug here? On Feb 8, 2012 1:46 AM, "Gergely Nagy" <algernon@balabit.hu> wrote:
"T. A. Smooth" <catdaaaady@gmail.com> writes:
I think I found the issue. I define the parser once, but i use it in two different log statments.
As Robert said, this is a known, and corrected bug. syslog-ng 3.3.4 should have it fixed.
-- |8]
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Syslogng PE On Feb 8, 2012 11:10 AM, "T. A. Smooth" <catdaaaady@gmail.com> wrote:
Hmm. I am running my tests in 4.1.3-1.rhel5
Similar bug here? On Feb 8, 2012 1:46 AM, "Gergely Nagy" <algernon@balabit.hu> wrote:
"T. A. Smooth" <catdaaaady@gmail.com> writes:
I think I found the issue. I define the parser once, but i use it in two different log statments.
As Robert said, this is a known, and corrected bug. syslog-ng 3.3.4 should have it fixed.
-- |8]
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Found git commits related to it back in December and January. Must not have bubbled down steam yet. On Feb 8, 2012 11:14 AM, "T. A. Smooth" <catdaaaady@gmail.com> wrote:
Syslogng PE On Feb 8, 2012 11:10 AM, "T. A. Smooth" <catdaaaady@gmail.com> wrote:
Hmm. I am running my tests in 4.1.3-1.rhel5
Similar bug here? On Feb 8, 2012 1:46 AM, "Gergely Nagy" <algernon@balabit.hu> wrote:
"T. A. Smooth" <catdaaaady@gmail.com> writes:
I think I found the issue. I define the parser once, but i use it in two different log statments.
As Robert said, this is a known, and corrected bug. syslog-ng 3.3.4 should have it fixed.
-- |8]
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
On Wed, 2012-02-08 at 12:41 -0800, T. A. Smooth wrote:
Found git commits related to it back in December and January. Must not have bubbled down steam yet.
yup, but you can get engineering releases from BalaBit support to have them fixed. They should show up in 4.2.2 AFAIK.
On Feb 8, 2012 11:14 AM, "T. A. Smooth" <catdaaaady@gmail.com> wrote: Syslogng PE
On Feb 8, 2012 11:10 AM, "T. A. Smooth" <catdaaaady@gmail.com> wrote: Hmm. I am running my tests in 4.1.3-1.rhel5
Similar bug here?
On Feb 8, 2012 1:46 AM, "Gergely Nagy" <algernon@balabit.hu> wrote: "T. A. Smooth" <catdaaaady@gmail.com> writes:
> I think I found the issue. I define the parser once, but i use it in two > different log statments.
As Robert said, this is a known, and corrected bug. syslog-ng 3.3.4 should have it fixed.
-- |8]
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Bazsi
participants (4)
-
Balazs Scheidler
-
Fekete Robert
-
Gergely Nagy
-
T. A. Smooth