[syslog-ng] Multi-line support issue
Balazs Scheidler
bazsi77 at gmail.com
Sat Aug 3 09:38:49 CEST 2013
Hi,
I have just added regexp based multiline support to the 3.5 version. Just
grab the latest master, recompile, and you'll have these options:
multi-line-mode(regexp) multi-line-prefix(...) multi-line-garbage(...)
On Mon, Jul 22, 2013 at 11:23 PM, Balazs Scheidler <bazsi77 at gmail.com>wrote:
> Sorry, I was on holiday, wo access to emails. It would be nice to see what
> exactly log4j sends to syslog-ng.
>
> Can you make a packet dump using tcpdump/wireshark?
> On Jul 12, 2013 8:16 PM, "Satish Patel" <satish.txt at gmail.com> wrote:
>
>> Tomcat7 log4j sending logs to syslog-ng. I have installed 3.5. look like
>> log4j doesn't know about white space, do you have any experience with that?
>> but in syslog-ng documents they have mention you can use multi-line-prefix
>> to solve this issue but it seem that option doesn't available in 3.5 version
>>
>>
>> On Thu, Jul 11, 2013 at 5:03 PM, Balazs Scheidler <bazsi77 at gmail.com>wrote:
>>
>>> It's abailable in the git repo, Algernon (cc) may have published
>>> binaries.
>>>
>>> For syslog(transport(udp)) you don't need this flag, as UDP supports
>>> multiline just fine. The original sender decides whether it sends the
>>> message with newlines or not. What client sends you messages?
>>> On Jul 11, 2013 6:54 PM, "Satish Patel" <satish.txt at gmail.com> wrote:
>>>
>>>> ah!!! where do i download 3.5 OpenSource? could you please point me
>>>> out.. also in my case i am using UDP port for source so my syntex would be
>>>> like following? right?
>>>>
>>>> source s_tomcat {
>>>> syslog( transport("udp") multi-line-mode(indented));
>>>> };
>>>>
>>>>
>>>> On Thu, Jul 11, 2013 at 12:40 PM, Balazs Scheidler <bazsi77 at gmail.com>wrote:
>>>>
>>>>> My gosh, I incorrectly remembered a number of vital details, sorry for
>>>>> that.
>>>>>
>>>>> The syntax has been changed from the flags format, it's like this:
>>>>>
>>>>> file('tomcat.log' multi-line-mode(indented));
>>>>>
>>>>> I have actually tried this one, however I have one other bad news,
>>>>> this feature missed 3.4 so it's only available in the 3.5 branch. IIRC
>>>>> Algernon already published 3.5 binaries for Debian/Ubuntu distros.
>>>>> On Jul 11, 2013 4:22 PM, "Satish Patel" <satish.txt at gmail.com> wrote:
>>>>>
>>>>>> This is my source declaration and i have put flags which you have
>>>>>> mentioned.
>>>>>>
>>>>>> source s_tomcat {
>>>>>> syslog( transport("udp") flags(indent-multi-line));
>>>>>> };
>>>>>>
>>>>>> I got following error when i am trying to put flags
>>>>>>
>>>>>> Error parsing afsocket, Unknown flag indent-multi-line in
>>>>>> /usr/local/syslog-ng-3.4.2/etc/syslog-ng.conf at line 54, column 33:
>>>>>>
>>>>>> syslog( transport("udp") flags(indent-multi-line) );
>>>>>> ^^^^^^^^^^^^^^^^^
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jul 11, 2013 at 7:53 AM, Balazs Scheidler <bazsi at balabit.hu>wrote:
>>>>>>
>>>>>>>
>>>>>>> I can't see the source declaration, it must be something along the
>>>>>>> lines
>>>>>>> of:
>>>>>>>
>>>>>>> source s_tomcat {
>>>>>>> file("/var/log/tomcat/xxx.log" flags(indent-multi-line));
>>>>>>> };
>>>>>>>
>>>>>>> On Wed, 2013-07-10 at 12:54 -0400, Satish Patel wrote:
>>>>>>> > Hi Balazs,
>>>>>>> >
>>>>>>> >
>>>>>>> > what is your thought about my config? did you see?
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> > On Mon, Jul 8, 2013 at 12:30 PM, Satish Patel <
>>>>>>> satish.txt at gmail.com>
>>>>>>> > wrote:
>>>>>>> > This is what i have configured and no luck with it.. can
>>>>>>> you
>>>>>>> > suggest what i am missing?
>>>>>>> >
>>>>>>> > destination d02_tc74_log
>>>>>>> > {
>>>>>>> file("/logs/server1/tomcat7.4/catalina_$YEAR$MONTH$DAY.log"
>>>>>>> > template("$(indent-multi-line ${MESSAGE})\n")
>>>>>>> > template(t_tomcatlog) owner("root") group("root")
>>>>>>> perm(0644)
>>>>>>> > dir_perm(0755) create_dirs(yes)); };
>>>>>>> > filter server1 { host("server1.example.com") };
>>>>>>> > log {
>>>>>>> > source (s_tomcat);
>>>>>>> > filter (server1);
>>>>>>> > filter (tomcat7_4);
>>>>>>> > destination (d02_tc74_log);
>>>>>>> > };
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> > On Mon, Jul 8, 2013 at 12:08 PM, Satish Patel
>>>>>>> > <satish.txt at gmail.com> wrote:
>>>>>>> > How do i use indented-multi-line ? I meant where
>>>>>>> do i
>>>>>>> > configure it? I tried but my syslog-ng doesn't
>>>>>>> > recognizing this option i have syslog-ng 3.3.7
>>>>>>> could
>>>>>>> > you give me example where and how do i check
>>>>>>> whether
>>>>>>> > it is supported or not
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> > On Sat, Jul 6, 2013 at 2:12 AM, Balazs Scheidler
>>>>>>> > <bazsi77 at gmail.com> wrote:
>>>>>>> > This looks.like the format that should be
>>>>>>> > supported by indented-multi-line
>>>>>>> >
>>>>>>> > On Jul 5, 2013 9:33 PM, "Satish Patel"
>>>>>>> > <satish.txt at gmail.com> wrote:
>>>>>>> > Here is my tomcat catalina.out log
>>>>>>> > file sample. See there is a tab
>>>>>>> space
>>>>>>> > in logs
>>>>>>> >
>>>>>>> > 2013-06-27 05:30:00,065
>>>>>>> > [EDISN-Scheduler_Worker-2] ERROR
>>>>>>> > com.example.edisn.sftp.SftpSession
>>>>>>> -
>>>>>>> > Exception attempting to work with
>>>>>>> an
>>>>>>> > SFTP Session: connection is closed
>>>>>>> by
>>>>>>> > foreign host
>>>>>>> > 2013-06-27 05:30:00,066
>>>>>>> > [EDISN-Scheduler_Worker-2] ERROR
>>>>>>> > org.quartz.core.JobRunShell - Job
>>>>>>> > EDISN.CTMS_Upload threw an
>>>>>>> unhandled
>>>>>>> > Exception:
>>>>>>> >
>>>>>>> com.example.edisn.EdisnRuntimeException: Exception attempting to work with
>>>>>>> an SFTP Session: connection is closed by foreign host
>>>>>>> > at
>>>>>>> >
>>>>>>> com.example.edisn.sftp.SftpSession.doSession(SftpSession.java:64)
>>>>>>> > at
>>>>>>> >
>>>>>>> com.example.edisn.EdisnSession.exec(EdisnSession.java:13)
>>>>>>> > at
>>>>>>> >
>>>>>>> com.example.ctms.CtmsScheduledJob.executeInternal(CtmsScheduledJob.java:27)
>>>>>>> > at
>>>>>>> >
>>>>>>> org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86)
>>>>>>> > at
>>>>>>> >
>>>>>>> org.quartz.core.JobRunShell.run(JobRunShell.java:202)
>>>>>>> > at
>>>>>>> > org.quartz.simpl.SimpleThreadPool
>>>>>>> >
>>>>>>> $WorkerThread.run(SimpleThreadPool.java:525)
>>>>>>> > Caused by:
>>>>>>> > com.jcraft.jsch.JSchException:
>>>>>>> > connection is closed by foreign
>>>>>>> host
>>>>>>> > at
>>>>>>> >
>>>>>>> com.jcraft.jsch.Session.connect(Unknown Source)
>>>>>>> > at
>>>>>>> >
>>>>>>> com.jcraft.jsch.Session.connect(Unknown Source)
>>>>>>> > at
>>>>>>> >
>>>>>>> com.example.edisn.sftp.SftpSession.doSession(SftpSession.java:45)
>>>>>>> > ... 5 more
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> > On Fri, Jul 5, 2013 at 3:27 PM,
>>>>>>> Balazs
>>>>>>> > Scheidler <bazsi77 at gmail.com>
>>>>>>> wrote:
>>>>>>> > No, I implemented a
>>>>>>> different
>>>>>>> > multiline style support
>>>>>>> first
>>>>>>> > (that is not in pe), where
>>>>>>> > continuation lines are
>>>>>>> > indicated by indentation,
>>>>>>> like
>>>>>>> > mime.
>>>>>>> >
>>>>>>> > Iirc tomcat has this kind
>>>>>>> of
>>>>>>> > log file. Can you show a
>>>>>>> > sample log entry?
>>>>>>> >
>>>>>>> > The infrastructure for
>>>>>>> > multiline-prefix is also
>>>>>>> there
>>>>>>> > but not added yet.
>>>>>>> >
>>>>>>> > Let me see the sample, I'll
>>>>>>> > tell if the current
>>>>>>> solution
>>>>>>> > works or not.
>>>>>>> >
>>>>>>> > On Jul 5, 2013 8:24 PM,
>>>>>>> > "Satish Patel"
>>>>>>> > <satish.txt at gmail.com>
>>>>>>> wrote:
>>>>>>> > Thanks for reply
>>>>>>> > Balazs,
>>>>>>> >
>>>>>>> >
>>>>>>> > You mean say this
>>>>>>> > feature is
>>>>>>> available
>>>>>>> > in Open Source
>>>>>>> Edition
>>>>>>> > (OSE) 3.4? Once
>>>>>>> after
>>>>>>> > specifying flag
>>>>>>> >
>>>>>>> "indented-multi-line"
>>>>>>> > i can use
>>>>>>> > multi-line-prefix?
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> > On Fri, Jul 5,
>>>>>>> 2013 at
>>>>>>> > 1:26 PM, Balazs
>>>>>>> > Scheidler
>>>>>>> > <bazsi77 at gmail.com
>>>>>>> >
>>>>>>> > wrote:
>>>>>>> > You have
>>>>>>> found
>>>>>>> > the PE
>>>>>>> >
>>>>>>> documentation
>>>>>>> > but I have
>>>>>>> > already
>>>>>>> ported
>>>>>>> > this to the
>>>>>>> > OSE tree
>>>>>>> and
>>>>>>> > has been
>>>>>>> > released as
>>>>>>> > part of
>>>>>>> 3.4.
>>>>>>> >
>>>>>>> > You have to
>>>>>>> > specify
>>>>>>> >
>>>>>>> indented-multi-line as a flag to the file source.
>>>>>>> >
>>>>>>> > On Jul 5,
>>>>>>> 2013
>>>>>>> > 6:28 PM,
>>>>>>> > "Satish
>>>>>>> Patel"
>>>>>>> > <
>>>>>>> satish.txt at gmail.com> wrote:
>>>>>>> >
>>>>>>> > We
>>>>>>> >
>>>>>>> have
>>>>>>> >
>>>>>>> tomcat
>>>>>>> >
>>>>>>> shop
>>>>>>> >
>>>>>>> and at
>>>>>>> >
>>>>>>> everyone know tomcat has a java call trace in logs with tab space but
>>>>>>> syslog-ng doesn't know about it and printing lines as a new line. I have
>>>>>>> read here syslog-ng 3.x does support multi-line logs
>>>>>>> http://www.balabit.com/sites/default/files/documents/syslog-ng-pe-4.0-guides/en/syslog-ng-pe-v4.0-guide-admin-en/html/reference_source_syslog.html
>>>>>>> >
>>>>>>> >
>>>>>>> > But
>>>>>>> >
>>>>>>> does
>>>>>>> >
>>>>>>> this
>>>>>>> >
>>>>>>> feature available in Open Source syslog-ng? If yes then why its not working
>>>>>>> for me?
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> ______________________________________________________________________________
>>>>>>> >
>>>>>>> Member
>>>>>>> >
>>>>>>> info:
>>>>>>> >
>>>>>>> https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>> >
>>>>>>> Documentation:
>>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>>> >
>>>>>>> FAQ:
>>>>>>> >
>>>>>>> http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> ______________________________________________________________________________
>>>>>>> > Member
>>>>>>> info:
>>>>>>> >
>>>>>>> https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>> >
>>>>>>> Documentation:
>>>>>>> >
>>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>>> > FAQ:
>>>>>>> >
>>>>>>> http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> ______________________________________________________________________________
>>>>>>> > Member info:
>>>>>>> >
>>>>>>> https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>> > Documentation:
>>>>>>> >
>>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>>> > FAQ:
>>>>>>> >
>>>>>>> http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> ______________________________________________________________________________
>>>>>>> > Member info:
>>>>>>> >
>>>>>>> https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>> > Documentation:
>>>>>>> >
>>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>>> > FAQ:
>>>>>>> >
>>>>>>> http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> ______________________________________________________________________________
>>>>>>> > Member info:
>>>>>>> >
>>>>>>> https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>> > Documentation:
>>>>>>> >
>>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>>> > FAQ:
>>>>>>> >
>>>>>>> http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> ______________________________________________________________________________
>>>>>>> > Member info:
>>>>>>> >
>>>>>>> https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>> > Documentation:
>>>>>>> >
>>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>>> > FAQ:
>>>>>>> http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>> ______________________________________________________________________________
>>>>>>> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>> > Documentation:
>>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>>> > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>> >
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ______________________________________________________________________________
>>>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>>> Documentation:
>>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ______________________________________________________________________________
>>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>>> Documentation:
>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> ______________________________________________________________________________
>>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>>> Documentation:
>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> ______________________________________________________________________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation:
>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>>>
>>>>
>>>
>>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation:
>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
>>>
>>
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>>
--
Bazsi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20130803/b16c01f8/attachment-0001.htm
More information about the syslog-ng
mailing list