proper filtering messages with java Exceptions
Hello I have a problem with filtering messages having java Exceptions. I send logs from many jvm processes to one syslog-ng server and save them to different files. Example one message from log4j (dump from wireshark) <139>Jun 18 10:53:03 td2/td2 [PROGRAM NAME - MainThread] ERROR - FATAL ERROR <139>Jun 18 10:53:03 td2/td2 java.lang.Exception: Invalid value TryCount: 0 <139>Jun 18 10:53:03 td2/td2 at com.td.Sender.initialize(Sender.java:30) <139>Jun 18 10:53:03 td2/td2 td2 at com.td.Main.main(Main.java:123) I use changed thread name (PROGRAM NAME) as filter in syslog-ng configuration filter f_td_p1 { match("PROGRAM NAME"); }; but in this configuration ony first line of message are saved (from log file) 2010-06-18 10:53:03 td2 [PROGRAM NAME-MainThread] ERROR - FATAL ERROR Is it possible to correct save all logs ? Of course all messages in examle is one log message. Log4j save it like this: 2010-06-18 10:53:03,242 [PROGRAM NAME-MainThread-MainThread] ERROR - FATAL ERROR java.lang.Exception: Invalid value TryCount: 0 at com.td.Sender.initialize(Sender.java:30) at com.td.Main.main(Main.java:123) I want to obtain same effect in syslog-ng. Have you any suggestion ? Please help. Regards Frank
Hi, You cannot do that now, because syslog-ng processes every line as a new message. In PE 3.2 (coming soon) you can give regular expression to specify multiple lines as a message, so it will solve these problems (like java logs, tomcat logs and so on). As far as I know this feature will also be available in OSE 3.2 (but I am not sure in this) Franciszek Zmarły wrote:
Hello
I have a problem with filtering messages having java Exceptions. I send logs from many jvm processes to one syslog-ng server and save them to different files.
Example one message from log4j (dump from wireshark) <139>Jun 18 10:53:03 td2/td2 [PROGRAM NAME - MainThread] ERROR - FATAL ERROR <139>Jun 18 10:53:03 td2/td2 java.lang.Exception: Invalid value TryCount: 0 <139>Jun 18 10:53:03 td2/td2 at com.td.Sender.initialize(Sender.java:30) <139>Jun 18 10:53:03 td2/td2 td2 at com.td.Main.main(Main.java:123)
I use changed thread name (PROGRAM NAME) as filter in syslog-ng configuration filter f_td_p1 { match("PROGRAM NAME"); };
but in this configuration ony first line of message are saved (from log file) 2010-06-18 10:53:03 td2 [PROGRAM NAME-MainThread] ERROR - FATAL ERROR
Is it possible to correct save all logs ? Of course all messages in examle is one log message. Log4j save it like this: 2010-06-18 10:53:03,242 [PROGRAM NAME-MainThread-MainThread] ERROR - FATAL ERROR java.lang.Exception: Invalid value TryCount: 0 at com.td.Sender.initialize(Sender.java:30) at com.td.Main.main(Main.java:123)
I want to obtain same effect in syslog-ng. Have you any suggestion ? Please help.
Regards Frank
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
-- pzolee
On Tue, 2010-06-22 at 14:03 +0200, Zoltán Pallagi wrote:
Hi,
You cannot do that now, because syslog-ng processes every line as a new message.
In PE 3.2 (coming soon) you can give regular expression to specify multiple lines as a message, so it will solve these problems (like java logs, tomcat logs and so on). As far as I know this feature will also be available in OSE 3.2 (but I am not sure in this)
It'll be available in OSE as well, I'm not completely sure however if it's going to 3.2 or 3.3 as of yet. -- Bazsi
On Tue, 2010-06-22 at 18:01 +0200, Balazs Scheidler wrote:
On Tue, 2010-06-22 at 14:03 +0200, Zoltán Pallagi wrote:
Hi,
You cannot do that now, because syslog-ng processes every line as a new message.
In PE 3.2 (coming soon) you can give regular expression to specify multiple lines as a message, so it will solve these problems (like java logs, tomcat logs and so on). As far as I know this feature will also be available in OSE 3.2 (but I am not sure in this)
It'll be available in OSE as well, I'm not completely sure however if it's going to 3.2 or 3.3 as of yet.
Ahh, rereading this mail, this can be somewhat misleading. The reason I don't know if it is going to 3.2 or 3.3 is that I already have plenties of stuff waiting to be released, and I'm not completely sure that I want to add yet another thing. So it is not a commercial/OSE reason, simply a matter of scheduling and decide how much risk it is to integrate it. -- Bazsi
Is it possible to correct save all logs ? Of course all messages in examle is one log message. Log4j save it like this: 2010-06-18 10:53:03,242 [PROGRAM NAME-MainThread-MainThread] ERROR - FATAL ERROR java.lang.Exception: Invalid value TryCount: 0 at com.td.Sender.initialize(Sender.java:30) at com.td.Main.main(Main.java:123)
I want to obtain same effect in syslog-ng. Have you any suggestion ? Please help.
Regards
Frank
Hi Frank, I have had success matching and logging multi-line logs (specifically lighttpd) by using unix-dgram("/dev/log") in my main local source rather than unix-stream("/dev/log"). This unfortunately assumes you are logging to a local syslog-ng client. If you are logging straight from your java app to a (network) syslog-ng server via UDP or TCP, sorry it looks like you'll have to wait for PE 3.2 -- Lance Laursen Demonware Systems Engineer
participants (4)
-
Balazs Scheidler
-
Franciszek Zmarły
-
Lance Laursen
-
Zoltán Pallagi