Syslog-ng OSE multiline messages support
Hello, I have one question, does syslog-ng OSE support multiline parsing logs? i have one applications that send multiline messages and syslog-ng save the log of the first line only. Thanks for your support in adavnce, Daniel
----- Original message -----
Hello,
I have one question, does syslog-ng OSE support multiline parsing logs? i have one applications that send multiline messages and syslog-ng save the log of the first line only.
Thanks for your support in adavnce,
syslog-ng core is multiline aware, however a transport is needed that supports multiline messages. such a transport is udp(), which has other issues. syslog() with either udp, tcp or tls supports multiline messages. similarly unix-dgram should work for locally generated multiline messages. the only missing thing is the ability to read local files and recognize multiline barriers, but Algernon is working on solving this.
Balazs Scheidler <bazsi77@gmail.com> writes:
I have one question, does syslog-ng OSE support multiline parsing logs? i have one applications that send multiline messages and syslog-ng save the log of the first line only.
syslog-ng core is multiline aware, however a transport is needed that supports multiline messages.
such a transport is udp(), which has other issues. syslog() with either udp, tcp or tls supports multiline messages.
similarly unix-dgram should work for locally generated multiline messages.
the only missing thing is the ability to read local files and recognize multiline barriers, but Algernon is working on solving this.
It is progressing nicely, and it will be available in syslog-ng 3.4 if all goes well. I already have indented-multiline support in a state I'm reasonably happy with[1], a more flexible solution will be implemented once a few other pending issues are resolved. [1]: https://github.com/algernon/syslog-ng/tree/feature/3.4/indented-multiline Meanwhile, I'd like to ask what kind of multiline logs does your application produce? Can you show a sample, by any chance? That'd help me make sure that the multiline reader I'm working on will work for all kinds of use-cases. Thanks in advance! -- |8]
This is one example: [parser/msg_parser.c:714]: ERROR: parse_msg: message=<SIP/2.0 CSeq:713601 INVITE Via: SIP/2.0/UDP192.168.1.1;branch=z9hG4bKd3a2.99b5ab56.0;received=192.168.1.1 Rsyslog replaces the \n with the string #015#012 so i keep my message in one line [parser/msg_parser.c:714]: ERROR: parse_msg: message=<SIP/2.0 #015#012CSeq:713601 INVITE#015#012Via: SIP/2.0/UDP192.168.1.1;branch=z9hG4bKd3a2.99b5ab56.0;received=192.168.1.1 But syslog-ng only stores the first line of the message [parser/msg_parser.c:714]: ERROR: parse_msg: message=<SIP/2.0 Regards, Daniel 2012/10/25 Gergely Nagy <algernon@balabit.hu>
Balazs Scheidler <bazsi77@gmail.com> writes:
I have one question, does syslog-ng OSE support multiline parsing logs? i have one applications that send multiline messages and syslog-ng save the log of the first line only.
syslog-ng core is multiline aware, however a transport is needed that supports multiline messages.
such a transport is udp(), which has other issues. syslog() with either udp, tcp or tls supports multiline messages.
similarly unix-dgram should work for locally generated multiline messages.
the only missing thing is the ability to read local files and recognize multiline barriers, but Algernon is working on solving this.
It is progressing nicely, and it will be available in syslog-ng 3.4 if all goes well. I already have indented-multiline support in a state I'm reasonably happy with[1], a more flexible solution will be implemented once a few other pending issues are resolved.
[1]: https://github.com/algernon/syslog-ng/tree/feature/3.4/indented-multiline
Meanwhile, I'd like to ask what kind of multiline logs does your application produce? Can you show a sample, by any chance? That'd help me make sure that the multiline reader I'm working on will work for all kinds of use-cases.
Thanks in advance!
-- |8]
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
and how does this message get to rsyslog? ----- Original message -----
This is one example:
[parser/msg_parser.c:714]: ERROR: parse_msg: message=<SIP/2.0 CSeq:713601 INVITE Via: SIP/2.0/UDP192.168.1.1;branch=z9hG4bKd3a2.99b5ab56.0;received=192.168.1.1
Rsyslog replaces the \n with the string #015#012 so i keep my message in one line
[parser/msg_parser.c:714]: ERROR: parse_msg: message=<SIP/2.0 #015#012CSeq:713601 INVITE#015#012Via: SIP/2.0/UDP192.168.1.1;branch=z9hG4bKd3a2.99b5ab56.0;received=192.168.1.1
But syslog-ng only stores the first line of the message [parser/msg_parser.c:714]: ERROR: parse_msg: message=<SIP/2.0
Regards,
Daniel
2012/10/25 Gergely Nagy <algernon@balabit.hu>
Balazs Scheidler <bazsi77@gmail.com> writes:
I have one question, does syslog-ng OSE support multiline parsing logs? i have one applications that send multiline messages and syslog-ng save the log of the first line only.
syslog-ng core is multiline aware, however a transport is needed that supports multiline messages.
such a transport is udp(), which has other issues. syslog() with either udp, tcp or tls supports multiline messages.
similarly unix-dgram should work for locally generated multiline messages.
the only missing thing is the ability to read local files and recognize multiline barriers, but Algernon is working on solving this.
It is progressing nicely, and it will be available in syslog-ng 3.4 if all goes well. I already have indented-multiline support in a state I'm reasonably happy with[1], a more flexible solution will be implemented once a few other pending issues are resolved.
[1]: https://github.com/algernon/syslog-ng/tree/feature/3.4/indented-multiline
Meanwhile, I'd like to ask what kind of multiline logs does your application produce? Can you show a sample, by any chance? That'd help me make sure that the multiline reader I'm working on will work for all kinds of use-cases.
Thanks in advance!
-- |8]
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (3)
-
Balazs Scheidler
-
Daniel Josue Rodriguez Paiz
-
Gergely Nagy