[syslog-ng] Fwd: Converting multiline text input to concatanated single-line syslog format

[Scheidler, Balázs]

Hi,
This seems to be a pe customer. Helping him publicly is a great way to push
publicity. Basically he is already a reference with a great company name.

Any takers on his problem?

Cheers
Bazsi
---------- Forwarded message ----------
From: "Newport, Brendan (Contractor - Security Operations - Development &
Support)" <Brendan.Newport at lloydsbanking.com>
Date: Sep 29, 2016 15:04
Subject: [syslog-ng] Converting multiline text input to concatanated
single-line syslog format
To: "syslog-ng at lists.balabit.hu" <syslog-ng at lists.balabit.hu>
Cc:

*Classification: Public*
>
> Hi!
>
>
>
> My first posting on the mailing list.
>
>
>
> I’ve run into a snag trying to get multiline logs concatenated onto one
> line and written as syslog-format messages.
>
>
>
> Actually the second part is working fine; I can get the first line
> identified and incorporated into a syslog message, but all subsequent lines
> aren’t included.
>
>
>
> Just for testing I’ve some simple input;
>
>
>
>
>
> ‘IMP-386: ORACLE error 386 encountered
>
> ORA-01017: invalid username/password; logon deniedUsername:
>
> Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 -
> 64bit Production
>
> With the Partitioning, OLAP and Data Mining options
>
>
>
> Export file created by EXPORT:V10.02.01 via conventional path
>
>
>
> Warning: the objects were exported by FALCON, not by you
>
>
>
> import done in WE8ISO8859P1 character set and AL16UTF16 NCHAR character set
>
> IMP-00034: Warning: FromUser "FALBOS" not found in export file’
>
>
>
> What I’m aiming to do is log only the Oracle errors, commencing at the
> string ‘IMP’ and ending only when the next line is found with ‘IMP’ (this
> log only sees errors)
>
>
>
> So in syslog-ng.conf (version 5.0.6b PE)
>
>
>
> source s_table { file("/path to Table.log" multi-line-prefix("IMP")
> multi-line-garbage("set$") flags(no-p
>
> arse) flags(no-multi-line) program_override("Table")
> default-facility(local5) default-priority(info)); };
>
>
>
> Just to try to get things working, I’ve hard-coded the final string
> present in my sample input - ‘set’
>
>
>
> The output is scheduled to go off-server, but for the moment I want to see
> the transformed messages in their own file;
>
>
>
> destination d_table { file("/auditsox/table.log"); };
>
> log { source(s_table); destination(d_table); };
>
>
>
> What I get from the above (in d_table) is;
>
>
>
>
>
> Sep 29 14:07:15 p14425dev022 Table: IMP-386: ORACLE error 386
> encountered     < Ok, first line transformed into syslog messages
>
> ORA-01017: invalid username/password; logon deniedUsername:
>
> Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 -
> 64bit Production
>
> With the Partitioning, OLAP and Data Mining options
>
>
>
> Export file created by EXPORT:V10.02.01 via conventional path
>
>
>
> Warning: the objects were exported by FALCON, not by you
>
>
>
> import done in WE8ISO8859P1 character set and AL16UTF16 NCHAR character
>
> Sep 29 14:07:26 p14425dev022 Table: IMP-00034: Warning: FromUser "FALBOS"
> not found in export file
>
>
>
> _____________________
>
>
>
> The snag is, the lines after the first line in the input aren’t
> concatenated, but are rather individual lines, with an LF.
>
>
>
> So I’m doing something wrong.
>
>
>
> With this source;
>
>
>
> source s_table { file("/path to Table.log " flags(no-multi-line)
> flags(no-parse) program_override("
>
> Table") default-facility(local5) default-priority(info)); };
>
>
>
> I can get every line of input to transform into a syslog message. Not much
> use though!
>
>
>
> Can anyone provide a pointer as to what I’m doing wonky?
>
>
>
> Thanks
>
>
>
>
>
> Brendan
>
> Lloyds Banking Group plc. Registered Office: The Mound, Edinburgh EH1 1YZ.
> Registered in Scotland no. SC95000. Telephone: 0131 225 4555. Lloyds Bank
> plc. Registered Office: 25 Gresham Street, London EC2V 7HN. Registered in
> England and Wales no. 2065. Telephone 0207626 1500. Bank of Scotland plc.
> Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no.
> SC327000. Telephone: 03457 801 801. Cheltenham & Gloucester plc. Registered
> Office: Barnett Way, Gloucester GL4 3RL. Registered in England and Wales
> 2299428. Telephone: 0345 603 1637
>
> Lloyds Bank plc, Bank of Scotland plc are authorised by the Prudential
> Regulation Authority and regulated by the Financial Conduct Authority and
> Prudential Regulation Authority.
>
> Cheltenham & Gloucester plc is authorised and regulated by the Financial
> Conduct Authority.
>
> Halifax is a division of Bank of Scotland plc. Cheltenham & Gloucester
> Savings is a division of Lloyds Bank plc.
>
> HBOS plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in
> Scotland no. SC218813.
>
> This e-mail (including any attachments) is private and confidential and
> may contain privileged material. If you have received this e-mail in error,
> please notify the sender and delete it (including any attachments)
> immediately. You must not copy, distribute, disclose or use any of the
> information in it or any attachments. Telephone calls may be monitored or
> recorded.
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160929/8eebd819/attachment-0001.htm