setting sequenceId in forwarded log messages read from journald reader
Hi, I am currently trying to find a way to set meta.sequenceId of log messages that have been read from the locally running systemd-journal to forward them to a remote syslog server that expects the logs to contain a sequenceId according to RFC 5424 section 7.3.1. I found that a sequence number could be taken from the __CURSOR field "i=..." of the journald log: # journalctl -o json-pretty -f ... "__CURSOR" : "s=02a7b30ba17b4a43846f265706bd3a70;i=f01;b=ba633698f20848e480bca4e72476e4d3;m=1a355c1d5;t=5ab670340c8ea;x=33389988ef680e7e", ... My problem is that the journal reader does not seem to parse the __CURSOR string when reading from journald logs. Is there a way to get this information into meta.sequenceId of the forwarded log without modifying the systemd-journal module in syslog-ng ? Thank you for any ideas and best regards Peter Vollmer
Hi Peter, I double checked and it indeed seems like the __CURSOR field is not created by the systemd-journal source by default. So it seems very likely that in order to register it one would need to modify the source driver's source code. Best Regards, János -- Janos SZIGETVARI RHCE, License no. 150-053-692 LinkedIn: linkedin.com/in/janosszigetvari __@__˚V˚ Make the switch to open (source) applications, protocols, formats now: - windows -> Linux, iexplore -> Firefox, msoffice -> LibreOffice - msn -> jabber protocol (Pidgin, Google Talk) - mp3 -> ogg, wmv -> ogg, jpg -> png, doc/xls/ppt -> odt/ods/odp Peter Vollmer <peter.vollmer@gmail.com> ezt írta (időpont: 2020. júl. 27., H, 9:23):
Hi, I am currently trying to find a way to set meta.sequenceId of log messages that have been read from the locally running systemd-journal to forward them to a remote syslog server that expects the logs to contain a sequenceId according to RFC 5424 section 7.3.1.
I found that a sequence number could be taken from the __CURSOR field "i=..." of the journald log:
# journalctl -o json-pretty -f ... "__CURSOR" : "s=02a7b30ba17b4a43846f265706bd3a70;i=f01;b=ba633698f20848e480bca4e72476e4d3;m=1a355c1d5;t=5ab670340c8ea;x=33389988ef680e7e", ... My problem is that the journal reader does not seem to parse the __CURSOR string when reading from journald logs. Is there a way to get this information into meta.sequenceId of the forwarded log without modifying the systemd-journal module in syslog-ng ?
Thank you for any ideas and best regards
Peter Vollmer ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hello Peter, I do not know a way to read sequence id without modifying the journal module. (Syslog-ng already do it to persist it's current position in journal. So it is definitely feasible to read it.) I found this documentation about cursors: https://www.freedesktop.org/software/systemd/man/sd_journal_get_cursor.html, and I am worried about this part of the documentation: "The cursor string should be considered opaque and not be parsed by clients." So parsing and using the sequence id as a core functionality will be unwise. However I can imagine a feature behind a configuration option (turned off by default), where syslog-ng will put the whole __CURSOR string into the NVTable, so it can be accessed via a custom parser. Please tell me your opinion about this solution. (Especially: is there a particular reason why you don't want to modify the journal module?) Best regards, Laci ________________________________ From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Peter Vollmer <peter.vollmer@gmail.com> Sent: Monday, July 27, 2020 09:23 To: syslog-ng@lists.balabit.hu <syslog-ng@lists.balabit.hu> Subject: [syslog-ng] setting sequenceId in forwarded log messages read from journald reader CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Hi, I am currently trying to find a way to set meta.sequenceId of log messages that have been read from the locally running systemd-journal to forward them to a remote syslog server that expects the logs to contain a sequenceId according to RFC 5424 section 7.3.1. I found that a sequence number could be taken from the __CURSOR field "i=..." of the journald log: # journalctl -o json-pretty -f ... "__CURSOR" : "s=02a7b30ba17b4a43846f265706bd3a70;i=f01;b=ba633698f20848e480bca4e72476e4d3;m=1a355c1d5;t=5ab670340c8ea;x=33389988ef680e7e", ... My problem is that the journal reader does not seem to parse the __CURSOR string when reading from journald logs. Is there a way to get this information into meta.sequenceId of the forwarded log without modifying the systemd-journal module in syslog-ng ? Thank you for any ideas and best regards Peter Vollmer
participants (3)
-
Laszlo Szemere (lszemere)
-
Peter Vollmer
-
SZIGETVÁRI János