[SOLVED] Strange issue when forwarding using LEEF template
Hi, I have a strange issue with templates... Maybe some stupid issue on my side, maybe a bug I don't know *<spoiler> Solutions at the end of this </spoiler>* Here is the scenario Logs coming from Windows using Syslog-ng Agent (syslog destination) Logs received using latest OSE (but same issues at the customer with PE6) Logs must be forwarded to Qradar using a special LEEF template like this (there are also tabs in the template). I've tried both network and syslog destination template t_leefwin { template("<${PRI}>${BSDDATE} ${HOST} LEEF:1.0|Microsoft|Windows|2k8r2|${EVENT_ID}| devTime=${R_YEAR}-${R_MONTH}-${R_DAY}T${R_HOUR}:${R_MIN}:${R_SEC}GMT${TZOFFSET} devTimeFormat=yyyy-MM-dd'T'HH:mm:ssz cat=${EVENT_TYPE} sev=${EVENT_LEVEL} resource=${HOST} usrName=${EVENT_USERNAME} application=${EVENT_SOURCE} message=${EVENT_MSG}\n"); }; The forwarded results like this <46> IP-address LEEF:1.0|Microsoft|Windows|2k8r2||devTime=2020-05-28T09:10:02GMT+02:00 devTimeFormat=yyyy-MM-dd'T'HH:mm:ssz cat= sev= resource=172.17.24.174 usrName= application= message= So, the $macro-s are empty except the one which is not from .sdata (resource) I've created a local json and welf destination and I can see the macros there SDATA="[win@18372.4 EVENT_CATEGORY=\"None\" EVENT_FACILITY=\"16\" EVENT_ID=\"4098\" EVENT_LEVEL=\"3\" EVENT_NAME=\"Application\" EVENT_REC_NUM=\"73006\" EVENT_SID=\"S-1-5-18\" EVENT_SID_TYPE=\"User\" EVENT_SOURCE=\"Group Policy Services\" EVENT_TYPE=\"Figyelmeztetés\" EVENT_HOST=\"hostname\" EVENT_USERNAME=\"NT AUTHORITY\\\\SYSTEM\" I remember I used this method years ago to forward logs from syslog-ng agent to Qradar with proper LEEF format and it was ok... but now, some small thing is missing Versions syslog-ng 3 (3.25.1) Config version: 3.25 Installer-Version: 3.25.1 But, we have a similar issue with PE6 at the customer Any idea? --- so after playing with the welf output a bit, I figured out the short version of the macro names are not longer working somehow, but the long ones are ok like this template t_leefwin { template("<${PRI}>${BSDDATE} ${HOST} LEEF:1.0|Microsoft|Windows|2k8r2|${.SDATA.win@18372.4.EVENT_ID }|devTime=${R_YEAR}-${R_MONTH}-${R_DAY}T${R_HOUR}:${R_MIN}:${R_SEC}GMT${TZOFFSET} devTimeFormat=yyyy-MM-dd'T'HH:mm:ssz cat=${.SDATA.win@18372.4.EVENT_TYPE} sev=${.SDATA.win@18372.4.EVENT_LEVEL} resource=${SDATA.win@18372.4.EVENT_HOST} usrName=${.SDATA.win@18372.4.EVENT_USERNAME} application=${.SDATA.win@18372.4.EVENT_SOURCE} message=${MESSAGE}\n"); }; Thanks L:
Hi Laci, The short macro names are only available within the syslog-ng instance that runs the eventlog source. So in this case that would mean the PE 6 Windows Agent, or the already EOL PE server for Windows. As the logs are forwarded to an OSE instance, it will only know those macros/data fields as SDATA values, so that's the reason why you have to use the long SDATA names. You can recreate the short macro names from them via rewrite rules if needed, but I think it's easier to use the long names anyway. I hope that helps in clearing up that question. Best Regards, János -- Janos SZIGETVARI RHCE, License no. 150-053-692 <https://www.redhat.com/rhtapps/verify/?certId=150-053-692> LinkedIn: linkedin.com/in/janosszigetvari __@__˚V˚ Make the switch to open (source) applications, protocols, formats now: - windows -> Linux, iexplore -> Firefox, msoffice -> LibreOffice - msn -> jabber protocol (Pidgin, Google Talk) - mp3 -> ogg, wmv -> ogg, jpg -> png, doc/xls/ppt -> odt/ods/odp Pal, Laszlo <vlad@vlad.hu> ezt írta (időpont: 2020. máj. 28., Cs, 12:40):
Hi,
I have a strange issue with templates... Maybe some stupid issue on my side, maybe a bug I don't know
*<spoiler> Solutions at the end of this </spoiler>*
Here is the scenario
Logs coming from Windows using Syslog-ng Agent (syslog destination) Logs received using latest OSE (but same issues at the customer with PE6) Logs must be forwarded to Qradar using a special LEEF template like this (there are also tabs in the template). I've tried both network and syslog destination
template t_leefwin { template("<${PRI}>${BSDDATE} ${HOST} LEEF:1.0|Microsoft|Windows|2k8r2|${EVENT_ID}| devTime=${R_YEAR}-${R_MONTH}-${R_DAY}T${R_HOUR}:${R_MIN}:${R_SEC}GMT${TZOFFSET} devTimeFormat=yyyy-MM-dd'T'HH:mm:ssz cat=${EVENT_TYPE} sev=${EVENT_LEVEL} resource=${HOST} usrName=${EVENT_USERNAME} application=${EVENT_SOURCE} message=${EVENT_MSG}\n"); };
The forwarded results like this <46> IP-address LEEF:1.0|Microsoft|Windows|2k8r2||devTime=2020-05-28T09:10:02GMT+02:00 devTimeFormat=yyyy-MM-dd'T'HH:mm:ssz cat= sev= resource=172.17.24.174 usrName= application= message=
So, the $macro-s are empty except the one which is not from .sdata (resource)
I've created a local json and welf destination and I can see the macros there
SDATA="[win@18372.4 EVENT_CATEGORY=\"None\" EVENT_FACILITY=\"16\" EVENT_ID=\"4098\" EVENT_LEVEL=\"3\" EVENT_NAME=\"Application\" EVENT_REC_NUM=\"73006\" EVENT_SID=\"S-1-5-18\" EVENT_SID_TYPE=\"User\" EVENT_SOURCE=\"Group Policy Services\" EVENT_TYPE=\"Figyelmeztetés\" EVENT_HOST=\"hostname\" EVENT_USERNAME=\"NT AUTHORITY\\\\SYSTEM\"
I remember I used this method years ago to forward logs from syslog-ng agent to Qradar with proper LEEF format and it was ok... but now, some small thing is missing
Versions syslog-ng 3 (3.25.1) Config version: 3.25 Installer-Version: 3.25.1
But, we have a similar issue with PE6 at the customer
Any idea?
--- so after playing with the welf output a bit, I figured out the short version of the macro names are not longer working somehow, but the long ones are ok like this
template t_leefwin { template("<${PRI}>${BSDDATE} ${HOST} LEEF:1.0|Microsoft|Windows|2k8r2|${.SDATA.win@18372.4.EVENT_ID }|devTime=${R_YEAR}-${R_MONTH}-${R_DAY}T${R_HOUR}:${R_MIN}:${R_SEC}GMT${TZOFFSET} devTimeFormat=yyyy-MM-dd'T'HH:mm:ssz cat=${.SDATA.win@18372.4.EVENT_TYPE} sev=${.SDATA.win@18372.4.EVENT_LEVEL} resource=${SDATA.win@18372.4.EVENT_HOST} usrName=${.SDATA.win@18372.4.EVENT_USERNAME} application=${.SDATA.win@18372.4.EVENT_SOURCE} message=${MESSAGE}\n"); };
Thanks L:
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Thank you. I knew there is some Balabit spirit out there :) L On Thu, May 28, 2020 at 7:06 PM SZIGETVÁRI János <jszigetvari@gmail.com> wrote:
Hi Laci,
The short macro names are only available within the syslog-ng instance that runs the eventlog source. So in this case that would mean the PE 6 Windows Agent, or the already EOL PE server for Windows. As the logs are forwarded to an OSE instance, it will only know those macros/data fields as SDATA values, so that's the reason why you have to use the long SDATA names. You can recreate the short macro names from them via rewrite rules if needed, but I think it's easier to use the long names anyway. I hope that helps in clearing up that question.
Best Regards, János -- Janos SZIGETVARI RHCE, License no. 150-053-692 <https://www.redhat.com/rhtapps/verify/?certId=150-053-692>
LinkedIn: linkedin.com/in/janosszigetvari
__@__˚V˚ Make the switch to open (source) applications, protocols, formats now: - windows -> Linux, iexplore -> Firefox, msoffice -> LibreOffice - msn -> jabber protocol (Pidgin, Google Talk) - mp3 -> ogg, wmv -> ogg, jpg -> png, doc/xls/ppt -> odt/ods/odp
Pal, Laszlo <vlad@vlad.hu> ezt írta (időpont: 2020. máj. 28., Cs, 12:40):
Hi,
I have a strange issue with templates... Maybe some stupid issue on my side, maybe a bug I don't know
*<spoiler> Solutions at the end of this </spoiler>*
Here is the scenario
Logs coming from Windows using Syslog-ng Agent (syslog destination) Logs received using latest OSE (but same issues at the customer with PE6) Logs must be forwarded to Qradar using a special LEEF template like this (there are also tabs in the template). I've tried both network and syslog destination
template t_leefwin { template("<${PRI}>${BSDDATE} ${HOST} LEEF:1.0|Microsoft|Windows|2k8r2|${EVENT_ID}| devTime=${R_YEAR}-${R_MONTH}-${R_DAY}T${R_HOUR}:${R_MIN}:${R_SEC}GMT${TZOFFSET} devTimeFormat=yyyy-MM-dd'T'HH:mm:ssz cat=${EVENT_TYPE} sev=${EVENT_LEVEL} resource=${HOST} usrName=${EVENT_USERNAME} application=${EVENT_SOURCE} message=${EVENT_MSG}\n"); };
The forwarded results like this <46> IP-address LEEF:1.0|Microsoft|Windows|2k8r2||devTime=2020-05-28T09:10:02GMT+02:00 devTimeFormat=yyyy-MM-dd'T'HH:mm:ssz cat= sev= resource=172.17.24.174 usrName= application= message=
So, the $macro-s are empty except the one which is not from .sdata (resource)
I've created a local json and welf destination and I can see the macros there
SDATA="[win@18372.4 EVENT_CATEGORY=\"None\" EVENT_FACILITY=\"16\" EVENT_ID=\"4098\" EVENT_LEVEL=\"3\" EVENT_NAME=\"Application\" EVENT_REC_NUM=\"73006\" EVENT_SID=\"S-1-5-18\" EVENT_SID_TYPE=\"User\" EVENT_SOURCE=\"Group Policy Services\" EVENT_TYPE=\"Figyelmeztetés\" EVENT_HOST=\"hostname\" EVENT_USERNAME=\"NT AUTHORITY\\\\SYSTEM\"
I remember I used this method years ago to forward logs from syslog-ng agent to Qradar with proper LEEF format and it was ok... but now, some small thing is missing
Versions syslog-ng 3 (3.25.1) Config version: 3.25 Installer-Version: 3.25.1
But, we have a similar issue with PE6 at the customer
Any idea?
--- so after playing with the welf output a bit, I figured out the short version of the macro names are not longer working somehow, but the long ones are ok like this
template t_leefwin { template("<${PRI}>${BSDDATE} ${HOST} LEEF:1.0|Microsoft|Windows|2k8r2|${.SDATA.win@18372.4.EVENT_ID }|devTime=${R_YEAR}-${R_MONTH}-${R_DAY}T${R_HOUR}:${R_MIN}:${R_SEC}GMT${TZOFFSET} devTimeFormat=yyyy-MM-dd'T'HH:mm:ssz cat=${.SDATA.win@18372.4.EVENT_TYPE} sev=${.SDATA.win@18372.4.EVENT_LEVEL} resource=${SDATA.win@18372.4.EVENT_HOST} usrName=${.SDATA.win@18372.4.EVENT_USERNAME} application=${.SDATA.win@18372.4.EVENT_SOURCE} message=${MESSAGE}\n"); };
Thanks L:
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
participants (2)
-
Pal, Laszlo
-
SZIGETVÁRI János