[syslog-ng] Elasticscearh-http dest wish list

[Attila Szakacs (aszakacs)]

Hi Russel,

I can only guess, that I am doing what you are trying to achieve.
I used this ES documentation as my starting point: https://www.elastic.co/guide/en/elasticsearch/reference/current/ip.html
Accordingly, I added the "${SOURCEIP}" nv-pair to the "ip_addr" mapping field of elasticsearch.

Please try to set the "template" option of the elastic-http destination as follows:

destination d_elasticsearch {
  elasticsearch-http(
    url("127.0.0.1:9200/_bulk")
    index("alltilla")
    type("test")
    template("$(format-json --scope rfc5424 --exclude DATE --key ISODATE @timestamp=${ISODATE} ip_addr=${SOURCEIP})")
  );
};

I will talk about this change with the team, as because keep-hostname(), chain-hostname() options, and syslog-ng relays add another layer of complexity to this issue.

Please correct me, if I misunderstood something.

Best regards,
Attila
________________________________
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of Russell Fulton <r.fulton at auckland.ac.nz>
Sent: Sunday, September 1, 2019 7:02 AM
To: syslog-ng at lists.balabit.hu <syslog-ng at lists.balabit.hu>
Subject: [syslog-ng] Elasticscearh-http dest wish list

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.


It would be really nice if nv pairs parsed as IP addresses got pushed to ES with a field mapping of IP rather than text and keyword.

Russell at fulton.nz
______________________________________________________________________________
Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C42f6d8e328f943c163f508d72e99a1e4%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637029109736260847&sdata=1ky4LVw6%2Fj9DwcTx9iLbUQUrlQaQTVQ9mPBvgAvyqek%3D&reserved=0
Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C42f6d8e328f943c163f508d72e99a1e4%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637029109736260847&sdata=iyBAxO1Y%2FHBFYyv4CreO4n3zI6WXr%2FR7mMOEWH1lhZc%3D&reserved=0
FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C42f6d8e328f943c163f508d72e99a1e4%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637029109736260847&sdata=iZeI92QELpbjJsTy7QVCNUO9QFm%2FPcBpJwpJtsf24BE%3D&reserved=0

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20190902/4f851c22/attachment.html>