[syslog-ng] Elasticscearh-http dest wish list

[Attila Szakacs (aszakacs)]

Thanks Fabien, I think I understand now! 🙂

Answering to Russel:

As far as I know it is not possible to change the mapping type of an already created field in an already created index: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html#update-mapping
When started, syslog-ng does not create the index in ES, it relies on ES to create it itself with the default mapping types.
If you want to have an index with custom mappings, you will have to create it yourself, before sending logs to it from syslog-ng.

I can come up with a possible enhancement:
We could give the user an option, to set multiple field mapping types when configuring the elasticsearch-http() destination, and if it is set, syslog-ng will try to create the index with the given mapping types before sending the logs. Although, it does not fit really well with the current implementation of elasticsearch-http(), it might be possible, that we can make it work.

What do you think about this idea? Is this what you are looking for? 🙂

Best regards,
Attila
________________________________
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of Fabien Wernli <wernli at in2p3.fr>
Sent: Monday, September 2, 2019 10:26 AM
To: syslog-ng at lists.balabit.hu <syslog-ng at lists.balabit.hu>
Subject: Re: [syslog-ng] Elasticscearh-http dest wish list

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.


Hi,

On Mon, Sep 02, 2019 at 08:08:03AM +0000, Attila Szakacs (aszakacs) wrote:
> Please correct me, if I misunderstood something.

I think you misunderstood :)
Russel was talking about the ES side of things : ES templates.
The latter define the data types of fields in Elasticsearch.

See
https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.elastic.co%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Findices-templates.html&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C8fa318fdd83541e91e2908d72f7f3cb5%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637030095857078847&sdata=wPPnf6uO4gKDYZT1bmZNwAa1dTeBTcOuvg5UxLLUKEE%3D&reserved=0

______________________________________________________________________________
Member info: https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C8fa318fdd83541e91e2908d72f7f3cb5%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637030095857078847&sdata=nhXDR7qKdda4%2Btxq5PG8%2B3TPWnPLxB5z7v4R%2B%2FuJyKc%3D&reserved=0
Documentation: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C8fa318fdd83541e91e2908d72f7f3cb5%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637030095857078847&sdata=FL8H7deOLn5iDDxURGTz4QYYg2CYcOT5g3DX2NHZftw%3D&reserved=0
FAQ: https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C8fa318fdd83541e91e2908d72f7f3cb5%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637030095857078847&sdata=uFc4qtRKfVG2FLVWWzcz4ndyibDPcfl8lYT3sS8U9zA%3D&reserved=0

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20190902/e83fbbc5/attachment.html>