Hello, I'm trying to authenticate with searchguard. The destination I have specified includes the tls() configuration you suggested: destination d_elastic { elasticsearch-http( url("https://127.0.0.1:9200/_bulk") index("logs-${YEAR}.${MONTH}.${DAY}") type("syslog") client-mode ("transport") tls( ca-file("/path/to/ca.pem") cert-file("/path/to/syslog_ng.crt.pem") key-file("/path/to/syslog_ng.key.pem") peer-verify(yes) ) ); }; My problem now is it still doesn't seem to authenticate or work with elasticsearch. Should I have an entry in the elasticsearch.yml? Searchguard has already been configured for elasticsearch and kibana. Also is your elastic-http-plugin.conf referencing the yml file or the client-mode ("searchguard")? I'm not entirely sure what all needs to be configured. The specific errors I'm seeing are below: [2019-07-10T01:44:39.077952] Server disconnected while preparing messages for sending, trying again; driver='d_elastic#0', location='#buffer:4:3', worker_index='0', time_reopen='60', batch_size='3' [2019-07-10T01:44:39.100211] curl: error sending HTTP request; url='https://127.0.0.1:9200/_bulk', error='Problem with the local SSL certificate', worker_index='3', driver='d_elastic#0', location='#buffer:4:3' [2019-07-10T01:44:39.100230] Target server down, but no alternative server available. Falling back to retrying after time-reopen(); url='https://127.0.0.1:9200/_bulk', worker_index='3', driver='d_elastic#0', location='#buffer:4:3'----- When I check the indices available I do not see anything created for syslog-ng. I feel like its almost configured so I'm pretty excited to get this completed and documented on my end. Thanks again for all the support. Original Message----- From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Fabien Wernli Sent: Wednesday, July 10, 2019 1:20 AM To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] Cannot send Syslog-ng to Elasticsearch Hi, On Tue, Jul 09, 2019 at 09:56:50PM +0000, Allen Olivas wrote:
Ok I've got it configured but now its I think its not building the index and updating elasticsearch because of HTTPS and authentication. I have searchguard set up for elasticsearch and kibana. I'm assuming I need Syslog-ng to use the SSL certs searchguard has in place for elasticsearch.
You can use almost any authentication method supported by Searchguard. We use client certificates for syslog-ng, and here's how the config looks like: destination d_coloss { elasticsearch-http( url("https://node01:9200/_bulk" "https://node02:9200/_bulk") index("syslog-${YEAR}-${MONTH}-${DAY}") time-zone("UTC") type("") workers(4) batch_lines(128) batch_timeout(10000) timeout(100) tls( ca-file("/path/to/ca.pem") cert-file("/path/to/syslog_ng.crt.pem") key-file("/path/to/syslog_ng.key.pem") peer-verify(yes) ) ); }; And here are the searchguard permissions for the syslog-ng user's role: sg_role_syslog_ng: indices: "syslog": "*": - WRITE - CREATE_INDEX - indices:admin/mapping/put cluster: - CLUSTER_COMPOSITE_OPS - cluster:monitor/nodes/info - cluster:monitor/nodes/liveness - cluster:monitor/state ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq