[syslog-ng] [FORGED] [FORGED] [FORGED] [FORGED] errors with elasticsearch_http on 2.21 & 3.22 -- correction

Thu Apr 2 04:11:54 UTC 2020

Thanks Balazs

I managed to get things working again by reinstalling syslog-ng (3.26.1) using the default install prefix (/usr/local)
BTW syslog-ng seems agnostic as to whether there is a - or an _ in the name I have tried both in the conf and both work in this version

my previous practice (which I have used for years) was to use a prefix of /usr/local/syslog-ng-<version> and have a symlink in for syslog-ng in /user/local pointing to the current version.  With in the real install I moved etc to dist-etc and then linked etc to /etc/syslog-ng/
lrwxrwxrwx 1 root root 15 Mar 31 09:02 /usr/local/syslog-ng-3.26.1/etc -> /etc/syslog-ng/

I now have the weird situation where version installed in /usr/local/syslog-ng-3.26.1/ fails but the one in /usr/local works

rful011 at secmgrprd02:~$ /usr/local/syslog-ng-3.26.1/sbin/syslog-ng -s -f ~/short.conf
Error parsing destination statement, destination plugin elasticsearch-http not found in /home/rful011/short.conf:11:3-11:21:
6           network( transport("tcp") flags(no-multi-line) port(1514) keep-alive(yes));
7       };
8
9
10      destination d_elastic {
11---->   elasticsearch-http(
11---->   ^^^^^^^^^^^^^^^^^^

rful011 at secmgrprd02:~$ ls -l /usr/local/syslog-ng-3.26.1/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf
-rw-r--r-- 1 rful011 rful011 1901 Jan 25 02:54 /usr/local/syslog-ng-3.26.1/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf

rful011 at secmgrprd02:~$ /usr/local/sbin/syslog-ng -s -f ~/short.conf
rful011 at secmgrprd02:~$

rful011 at secmgrprd02:~$ ls -l /usr/local/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf
-rw-r--r-- 1 rful011 rful011 1901 Jan 25 02:54 /usr/local/share/syslog-ng/include/scl/elasticsearch/elastic-http.conf

Now comes the really weird bit.

While writing this I decided to try reinstalling my original 3.21 version by simply doing a make install and this works fine!

Which is really weird because that was the first thing I did after I realised things had been broken.  My next step was to install 3.22 and then 3.26 in /usr/local/syslog-ng<version>.  I then deleted all of them and installed 3.26.1 in /usr/local and that worked.

I am happy to spend more time on this if you want to get to the bottom of this even though I have my system back.  Like much of the rest of the world I am stuck at home, alone, under fairly strict lockdown so I am happy to contribute something back to the project!

Russell

On 2/04/2020, at 9:48 AM, Balazs Scheidler <bazsi77 at gmail.com<mailto:bazsi77 at gmail.com>> wrote:

elasticsearch-http is provided by https://github.com/syslog-ng/syslog-ng/blob/master/scl/elasticsearch/elastic-http.conf

Make sure that file is installed. Also the name of the block is using a dash, whereas you were using an underscore. I think it should be all the same as we generally convert lot of things from underscore to dashes but I would check this explicitly.

On Wed, Apr 1, 2020, 20:49 Russell Fulton <r.fulton at auckland.ac.nz> wrote:

On 31/03/2020, at 7:05 PM, Antal Nemes (anemes) <Antal.Nemes at oneidentity.com> wrote:

Just another idea that may give a clue.

If you start syslog-ng with foreground with debug and trace leve (syslog-ng -Fevdt)l, syslog-ng

a couple more data points:

 /usr/local/syslog-ng-3.26.1/sbin/syslog-ng -Fedv -s -f ~/short.conf
 gives no errors and includes

[2020-04-01T15:06:30.869576] Reading shared object for a candidate module; path='/usr/local/syslog-ng-3.26.1/lib/syslog-ng', fname='libhttp.so', module='http'
[2020-04-01T15:06:30.871503] Registering candidate plugin; module='http', context='destination', name='http'

endis with:

[2020-04-01T15:06:30.892770] Starting to read include file; filename='/usr/local/syslog-ng-3.26.1/share/syslog-ng/include/scl/syslogconf/plugin.conf', depth='2'
[2020-04-01T15:06:30.893592] Module loaded and initialized successfully; module='confgen'
[2020-04-01T15:06:30.894031] Finishing include; filename='/usr/local/syslog-ng-3.26.1/share/syslog-ng/include/scl/syslogconf/plugin.conf', depth='2'
[2020-04-01T15:06:30.894188] Finishing include; filename='/usr/local/syslog-ng-3.26.1/etc/scl.conf', depth='1'
[2020-04-01T15:06:30.894717] Module loaded and initialized successfully; module='afsocket'
Error parsing destination statement, destination plugin elasticsearch_http not found in /home/rful011/short.conf:11:3-11:21:
6           network( transport("tcp") flags(no-multi-line) port(1514) keep-alive(yes));
7       };
8
9
10      destination d_elastic {
11---->   elasticsearch_http(
11---->   ^^^^^^^^^^^^^^^^^^

and

rful011 at secmgrprd02:~$ /usr/local/syslog-ng/sbin/syslog-ng -V -s -f ~/short.conf
syslog-ng 3 (3.26.1)
Config version: 3.22
Installer-Version: 3.26.1
Revision:
Compile-Date: Mar 31 2020 08:54:40
Module-Directory: /usr/local/syslog-ng-3.26.1/lib/syslog-ng
Module-Path: /usr/local/syslog-ng-3.26.1/lib/syslog-ng
Include-Path: /usr/local/syslog-ng-3.26.1/share/syslog-ng/include
Available-Modules: add-contextual-data,affile,afprog,afsocket,afstomp,afuser,appmodel,azure-auth-header,basicfuncs,cef,confgen,cryptofuncs,csvparser,timestamp,dbparser,disk-buffer,examples,geoip2-plugin,tfgetent,graphite,hook-commands,http,json-plugin,kvformat,linux-kmsg-format,map-value-pairs,pseudofile,mod-python,snmptrapd-parser,stardate,syslogformat,system-source,tags-parser,xml
Enable-Debug: off
Enable-GProf: off
Enable-Memtrace: off
Enable-IPv6: on
Enable-Spoof-Source: off
Enable-TCP-Wrapper: off
Enable-Linux-Caps: off
Enable-Systemd: off

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20200402/3057a35b/attachment.html>