[syslog-ng] CentOS syslog-ng using tls won't start via systemd but runs fine manually (sort of)

craig bowser reswob10 at gmail.com
Fri Oct 12 15:40:15 UTC 2018


I'm been trying to figure this out for several days now.

I have Centos 7 installed on two servers and I want to use syslog-ng to
send logs between them securely using TLS.

On both I installed syslog-ng 3.5 using this method:
https://www.syslog-ng.com/community/b/blog/posts/installing-latest-syslog-ng-on-rhel-and-other-rpm-distributions

I have been following these methods as a starting point to set up:

https://www.nineproductions.com/syslog-ng-with-tls-setup-for-centralized-logging/
https://www.linux.com/blog/tls-encryption-and-mutual-authentication-using-syslog-ng-open-source-edition
https://www.flyn.org/notes/logging/index.html



On the receiving server, I have syslog-ng with the following config
file:


----------------------------------------------------------
@version:3.5
@include "scl.conf"

# syslog-ng configuration file.
#
# This should behave pretty much like the original syslog on RedHat. But
# it could be configured a lot smarter.
#
# See syslog-ng(8) and syslog-ng.conf(5) for more information.
#
# Note: it also sources additional configuration files (*.conf)
#       located in /etc/syslog-ng/conf.d/

options {
     flush_lines (0);
     time_reopen (10);
#    log_fifo_size (1000);
#    chain_hostnames (off);
     use_dns (no);
     use_fqdn (no);
     create_dirs (1);
     keep_hostname (yes);
};

source s_sys {
     system();
     internal();
     # udp(ip(0.0.0.0) port(514));
};

source s_net { tcp(ip(10.1.0.2) port(5441) tls(
ca_dir("/etc/syslog-ng/ca.d")
key_file("/etc/syslog-ng/newCA/serverkey.pem")
cert_file("/etc/syslog-ng/newCA/servercert.pem")
peer_verify(optional-untrusted)) ); };

destination d_cons { file("/dev/console"); };
destination d_mesg { file("/var/log/messages"); };
destination d_auth { file("/var/log/secure"); };
destination d_mail { file("/var/log/maillog" flush_lines(10)); };
destination d_spol { file("/var/log/spooler"); };
destination d_boot { file("/var/log/boot.log"); };
destination d_cron { file("/var/log/cron"); };
destination d_kern { file("/var/log/kern"); };
destination d_mlal { usertty("*"); };
destination d_log { file("/opt/syslog/$YEAR$MONTH$DAY-logs.log"
create_dirs(yes) dir_group(logstash) dir_owner(logstash) owner(logstash)
group(logstash));};

filter f_kernel     { facility(kern); };
filter f_default    { level(info..emerg) and
                         not (facility(mail)
                         or facility(authpriv)
                         or facility(cron)); };
filter f_auth       { facility(authpriv); };
filter f_mail       { facility(mail); };
filter f_emergency  { level(emerg); };
filter f_news       { facility(uucp) or
                         (facility(news)
                         and level(crit..emerg)); };
filter f_boot   { facility(local7); };
filter f_cron   { facility(cron); };

#log { source(s_sys); filter(f_kernel); destination(d_cons); };
log { source(s_sys); filter(f_kernel); destination(d_kern); };
log { source(s_sys); filter(f_default); destination(d_mesg); };
log { source(s_sys); filter(f_auth); destination(d_auth); };
log { source(s_sys); filter(f_mail); destination(d_mail); };
log { source(s_sys); filter(f_emergency); destination(d_mlal); };
log { source(s_sys); filter(f_news); destination(d_spol); };
log { source(s_sys); filter(f_boot); destination(d_boot); };
log { source(s_sys); filter(f_cron); destination(d_cron); };

log { source(s_net); destination(d_log); };

# Source additional configuration files (.conf extension only)
@include "/etc/syslog-ng/conf.d/*.conf"


# vim:ft=syslog-ng:ai:si:ts=4:sw=4:et:

------------------------------------------------------

Here is my java verison:

# java -version
openjdk version "1.8.0_181"
OpenJDK Runtime Environment (build 1.8.0_181-b13)
OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)

This is my OS version:

# cat /etc/*release
CentOS Linux release 7.2.1511 (Core)
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

CentOS Linux release 7.2.1511 (Core)
CentOS Linux release 7.2.1511 (Core)



If I try to start syslog-ng via systemctl, this is what happens:


# systemctl start syslog-ng
Job for syslog-ng.service failed because the control process exited with
error code. See "systemctl status syslog-ng.service" and "journalctl
-xe" for details.


# systemctl status syslog-ng.service
● syslog-ng.service - System Logger Daemon
    Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled;
vendor preset: enabled)
    Active: failed (Result: start-limit) since Fri 2018-10-12 11:57:09
EDT; 13s ago
      Docs: man:syslog-ng(8)
   Process: 5706 ExecStart=/usr/sbin/syslog-ng -F -p /var/run/syslogd.pid
(code=exited, status=2)
  Main PID: 5706 (code=exited, status=2)
    Status: "Starting up... (Fri Oct 12 11:57:09 2018"

Oct 12 11:57:09 logger.com systemd[1]: Unit syslog-ng.service entered
failed state.
Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service failed.
Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service holdoff time
over, scheduling restart.
Oct 12 11:57:09 logger.com systemd[1]: start request repeated too
quickly for syslog-ng.service
Oct 12 11:57:09 logger.com systemd[1]: Failed to start System Logger
Daemon.
Oct 12 11:57:09 logger.com systemd[1]: Unit syslog-ng.service entered
failed state.
Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service failed.
Oct 12 11:57:09 logger.com systemd[1]: start request repeated too
quickly for syslog-ng.service
Oct 12 11:57:09 logger.com systemd[1]: Failed to start System Logger
Daemon.
Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service failed.


#journalctl -xe

Oct 12 11:57:08 logger.com polkitd[818]: Registered Authentication Agent
for unix-process:5693:6083335 (system bus name :1.97
[/usr/bin/pkttyagent --notify-fd 5 -
Oct 12 11:57:08 logger.com systemd[1]: Listening on Syslog Socket.
-- Subject: Unit syslog.socket has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit syslog.socket has finished starting up.
--
-- The start-up result is done.
Oct 12 11:57:08 logger.com systemd[1]: Starting Syslog Socket.
-- Subject: Unit syslog.socket has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit syslog.socket has begun starting up.
Oct 12 11:57:08 logger.com systemd[1]: Starting System Logger Daemon...
-- Subject: Unit syslog-ng.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit syslog-ng.service has begun starting up.
Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service: main process
exited, code=exited, status=2/INVALIDARGUMENT
Oct 12 11:57:08 logger.com systemd[1]: Failed to start System Logger
Daemon.
-- Subject: Unit syslog-ng.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit syslog-ng.service has failed.
--
-- The result is failed.
Oct 12 11:57:08 logger.com systemd[1]: Unit syslog-ng.service entered
failed state.
Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service failed.
Oct 12 11:57:08 logger.com polkitd[818]: Unregistered Authentication
Agent for unix-process:5693:6083335 (system bus name :1.97, object path
/org/freedesktop/Poli
Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service holdoff time
over, scheduling restart.
Oct 12 11:57:08 logger.com systemd[1]: Starting System Logger Daemon...
-- Subject: Unit syslog-ng.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit syslog-ng.service has begun starting up.
Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service: main process
exited, code=exited, status=2/INVALIDARGUMENT
Oct 12 11:57:08 logger.com systemd[1]: Failed to start System Logger
Daemon.
-- Subject: Unit syslog-ng.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit syslog-ng.service has failed.
--
-- The result is failed.
Oct 12 11:57:08 logger.com systemd[1]: Unit syslog-ng.service entered
failed state.
Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service failed.
Oct 12 11:57:08 logger.com systemd-journal[494]: Forwarding to syslog
missed 3 messages.
-- Subject: One or more messages could not be forwarded to syslog
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- One or more messages could not be forwarded to the syslog service
-- running side-by-side with journald. This usually indicates that the
-- syslog implementation has not been able to keep up with the speed of
-- messages queued.
Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service holdoff time
over, scheduling restart.
Oct 12 11:57:08 logger.com systemd[1]: Starting System Logger Daemon...
-- Subject: Unit syslog-ng.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit syslog-ng.service has begun starting up.
Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service: main process
exited, code=exited, status=2/INVALIDARGUMENT
Oct 12 11:57:08 logger.com systemd[1]: Failed to start System Logger
Daemon.
-- Subject: Unit syslog-ng.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit syslog-ng.service has failed.
--
-- The result is failed.
Oct 12 11:57:08 logger.com systemd[1]: Unit syslog-ng.service entered
failed state.
Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service failed.
Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service holdoff time
over, scheduling restart.
Oct 12 11:57:09 logger.com systemd[1]: Starting System Logger Daemon...
-- Subject: Unit syslog-ng.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit syslog-ng.service has begun starting up.
Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service: main process
exited, code=exited, status=2/INVALIDARGUMENT
Oct 12 11:57:09 logger.com systemd[1]: Failed to start System Logger
Daemon.
-- Subject: Unit syslog-ng.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit syslog-ng.service has failed.
--
-- The result is failed.
Oct 12 11:57:09 logger.com systemd[1]: Unit syslog-ng.service entered
failed state.
Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service failed.
Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service holdoff time
over, scheduling restart.
Oct 12 11:57:09 logger.com systemd[1]: Starting System Logger Daemon...
-- Subject: Unit syslog-ng.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit syslog-ng.service has begun starting up.
Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service: main process
exited, code=exited, status=2/INVALIDARGUMENT
Oct 12 11:57:09 logger.com systemd[1]: Failed to start System Logger
Daemon.
-- Subject: Unit syslog-ng.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit syslog-ng.service has failed.
--
-- The result is failed.
Oct 12 11:57:09 logger.com systemd[1]: Unit syslog-ng.service entered
failed state.
Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service failed.
Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service holdoff time
over, scheduling restart.
Oct 12 11:57:09 logger.com systemd[1]: start request repeated too
quickly for syslog-ng.service
Oct 12 11:57:09 logger.com systemd[1]: Failed to start System Logger
Daemon.
-- Subject: Unit syslog-ng.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit syslog-ng.service has failed.
--
-- The result is failed.
Oct 12 11:57:09 logger.com systemd[1]: Unit syslog-ng.service entered
failed state.
Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service failed.
Oct 12 11:57:09 logger.com systemd[1]: start request repeated too
quickly for syslog-ng.service
Oct 12 11:57:09 logger.com systemd[1]: Failed to start System Logger
Daemon.
-- Subject: Unit syslog-ng.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit syslog-ng.service has failed.
--
-- The result is failed.
Oct 12 11:57:09 logger.com systemd[1]: Unit syslog.socket entered failed
state.
Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service failed.




But if I start with the command line:






# /usr/sbin/syslog-ng -Fedv -p /var/run/syslogd.pid
Starting to read include file; filename='/etc/syslog-ng/scl.conf',
depth='1'
Global value changed; define='scl-root',
value='/usr/share/syslog-ng/include/scl'
Global value changed; define='include-path',
value='/etc/syslog-ng:/usr/share/syslog-ng/include'
Starting to read include file;
filename='/usr/share/syslog-ng/include/scl/system/plugin.conf',
depth='2'
Module loaded and initialized successfully; module='system-source'
Finishing include;
filename='/usr/share/syslog-ng/include/scl/system/plugin.conf',
depth='2'
Starting to read include file;
filename='/usr/share/syslog-ng/include/scl/pacct/plugin.conf', depth='2'
Reading path for candidate modules; path='//usr/lib64/syslog-ng'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='affile.so', module='affile'
Registering candidate plugin; module='affile', context='source',
name='file', preference='0'
Registering candidate plugin; module='affile', context='source',
name='pipe', preference='0'
Registering candidate plugin; module='affile', context='destination',
name='file', preference='0'
Registering candidate plugin; module='affile', context='destination',
name='pipe', preference='0'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='afprog.so', module='afprog'
Registering candidate plugin; module='afprog', context='source',
name='program', preference='0'
Registering candidate plugin; module='afprog', context='destination',
name='program', preference='0'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='afsocket-notls.so',
module='afsocket-notls'
Registering candidate plugin; module='afsocket-notls', context='source',
name='unix-stream', preference='0'
Registering candidate plugin; module='afsocket-notls',
context='destination', name='unix-stream', preference='0'
Registering candidate plugin; module='afsocket-notls', context='source',
name='unix-dgram', preference='0'
Registering candidate plugin; module='afsocket-notls',
context='destination', name='unix-dgram', preference='0'
Registering candidate plugin; module='afsocket-notls', context='source',
name='tcp', preference='0'
Registering candidate plugin; module='afsocket-notls',
context='destination', name='tcp', preference='0'
Registering candidate plugin; module='afsocket-notls', context='source',
name='tcp6', preference='0'
Registering candidate plugin; module='afsocket-notls',
context='destination', name='tcp6', preference='0'
Registering candidate plugin; module='afsocket-notls', context='source',
name='udp', preference='0'
Registering candidate plugin; module='afsocket-notls',
context='destination', name='udp', preference='0'
Registering candidate plugin; module='afsocket-notls', context='source',
name='udp6', preference='0'
Registering candidate plugin; module='afsocket-notls',
context='destination', name='udp6', preference='0'
Registering candidate plugin; module='afsocket-notls', context='source',
name='syslog', preference='0'
Registering candidate plugin; module='afsocket-notls',
context='destination', name='syslog', preference='0'
Registering candidate plugin; module='afsocket-notls', context='source',
name='network', preference='0'
Registering candidate plugin; module='afsocket-notls',
context='destination', name='network', preference='0'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='afsocket-tls.so',
module='afsocket-tls'
Registering candidate plugin; module='afsocket-tls', context='source',
name='unix-stream', preference='100'
Registering candidate plugin; module='afsocket-tls',
context='destination', name='unix-stream', preference='100'
Registering candidate plugin; module='afsocket-tls', context='source',
name='unix-dgram', preference='100'
Registering candidate plugin; module='afsocket-tls',
context='destination', name='unix-dgram', preference='100'
Registering candidate plugin; module='afsocket-tls', context='source',
name='tcp', preference='100'
Registering candidate plugin; module='afsocket-tls',
context='destination', name='tcp', preference='100'
Registering candidate plugin; module='afsocket-tls', context='source',
name='tcp6', preference='100'
Registering candidate plugin; module='afsocket-tls',
context='destination', name='tcp6', preference='100'
Registering candidate plugin; module='afsocket-tls', context='source',
name='udp', preference='100'
Registering candidate plugin; module='afsocket-tls',
context='destination', name='udp', preference='100'
Registering candidate plugin; module='afsocket-tls', context='source',
name='udp6', preference='100'
Registering candidate plugin; module='afsocket-tls',
context='destination', name='udp6', preference='100'
Registering candidate plugin; module='afsocket-tls', context='source',
name='syslog', preference='100'
Registering candidate plugin; module='afsocket-tls',
context='destination', name='syslog', preference='100'
Registering candidate plugin; module='afsocket-tls', context='source',
name='network', preference='100'
Registering candidate plugin; module='afsocket-tls',
context='destination', name='network', preference='100'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='afsocket.so', module='afsocket'
Registering candidate plugin; module='afsocket', context='source',
name='unix-stream', preference='100'
Registering candidate plugin; module='afsocket', context='destination',
name='unix-stream', preference='100'
Registering candidate plugin; module='afsocket', context='source',
name='unix-dgram', preference='100'
Registering candidate plugin; module='afsocket', context='destination',
name='unix-dgram', preference='100'
Registering candidate plugin; module='afsocket', context='source',
name='tcp', preference='100'
Registering candidate plugin; module='afsocket', context='destination',
name='tcp', preference='100'
Registering candidate plugin; module='afsocket', context='source',
name='tcp6', preference='100'
Registering candidate plugin; module='afsocket', context='destination',
name='tcp6', preference='100'
Registering candidate plugin; module='afsocket', context='source',
name='udp', preference='100'
Registering candidate plugin; module='afsocket', context='destination',
name='udp', preference='100'
Registering candidate plugin; module='afsocket', context='source',
name='udp6', preference='100'
Registering candidate plugin; module='afsocket', context='destination',
name='udp6', preference='100'
Registering candidate plugin; module='afsocket', context='source',
name='syslog', preference='100'
Registering candidate plugin; module='afsocket', context='destination',
name='syslog', preference='100'
Registering candidate plugin; module='afsocket', context='source',
name='network', preference='100'
Registering candidate plugin; module='afsocket', context='destination',
name='network', preference='100'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='afstomp.so', module='afstomp'
Registering candidate plugin; module='afstomp', context='destination',
name='stomp', preference='0'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='afuser.so', module='afuser'
Registering candidate plugin; module='afuser', context='destination',
name='usertty', preference='0'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='basicfuncs.so', module='basicfuncs'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='grep', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='if', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='echo', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='length', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='substr', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='strip', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='sanitize', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='lowercase', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='uppercase', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='replace-delimiter', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='+', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='-', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='*', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='/', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='%', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='ipv4-to-int', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='indent-multi-line', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='context-length', preference='0'
Registering candidate plugin; module='basicfuncs',
context='template-func', name='env', preference='0'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='confgen.so', module='confgen'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='cryptofuncs.so',
module='cryptofuncs'
Registering candidate plugin; module='cryptofuncs',
context='template-func', name='uuid', preference='0'
Registering candidate plugin; module='cryptofuncs',
context='template-func', name='hash', preference='0'
Registering candidate plugin; module='cryptofuncs',
context='template-func', name='sha1', preference='0'
Registering candidate plugin; module='cryptofuncs',
context='template-func', name='sha256', preference='0'
Registering candidate plugin; module='cryptofuncs',
context='template-func', name='sha512', preference='0'
Registering candidate plugin; module='cryptofuncs',
context='template-func', name='md4', preference='0'
Registering candidate plugin; module='cryptofuncs',
context='template-func', name='md5', preference='0'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='csvparser.so', module='csvparser'
Registering candidate plugin; module='csvparser', context='parser',
name='csv-parser', preference='0'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='dbparser.so', module='dbparser'
Registering candidate plugin; module='dbparser', context='parser',
name='db-parser', preference='0'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='linux-kmsg-format.so',
module='linux-kmsg-format'
Registering candidate plugin; module='linux-kmsg-format',
context='format', name='linux-kmsg', preference='0'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='syslogformat.so',
module='syslogformat'
Registering candidate plugin; module='syslogformat', context='format',
name='syslog', preference='0'
Registering candidate plugin; module='syslogformat', context='parser',
name='syslog-parser', preference='0'
Reading shared object for a candidate module;
path='//usr/lib64/syslog-ng', fname='system-source.so',
module='system-source'
Finishing include;
filename='/usr/share/syslog-ng/include/scl/pacct/plugin.conf', depth='2'
Starting to read include file;
filename='/usr/share/syslog-ng/include/scl/syslogconf/plugin.conf',
depth='2'
Module loaded and initialized successfully; module='confgen'
Finishing include;
filename='/usr/share/syslog-ng/include/scl/syslogconf/plugin.conf',
depth='2'
Finishing include; filename='/etc/syslog-ng/scl.conf', depth='1'
Module loaded and initialized successfully; module='afsocket-tls'
Module loaded and initialized successfully; module='affile'
Finishing include; content='source confgen system', depth='1'
Module loaded and initialized successfully; module='afuser'
Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:5]
   Compiling s_sys reference [source] at
[/etc/syslog-ng/syslog-ng.conf:5]
     Compiling s_sys sequence [source] at
[/etc/syslog-ng/syslog-ng.conf:2]
       Compiling #unnamed junction [log] at
[/etc/syslog-ng/syslog-ng.conf:2]
         Compiling #unnamed single [log] at [#buffer:1:1]
         Compiling #unnamed single [log] at [#buffer:2:1]
         Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:2]
   Compiling f_kernel reference [filter] at
[/etc/syslog-ng/syslog-ng.conf:5]
     Compiling f_kernel sequence [filter] at
[/etc/syslog-ng/syslog-ng.conf:4]
       Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:4]
   Compiling d_kern reference [destination] at
[/etc/syslog-ng/syslog-ng.conf:5]
     Compiling d_kern sequence [destination] at
[/etc/syslog-ng/syslog-ng.conf:4]
       Compiling #unnamed junction [log] at
[/etc/syslog-ng/syslog-ng.conf:4]
         Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:4]
Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6]
   Compiling s_sys reference [source] at
[/etc/syslog-ng/syslog-ng.conf:6]
   Compiling f_default reference [filter] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling f_default sequence [filter] at
[/etc/syslog-ng/syslog-ng.conf:4]
       Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:4]
   Compiling d_mesg reference [destination] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling d_mesg sequence [destination] at
[/etc/syslog-ng/syslog-ng.conf:3]
       Compiling #unnamed junction [log] at
[/etc/syslog-ng/syslog-ng.conf:3]
         Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:3]
Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6]
   Compiling s_sys reference [source] at
[/etc/syslog-ng/syslog-ng.conf:6]
   Compiling f_auth reference [filter] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling f_auth sequence [filter] at
[/etc/syslog-ng/syslog-ng.conf:4]
       Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:4]
   Compiling d_auth reference [destination] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling d_auth sequence [destination] at
[/etc/syslog-ng/syslog-ng.conf:3]
       Compiling #unnamed junction [log] at
[/etc/syslog-ng/syslog-ng.conf:3]
         Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:3]
Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6]
   Compiling s_sys reference [source] at
[/etc/syslog-ng/syslog-ng.conf:6]
   Compiling f_mail reference [filter] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling f_mail sequence [filter] at
[/etc/syslog-ng/syslog-ng.conf:5]
       Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:5]
   Compiling d_mail reference [destination] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling d_mail sequence [destination] at
[/etc/syslog-ng/syslog-ng.conf:3]
       Compiling #unnamed junction [log] at
[/etc/syslog-ng/syslog-ng.conf:3]
         Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:3]
Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6]
   Compiling s_sys reference [source] at
[/etc/syslog-ng/syslog-ng.conf:6]
   Compiling f_emergency reference [filter] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling f_emergency sequence [filter] at
[/etc/syslog-ng/syslog-ng.conf:5]
       Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:5]
   Compiling d_mlal reference [destination] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling d_mlal sequence [destination] at
[/etc/syslog-ng/syslog-ng.conf:4]
       Compiling #unnamed junction [log] at
[/etc/syslog-ng/syslog-ng.conf:4]
         Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:4]
Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6]
   Compiling s_sys reference [source] at
[/etc/syslog-ng/syslog-ng.conf:6]
   Compiling f_news reference [filter] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling f_news sequence [filter] at
[/etc/syslog-ng/syslog-ng.conf:5]
       Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:5]
   Compiling d_spol reference [destination] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling d_spol sequence [destination] at
[/etc/syslog-ng/syslog-ng.conf:3]
       Compiling #unnamed junction [log] at
[/etc/syslog-ng/syslog-ng.conf:3]
         Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:3]
Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6]
   Compiling s_sys reference [source] at
[/etc/syslog-ng/syslog-ng.conf:6]
   Compiling f_boot reference [filter] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling f_boot sequence [filter] at
[/etc/syslog-ng/syslog-ng.conf:5]
       Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:5]
   Compiling d_boot reference [destination] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling d_boot sequence [destination] at
[/etc/syslog-ng/syslog-ng.conf:3]
       Compiling #unnamed junction [log] at
[/etc/syslog-ng/syslog-ng.conf:3]
         Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:3]
Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6]
   Compiling s_sys reference [source] at
[/etc/syslog-ng/syslog-ng.conf:6]
   Compiling f_cron reference [filter] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling f_cron sequence [filter] at
[/etc/syslog-ng/syslog-ng.conf:5]
       Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:5]
   Compiling d_cron reference [destination] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling d_cron sequence [destination] at
[/etc/syslog-ng/syslog-ng.conf:3]
       Compiling #unnamed junction [log] at
[/etc/syslog-ng/syslog-ng.conf:3]
         Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:3]
Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6]
   Compiling s_net reference [source] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling s_net sequence [source] at
[/etc/syslog-ng/syslog-ng.conf:3]
       Compiling #unnamed junction [log] at
[/etc/syslog-ng/syslog-ng.conf:3]
         Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:3]
   Compiling d_log reference [destination] at
[/etc/syslog-ng/syslog-ng.conf:6]
     Compiling d_log sequence [destination] at
[/etc/syslog-ng/syslog-ng.conf:4]
       Compiling #unnamed junction [log] at
[/etc/syslog-ng/syslog-ng.conf:4]
         Compiling #unnamed single [log] at
[/etc/syslog-ng/syslog-ng.conf:4]
Module loaded and initialized successfully; module='syslogformat'
Module loaded and initialized successfully; module='linux-kmsg-format'
Running application hooks; hook='1'
Running application hooks; hook='3'
syslog-ng starting up; version='3.5.6'



If I comment out this line from syslog-ng.conf:



# log  { source(s_net); destination(d_log); };

Then restart syslog-ng (I know, I should use restart...)

# systemctl start syslog-ng

Starts no problem.


# systemctl status syslog-ng.service
● syslog-ng.service - System Logger Daemon
    Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled;
vendor preset: enabled)
    Active: active (running) since Fri 2018-10-12 12:00:47 EDT; 38s ago
      Docs: man:syslog-ng(8)
  Main PID: 5727 (syslog-ng)
    CGroup: /system.slice/syslog-ng.service
            └─5727 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid

Oct 12 12:00:47 logger.com systemd[1]: Starting System Logger Daemon...
Oct 12 12:00:47 logger.com systemd[1]: Started System Logger Daemon.


It just won't stop (I guess by design)

# systemctl stop syslog-ng
Warning: Stopping syslog-ng.service, but it can still be activated by:
   syslog.socket
# systemctl status syslog-ng.service
● syslog-ng.service - System Logger Daemon
    Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled;
vendor preset: enabled)
    Active: active (running) since Fri 2018-10-12 12:01:55 EDT; 1s ago
      Docs: man:syslog-ng(8)
  Main PID: 5755 (syslog-ng)
    CGroup: /system.slice/syslog-ng.service
            └─5755 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid

Oct 12 12:01:55 logger.com systemd[1]: Starting System Logger Daemon...
Oct 12 12:01:55 logger.com systemd[1]: Started System Logger Daemon.


But if I uncomment that line and restart:


# systemctl restart syslog-ng
Job for syslog-ng.service failed because the control process exited with
error code. See "systemctl status syslog-ng.service" and "journalctl
-xe" for details.
# service syslog-ng status
Redirecting to /bin/systemctl status  syslog-ng.service
● syslog-ng.service - System Logger Daemon
    Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled;
vendor preset: enabled)
    Active: failed (Result: start-limit) since Fri 2018-10-12 12:04:18
EDT; 2s ago
      Docs: man:syslog-ng(8)
   Process: 5811 ExecStart=/usr/sbin/syslog-ng -F -p /var/run/syslogd.pid
(code=exited, status=2)
  Main PID: 5811 (code=exited, status=2)
    Status: "Starting up... (Fri Oct 12 12:04:18 2018"

Oct 12 12:04:18 logger.com systemd[1]: Unit syslog-ng.service entered
failed state.
Oct 12 12:04:18 logger.com systemd[1]: syslog-ng.service failed.
Oct 12 12:04:18 logger.com systemd[1]: syslog-ng.service holdoff time
over, scheduling restart.
Oct 12 12:04:18 logger.com systemd[1]: start request repeated too
quickly for syslog-ng.service
Oct 12 12:04:18 logger.com systemd[1]: Failed to start System Logger
Daemon.
Oct 12 12:04:18 logger.com systemd[1]: Unit syslog-ng.service entered
failed state.
Oct 12 12:04:18 logger.com systemd[1]: syslog-ng.service failed.
Oct 12 12:04:18 logger.com systemd[1]: start request repeated too
quickly for syslog-ng.service
Oct 12 12:04:18 logger.com systemd[1]: Failed to start System Logger
Daemon.
Oct 12 12:04:18 logger.com systemd[1]: syslog-ng.service failed.


It won't work.

If I change syslog-ng.conf to a specific IP in the source I get nearly
the same results.


One note:  Occasionally, in the journalctl -xe, I get the following:

Oct 12 12:08:55 logger.com systemd[1]: Unit syslog-ng.service entered
failed state.
Oct 12 12:08:55 logger.com systemd[1]: syslog-ng.service failed.
Oct 12 12:08:55 logger.com systemd-journal[494]: Forwarding to syslog
missed 3 messages.
-- Subject: One or more messages could not be forwarded to syslog
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- One or more messages could not be forwarded to the syslog service
-- running side-by-side with journald. This usually indicates that the
-- syslog implementation has not been able to keep up with the speed of
-- messages queued.


But that doesn't show up all the time.

As far as options, it doesn't look like

https://www.centos.org/forums/viewtopic.php?t=58296
https://www.syslog-ng.com/community/b/blog/posts/troubleshooting-java-support-syslog-ng

rsyslog is disabled

# systemctl status rsyslog
● rsyslog.service - System Logging Service
    Loaded: loaded (/usr/lib/systemd/system/rsyslog.service; disabled;
vendor preset: enabled)
    Active: inactive (dead)

Oct 11 19:03:17 logger.com systemd[1]: Starting System Logging
Service...
Oct 11 19:03:17 logger.com systemd[1]: Started System Logging Service.
Oct 11 20:52:56 logger.com systemd[1]: Stopping System Logging
Service...
Oct 11 20:52:56 logger.com systemd[1]: Stopped System Logging Service.

https://www.centos.org/forums/viewtopic.php?t=57379

Nothing seems to be blocking access to the log directory:

# lsof /dev/log
COMMAND   PID USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
systemd     1 root   42u  unix 0xffff8800368952c0      0t0 9277 /dev/log
systemd-j 494 root    5u  unix 0xffff8800368952c0      0t0 9277 /dev/log

https://www.reddit.com/r/linuxadmin/comments/5ezj5c/syslogng_service_doesnt_start_with_boot_but/


I've also looked at
https://unix.stackexchange.com/questions/202044/syslog-ng-service-not-starting-with-systemd-but-command-works-fine

and

https://lists.balabit.hu/pipermail/syslog-ng/2017-September/024024.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20181012/99c91185/attachment-0001.html>


More information about the syslog-ng mailing list