https://bugzilla.balabit.com/show_bug.cgi?id=279 Martin <hlavacek@gmx.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |hlavacek@gmx.com -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

https://bugzilla.balabit.com/show_bug.cgi?id=279 --- Comment #1 from Gergely Nagy <algernon@balabit.hu> 2014-04-29 12:29:08 --- Without debug symbols, the backtrace is fairly useless for debugging purposes, unfortunately. It would help tremendously, if you could reproduce the problem with a non-stripped binary, so we see the functions in the backtrace. Meanwhile, can I ask what config you use on the host where the segmentation fault happened? Maybe we can figure something out from that... Thanks! -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

https://bugzilla.balabit.com/show_bug.cgi?id=279 --- Comment #3 from Martin <hlavacek@gmx.com> 2014-04-29 13:32:02 --- Reguired configuration: ======================= SERVER(syslog1 (with seg fault)): --------------------------------- On server there is the second instance syslog-ng-remote which is handling central logging from clients. Central config is devided into 2 parts - central logging for remote standard logs (like messages, auth.log, syslog, ...) and for remote application logs (those logs are every hour added to the included conf by script on clients machines). cat /etc/syslog-ng/syslog-ng-remote.conf: @version: 3.4 # $Header: /var/cvsroot/gentoo-x86/app-admin/syslog-ng/files/syslog-ng.conf.gentoo.3.2,v 1.1 2011/01/18 17:44:14 mr_bones_ Exp $ # # Syslog-ng default configuration file for Gentoo Linux options { threaded(yes); chain_hostnames(no); #jmena jednotlivich clientu nebude server overovat keep_hostname(yes); # The default action of syslog-ng is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # how many messages syslog-ng missed (0). stats_freq(3600); # The default action of syslog-ng is to log a MARK line # to the file every 20 minutes. That's seems high for most # people so turn it down to once an hour. Set it to zero # if you don't want the functionality at all. mark_freq(600); # S nastavenim perm, dir_perm, create_dirs, owner pocita vzdalene # logovani owner(root); group(logadm); dir_group(logadm); perm(0640); dir_perm(0750); create_dirs(yes); dns_cache_hosts(/etc/hosts); log_fifo_size(20000000); time_reopen(5); }; source s_local { internal(); }; destination d_local { file("/var/log/syslog-remote" group(logadm)); }; log { source(s_local); destination(d_local);}; #################### # Remote Standard #################### source remote_client_std { tcp( port(5514) max-connections(5000) log_iw_size(1000000) so_keepalive(yes) tls( ca_dir("/etc/syslog-ng/cert/CA") key_file("/etc/syslog-ng/cert/CA/server/server.key") cert_file("/etc/syslog-ng/cert/CA/server/server.cert") peer_verify(required-trusted) ) ); }; filter f_auth { facility(auth); }; filter f_authpriv { facility(auth, authpriv); }; filter f_syslog { not facility(authpriv, mail); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kern { facility(kern); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; filter f_news { facility(news); }; filter f_debug { not facility(auth, authpriv, news, mail); }; filter f_messages { level(info..warn) and not facility(auth, authpriv, mail, news); }; filter f_emergency { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; filter f_avc { message(".*avc: .*"); }; filter f_audit { message("^(\\[.*\..*\] |)audit.*") and not message(".*avc: .*"); }; filter f_pax { message("^(\\[.*\..*\] |)PAX:.*"); }; filter f_grsec { message("^(\\[.*\..*\] |)grsec:.*"); }; destination authlog_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/auth.log" group(logadm)); }; destination _syslog_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/syslog" group(logadm)); }; destination cron_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/cron.log" group(logadm)); }; destination daemon_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/daemon.log" group(logadm)); }; destination kern_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/kern.log" group(logadm)); }; destination lpr_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/lpr.log" group(logadm)); }; destination user_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/user.log" group(logadm)); }; destination uucp_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/uucp.log" group(logadm)); }; destination mail_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/mail.log" group(logadm)); }; destination avc_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/avc.log" group(logadm)); }; destination audit_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/audit.log" group(logadm)); }; destination pax_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/pax.log" group(logadm)); }; destination grsec_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/grsec.log" group(logadm)); }; destination mailinfo_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/mail.info" group(logadm)); }; destination mailwarn_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/mail.warn" group(logadm)); }; destination mailerr_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/mail.err" group(logadm)); }; destination newscrit_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/news/news.crit" group(logadm)); }; destination newserr_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/news/news.err" group(logadm)); }; destination newsnotice_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/news/news.notice" group(logadm)); }; destination debug_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/debug" group(logadm)); }; destination messages_net { file("/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/messages" group(logadm)); }; log { source(remote_client_std); filter(f_authpriv); destination(authlog_net); }; log { source(remote_client_std); filter(f_syslog); destination(_syslog_net); }; log { source(remote_client_std); filter(f_cron); destination(cron_net); }; log { source(remote_client_std); filter(f_daemon); destination(daemon_net); }; log { source(remote_client_std); filter(f_kern); destination(kern_net); }; log { source(remote_client_std); filter(f_lpr); destination(lpr_net); }; log { source(remote_client_std); filter(f_mail); destination(mail_net); }; log { source(remote_client_std); filter(f_user); destination(user_net); }; log { source(remote_client_std); filter(f_uucp); destination(uucp_net); }; log { source(remote_client_std); filter(f_pax); destination(pax_net); }; log { source(remote_client_std); filter(f_grsec); destination(grsec_net); }; log { source(remote_client_std); filter(f_audit); destination(audit_net); }; log { source(remote_client_std); filter(f_avc); destination(avc_net); }; log { source(remote_client_std); filter(f_news); filter(f_crit); destination(newscrit_net); }; log { source(remote_client_std); filter(f_news); filter(f_err); destination(newserr_net); }; log { source(remote_client_std); filter(f_news); filter(f_notice); destination(newsnotice_net); }; log { source(remote_client_std); filter(f_debug); destination(debug_net); }; log { source(remote_client_std); filter(f_messages); destination(messages_net); }; ########################################################################### # Sluzby/aplikace nestandardni, ktere se automaticky generuji skriptem # bez nutnosti upravy syslog serveru ########################################################################### ################ # Remote Apps ################ source remote_client_app { tcp( port(5515) max-connections(5000) log_iw_size(1000000) so_keepalive(yes) tls( ca_dir("/etc/syslog-ng/cert/CA") key_file("/etc/syslog-ng/cert/CA/server/server.key") cert_file("/etc/syslog-ng/cert/CA/server/server.cert") peer_verify(required-trusted) ) ); }; destination dest_remote_file { file( "/var/log/HOSTS/$HOST/$YEAR/$MONTH/$DAY/APPS/$PROGRAM" #template("$DAY/$MONTH/$YEAR $HOUR:$MIN:$SEC [$LEVEL] [$FACILITY] [$PROGRAM] $MSG\n") template_escape(no) #template krery by jen zkopiroval zpravu bez zdanych dalsich informaci od syslogu template("$MSG\n") template_escape(no) group(logadm) dir_group(logadm) ); }; log { source(remote_client_app); destination(dest_remote_file); flags(flow-control); }; syslog CLIENT conf: ------------------- sysmon3 ~ # cat /etc/syslog-ng/syslog-ng.conf @version: 3.4 # $Id: syslog-ng.conf 8 2014-01-25 10:46:30Z root $ # # This config file is globaly managed # DO NOT edit this file localy # for custom setting use included config # # # GLOBAL OPTIONS # options { chain_hostnames(no); stats_freq(43200); mark_freq(3600); create_dirs(yes); dns_cache_hosts(/etc/hosts); time_reopen(5); }; # # DEFAULT LOGGING SOURCES # source src { unix-stream("/dev/log"); internal(); }; source kernsrc { file("/proc/kmsg"); }; # # DEFAULT LOGGING DESTINATIONS # destination authlog { file("/var/log/auth.log"); }; destination _syslog { file("/var/log/syslog"); }; destination cron { file("/var/log/cron.log"); }; destination daemon { file("/var/log/daemon.log"); }; destination kern { file("/var/log/kern.log"); file("/dev/tty12"); }; destination lpr { file("/var/log/lpr.log"); }; destination user { file("/var/log/user.log"); }; destination uucp { file("/var/log/uucp.log"); }; destination mail { file("/var/log/mail.log"); }; destination avc { file("/var/log/avc.log"); }; destination audit { file("/var/log/audit.log"); }; destination pax { file("/var/log/pax.log"); }; destination grsec { file("/var/log/grsec.log"); }; destination mailinfo { file("/var/log/mail.info"); }; destination mailwarn { file("/var/log/mail.warn"); }; destination mailerr { file("/var/log/mail.err"); }; destination newscrit { file("/var/log/news/news.crit"); }; destination newserr { file("/var/log/news/news.err"); }; destination newsnotice { file("/var/log/news/news.notice"); }; destination debug { file("/var/log/debug"); }; destination messages { file("/var/log/messages"); }; destination console { usertty("root"); }; destination console_all { file("/dev/tty12"); }; destination xconsole { pipe("/dev/xconsole"); }; # # DEFAULT LOGGING FILTERS # filter f_auth { facility(auth); }; filter f_authpriv { facility(auth, authpriv); }; filter f_syslog { not facility(authpriv, mail, local0); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; # exclude firewall (tag FWALL) messages filter f_kern { facility(kern) and not message( FWALL ); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; #filter f_ppp { facility(ppp); }; filter f_news { facility(news); }; filter f_debug { not facility(auth, authpriv, news, mail, local0); }; filter f_messages { level(info..warn) and not facility(auth, authpriv, mail, news, local0); }; filter f_emergency { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; filter f_avc { message(".*avc: .*"); }; filter f_audit { message("^(\[.*..*] |)audit.*") and not message(".*avc: .*"); }; filter f_pax { message("^(\[.*..*] |)PAX:.*"); }; filter f_grsec { message("^(\[.*..*] |)grsec:.*"); }; # # DEFAULT LOGGING LOGS # log { source(src); filter(f_authpriv); destination(authlog); }; log { source(src); filter(f_syslog); destination(_syslog); }; log { source(src); filter(f_cron); destination(cron); }; log { source(src); filter(f_daemon); destination(daemon); }; log { source(kernsrc); filter(f_kern); destination(kern); }; log { source(src); filter(f_lpr); destination(lpr); }; log { source(src); filter(f_mail); destination(mail); }; log { source(src); filter(f_user); destination(user); }; log { source(src); filter(f_uucp); destination(uucp); }; log { source(kernsrc); filter(f_pax); destination(pax); }; log { source(kernsrc); filter(f_grsec); destination(grsec); }; log { source(kernsrc); filter(f_audit); destination(audit); }; log { source(kernsrc); filter(f_avc); destination(avc); }; log { source(src); filter(f_news); filter(f_crit); destination(newscrit); }; log { source(src); filter(f_news); filter(f_err); destination(newserr); }; log { source(src); filter(f_news); filter(f_notice); destination(newsnotice); }; log { source(src); filter(f_debug); destination(debug); }; log { source(src); filter(f_messages); destination(messages); }; log { source(src); filter(f_emergency); destination(console); }; log { source(src); destination(console_all); }; # # COROSYNC LOGGING # destination corosync { file("/var/log/corosync.log"); }; filter f_local0 { facility(local0); }; log { source(src); filter(f_local0); destination(corosync); }; # # FIREWALL LOGGING # destination fwall { file("/var/log/fwall.log"); }; filter f_fwall { facility(kern) and message( FWALL ); }; log { source(kernsrc); filter(f_fwall); destination(fwall); }; # # INCLUDE CENTRAL LOGGING CONFIG FILE # include /etc/syslog-ng/syslog-ng.centrlog.conf; # # INCLUDE LOCAL (CUSTOM) LOGGING CONFIG FILE # include /etc/syslog-ng/syslog-ng.local.conf; ---------- sysmon3 ~ # cat /etc/syslog-ng/syslog-ng.centrlog.conf ################################################################################## # CENTRALNI logovani - Standardni aplikace, ktere loguji pres syslog ################################################################################## # definice vzdaleneho syslog serveru, poslu tam veskera data z /dev/log a kern, # ktera s epouzivaji prostandardni aplikace destination remote_server_std { tcp( "syslog1.mobil.cz" port(5514) tls( ca_dir("/etc/syslog-ng/cert/CA") key_file("/etc/syslog-ng/cert/client.key") cert_file("/etc/syslog-ng/cert/client.cert") ) ); }; log { source(src); destination(remote_server_std); }; log { source(kernsrc); destination(remote_server_std); }; ################################################################################## # CENTRALNI logovani - NEStandardni aplikace, ktere se includuji pomoci scriptu # /etc/syslog-ng/syslog-ng.apps.scr - script # /etc/syslog-ng/syslog-ng.apps.conf - includovany konfigurak pro ostatni # /etc/syslog-ng/blacklist.txt - blacklist pro nacitani novych logu z /var/log ################################################################################## destination remote_server_app { tcp( "syslog1.mobil.cz" port(5515) #aplikace maji jiny port tls( ca_dir("/etc/syslog-ng/cert/CA") key_file("/etc/syslog-ng/cert/client.key") cert_file("/etc/syslog-ng/cert/client.cert")) ); }; # INCLUDE config file pro aplikace, ktery je generovany skriptem # zde jsou nastaveny patricne destinace pro jedlotlive logy include /etc/syslog-ng/syslog-ng.apps.conf; ------------ sysmon3 ~ # cat /etc/syslog-ng/syslog-ng.apps.conf source src_apache2/access_log { file(/var/log/apache2/access_log flags(no-parse) ); }; rewrite rewrite_apache2/access_log { set(apache2/access_log, value(PROGRAM)); set(sysmon3, value(HOST)); }; log { source(src_apache2/access_log); rewrite(rewrite_apache2/access_log); destination(remote_server_app); flags(flow-control);}; source src_apache2/ssl_request_log { file(/var/log/apache2/ssl_request_log flags(no-parse) ); }; rewrite rewrite_apache2/ssl_request_log { set(apache2/ssl_request_log, value(PROGRAM)); set(sysmon3, value(HOST)); }; log { source(src_apache2/ssl_request_log); rewrite(rewrite_apache2/ssl_request_log); destination(remote_server_app); flags(flow-control);}; source src_apache2/ssl_access_zabbix_log { file(/var/log/apache2/ssl_access_zabbix_log flags(no-parse) ); }; rewrite rewrite_apache2/ssl_access_zabbix_log { set(apache2/ssl_access_zabbix_log, value(PROGRAM)); set(sysmon3, value(HOST)); }; log { source(src_apache2/ssl_access_zabbix_log); rewrite(rewrite_apache2/ssl_access_zabbix_log); destination(remote_server_app); flags(flow-control);}; source src_apache2/error_log { file(/var/log/apache2/error_log flags(no-parse) ); }; rewrite rewrite_apache2/error_log { set(apache2/error_log, value(PROGRAM)); set(sysmon3, value(HOST)); }; log { source(src_apache2/error_log); rewrite(rewrite_apache2/error_log); destination(remote_server_app); flags(flow-control);}; source src_apache2/ssl_error_zabbix_log { file(/var/log/apache2/ssl_error_zabbix_log flags(no-parse) ); }; rewrite rewrite_apache2/ssl_error_zabbix_log { set(apache2/ssl_error_zabbix_log, value(PROGRAM)); set(sysmon3, value(HOST)); }; log { source(src_apache2/ssl_error_zabbix_log); rewrite(rewrite_apache2/ssl_error_zabbix_log); destination(remote_server_app); flags(flow-control);}; source src_munin/munin-node.log { file(/var/log/munin/munin-node.log flags(no-parse) ); }; rewrite rewrite_munin/munin-node.log { set(munin/munin-node.log, value(PROGRAM)); set(sysmon3, value(HOST)); }; log { source(src_munin/munin-node.log); rewrite(rewrite_munin/munin-node.log); destination(remote_server_app); flags(flow-control);}; source src_zabbix/zabbix_server.log { file(/var/log/zabbix/zabbix_server.log flags(no-parse) ); }; rewrite rewrite_zabbix/zabbix_server.log { set(zabbix/zabbix_server.log, value(PROGRAM)); set(sysmon3, value(HOST)); }; log { source(src_zabbix/zabbix_server.log); rewrite(rewrite_zabbix/zabbix_server.log); destination(remote_server_app); flags(flow-control);}; source src_zabbix/zabbix_agentd.log { file(/var/log/zabbix/zabbix_agentd.log flags(no-parse) ); }; rewrite rewrite_zabbix/zabbix_agentd.log { set(zabbix/zabbix_agentd.log, value(PROGRAM)); set(sysmon3, value(HOST)); }; log { source(src_zabbix/zabbix_agentd.log); rewrite(rewrite_zabbix/zabbix_agentd.log); destination(remote_server_app); flags(flow-control);}; source src_zabbix/zabbix_backup.log { file(/var/log/zabbix/zabbix_backup.log flags(no-parse) ); }; rewrite rewrite_zabbix/zabbix_backup.log { set(zabbix/zabbix_backup.log, value(PROGRAM)); set(sysmon3, value(HOST)); }; log { source(src_zabbix/zabbix_backup.log); rewrite(rewrite_zabbix/zabbix_backup.log); destination(remote_server_app); flags(flow-control);}; source src_wtmp { file(/var/log/wtmp flags(no-parse) ); }; rewrite rewrite_wtmp { set(wtmp, value(PROGRAM)); set(sysmon3, value(HOST)); }; log { source(src_wtmp); rewrite(rewrite_wtmp); destination(remote_server_app); flags(flow-control);}; source src_emerge.log { file(/var/log/emerge.log flags(no-parse) ); }; rewrite rewrite_emerge.log { set(emerge.log, value(PROGRAM)); set(sysmon3, value(HOST)); }; log { source(src_emerge.log); rewrite(rewrite_emerge.log); destination(remote_server_app); flags(flow-control);}; source src_lastlog { file(/var/log/lastlog flags(no-parse) ); }; rewrite rewrite_lastlog { set(lastlog, value(PROGRAM)); set(sysmon3, value(HOST)); }; log { source(src_lastlog); rewrite(rewrite_lastlog); destination(remote_server_app); flags(flow-control);}; source src_fail2ban.log { file(/var/log/fail2ban.log flags(no-parse) ); }; rewrite rewrite_fail2ban.log { set(fail2ban.log, value(PROGRAM)); set(sysmon3, value(HOST)); }; log { source(src_fail2ban.log); rewrite(rewrite_fail2ban.log); destination(remote_server_app); flags(flow-control);}; source src_sysng.log { file(/var/log/sysng.log flags(no-parse) ); }; rewrite rewrite_sysng.log { set(sysng.log, value(PROGRAM)); set(sysmon3, value(HOST)); }; log { source(src_sysng.log); rewrite(rewrite_sysng.log); destination(remote_server_app); flags(flow-control);}; -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

https://bugzilla.balabit.com/show_bug.cgi?id=279 --- Comment #5 from Martin <hlavacek@gmx.com> 2014-04-29 14:50:13 --- (In reply to comment #4)

(In reply to comment #2)

...

I thought that I have recompiled syslog with debug symbols because I have added --enable-debug to my ebuild:

--enable-debug enables debugging features, it does not stop Gentoo from stripping the binary of debugging symbols. As far as I see, you have to add "nostrip" to the "FEATURES" flag (see http://wiki.gentoo.org/wiki/Project:Quality_Assurance/Backtraces#Stripping), and then do a backtrace on the core.

Thanks, it seems that now it is OK. So I will provide additional backtrace when segfault occurs again. syslog1 run # ls -lah /usr/sbin/syslog-ng 32K -rwxr-xr-x 1 root root 30K Apr 29 14:38 /usr/sbin/syslog-ng* -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

https://bugzilla.balabit.com/show_bug.cgi?id=279 Martin <hlavacek@gmx.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |NEEDINFO --- Comment #7 from Martin <hlavacek@gmx.com> 2014-06-23 10:15:13 --- Any news? Martin -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.