[syslog-ng] R: UDP devices stop logging after network services restart.
Vincenti Francesco
Francesco.Vincenti at acciaiterni.it
Fri Jul 28 14:04:32 UTC 2017
Dear Jànos
Thank you for you answer.
In this moment syslog-ng is up and running, with the “rebel” network devices logging on the original NIC.
Anyway I’m going to send you what you need as attachment.
In ipaddr.txt I changed the IP of the card, for security reason; UDP “rebel” devices are logging on ens32 while they should log (and they logged before network services restart) on ens35.
Cheers.
Francesco Vincenti
RHCSA Area Data Center Open Source, Quality and Security
Aspasiel Divisione della Società
Acciai Speciali Terni S.p.A. con Unico Socio
Strada di Pentima, 3 – 05100 Terni
francesco.vincenti at acciaiterni.it<mailto:francesco.vincenti at acciaiterni.it>
www.aspasiel.it<http://www.aspasiel.it/>
Ufficio: +39 0744 203224
Fax: +39 0744 203444
Da: syslog-ng [mailto:syslog-ng-bounces at lists.balabit.hu] Per conto di SZIGETVÁRI János
Inviato: venerdì 28 luglio 2017 15:27
A: Syslog-ng users' and developers' mailing list
Oggetto: Re: [syslog-ng] UDP devices stop logging after network services restart.
Dear Francesco,
To rule out the possiblity of losing UDP logs, I would ask you to share the output of the following commands:
# cat /proc/net/udp
# netstat -su
# sysctl net.core.rmem_max
# ps auxfw
# top
# lspci
# /usr/sbin/syslog-ng --preprocess-into ~/syslog-ng.pp.conf
# ip addr show
Furthermore, you may find further useful information about sizing UDP buffers and other best practice infomation here:
https://www.balabit.com/documents/syslog-ng-pe-6.0-guides/en/syslog-ng-tutorial-udp-source/html-single/index.html
Another useful infomation is that the net.core.rmem_max kernel parameter has to be at least the double of so-rcvbuf() setting. See man 7 socket for further information.
In higher traffic UDP logging scenarios 256 MB for the rmem_max, and 128 MB for so-rcvbuf() might be useful.
Please share the above information with us so that we can get a better picture of your setup.
Thank you!
Cheers,
János
--
Janos SZIGETVARI
RHCE, License no. 150-053-692<https://www.redhat.com/rhtapps/verify/?certId=150-053-692>
__ at __˚V˚
Make the switch to open (source) applications, protocols, formats now:
- windows -> Linux, iexplore -> Firefox, msoffice -> LibreOffice
- msn -> jabber protocol (Pidgin, Google Talk)
- mp3 -> ogg, wmv -> ogg, jpg -> png, doc/xls/ppt -> odt/ods/odp
2017-07-27 8:50 GMT+02:00 Vincenti Francesco <Francesco.Vincenti at acciaiterni.it<mailto:Francesco.Vincenti at acciaiterni.it>>:
Hello
I have syslog-ng 3.7 installed on a CentOS7 Server with 4G RAM and 4 CPU.
The server is receiving logs from almost 300 devices, either network devices (UDP) and servers (TCP).
The problem I have is the following: to reduce incoming traffic on initial two NICs I added two more NICs to the server, where I sent some UDP devices and it worked immediately, without any issue.
But when I restarted network services (systemctl restart network) the devices stopped to log, they restarted only after I resent them to the initial NIC, without any other action on syslog-ng server.
When I started syslog-ng in debug mode to verify this behavior I saw that these devices ARE NOT considered by syslog-ng at all, they are not present in logs generated by debug, but they REACH the server (tcpdump shows them entering).
I'm getting mad about this issue, any suggestion will be welcome.
Following my configuration files:
- /etc/syslog-ng/syslog-ng.conf
# Note: it also sources additional configuration files (*.conf)
# located in /etc/syslog-ng/conf.d/
options {
flush_lines (0);
time_reopen (10);
log_fifo_size (1000);
chain_hostnames (off);
use_dns (persist_only);
dns-cache-hosts(/etc/hosts);
use_fqdn (no);
create_dirs (yes);
keep_hostname (yes);
owner ("1007");
group ("1007");
dir_owner ("1007");
dir_group ("1007");
dir_perm (0750);
};
source s_sys {
system();
internal();
# udp(ip(0.0.0.0) port(514));
};
destination d_cons { file("/dev/console"); };
destination d_mesg { file("/var/log/messages"); };
destination d_auth { file("/var/log/secure"); };
destination d_mail { file("/var/log/maillog" flush_lines(10)); };
destination d_spol { file("/var/log/spooler"); };
destination d_boot { file("/var/log/boot.log"); };
destination d_cron { file("/var/log/cron"); };
destination d_kern { file("/var/log/kern"); };
destination d_mlal { usertty("*"); };
destination d_auth_loc { network ("127.0.0.1" port (601) ); };
filter f_kernel { facility(kern); };
filter f_default { level(info..emerg) and
not (facility(mail)
or facility(authpriv)
or facility(cron)); };
filter f_auth { facility(authpriv); };
filter f_mail { facility(mail); };
filter f_emergency { level(emerg); };
filter f_news { facility(uucp) or
(facility(news)
and level(crit..emerg)); };
filter f_boot { facility(local7); };
filter f_cron { facility(cron); };
#log { source(s_sys); filter(f_kernel); destination(d_cons); };
log { source(s_sys); filter(f_kernel); destination(d_kern); };
log { source(s_sys); filter(f_default); destination(d_mesg); };
log { source(s_sys); filter(f_auth); destination(d_auth); destination(d_auth_loc); };
log { source(s_sys); filter(f_mail); destination(d_mail); };
log { source(s_sys); filter(f_emergency); destination(d_mlal); };
log { source(s_sys); filter(f_news); destination(d_spol); };
log { source(s_sys); filter(f_boot); destination(d_boot); };
log { source(s_sys); filter(f_cron); destination(d_cron); };
# Source additional configuration files (.conf extension only)
@include "/etc/syslog-ng/conf.d/*.conf"
# vim:ft=syslog-ng:ai:si:ts=4:sw=4:et:
- /etc/syslog-ng/conf.d/network_dev.conf
source s_network_appa
{
network ( ip(0.0.0.0) port (514) transport ("udp") so-rcvbuf (4096000) );
};
# Destinazioni
destination d_network_udp { file("/var/log/syslog-ng/APPARATI/${YEAR}${MONTH}${DAY}/${HOST}/${YEAR}${MONTH}${DAY}_hh${HOUR}_${HOST}" create_dirs(yes) dir_perm(0755) perm(0755) ) ; };
filter f_appa_rete { not host ("SymantecServer") and not host ("part8") and not host ("parti") and not host ("partd") and not host ("part1") and not host ("parte") and not host ("part10") and not host ("part11") and not host ("part12") and not host ("part3") and not host ("part6") and not host ("part8") and not host ("part9") and not host ("parta") and not host ("partc") and not host ("partd") and not host ("partf") and not host ("partg") and not host ("parth") and not host ("partn") and not host ("parto") and not host ("SLITES8736.sercom.aspasiel.it<http://SLITES8736.sercom.aspasiel.it>") and not host ("172.23.112.47") and not host ("SLITES4557.sercom.aspasiel.it<http://SLITES4557.sercom.aspasiel.it>") and not host ("172.23.X.Y") and not host ("SLITES4558.sercom.aspasiel.it<http://SLITES4558.sercom.aspasiel.it>") and not host ("172.23.X.Y") and not host ("SLITES4559.sercom.aspasiel.it<http://SLITES4559.sercom.aspasiel.it>") and not host ("172.23.X.Y") and not host ("SLITES4560.sercom.aspasiel.it<http://SLITES4560.sercom.aspasiel.it>") and not host ("172.23.X.Y") and not host ("SLITES8731.sercom.aspasiel.it<http://SLITES8731.sercom.aspasiel.it>") and not host ("172.23.X.Y") and not host ("SLITES8732.sercom.aspasiel.it<http://SLITES8732.sercom.aspasiel.it>") and not host ("172.23.X.Y") and not host ("SLITES8733.sercom.aspasiel.it<http://SLITES8733.sercom.aspasiel.it>") and not host ("172.23.X.Y") and not host ("SLITES8734.sercom.aspasiel.it<http://SLITES8734.sercom.aspasiel.it>") and not host ("172.23.X.Y") and not host ("SLITES8735.sercom.aspasiel.it<http://SLITES8735.sercom.aspasiel.it>") and not host ("172.23.X.Y") and not host ("slitdd2727"); };
log { source(s_network_appa); filter(f_appa_rete); destination(d_network_udp); };
- /etc/syslog-ng/conf.d/server_TCP.conf
source s_server_sop
{
network ( ip(0.0.0.0) port(601) transport("tcp") max-connections(200) log_fetch_limit(100) log_iw_size(20000) );
};
destination d_server_sop
{
file("/var/log/syslog-ng/SERVER/${YEAR}${MONTH}${DAY}/${HOST}/${YEAR}${MONTH}${DAY}_hh${HOUR}_${HOST}" create_dirs(yes) dir_perm(0755) perm(0755) flush_lines(100) );
};
# Indirizzo le destinazioni.
log { source(s_server_sop); destination(d_server_sop); flags(flow-control); };
I changed the following kernel parameters as follow:
net.core.rmem_max = 4096000
net.core.wmem_max = 4096000
Thanks in advance
Francesco Vincenti
RHCSA Area Data Center Open Source, Quality and Security
Aspasiel Divisione della Società
Acciai Speciali Terni S.p.A. con Unico Socio
Strada di Pentima, 3 – 05100 Terni
francesco.vincenti at acciaiterni.it<mailto:francesco.vincenti at acciaiterni.it>
www.aspasiel.it<http://www.aspasiel.it/>
Ufficio: +39 0744 203224<tel:+39%200744%20203224>
Fax: +39 0744 203444<tel:+39%200744%20203444>
This e-mail and any attachments is a confidential correspondence intended only for use of the individual or entity named above. If you are not the intended recipient or the agent responsible for delivering the message to the intended recipient, you are hereby notified that any disclosure, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the error at the following email address: helpdesk at aspasiel.it<mailto:helpdesk at aspasiel.it> or at Aspasiel Helpdesk Team by phone (phone number +390744203555<tel:+39%200744%20203555>), and then delete this message from your system.
P Please consider our environment and think before you print. Thank you! q
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq
This e-mail and any attachments is a confidential correspondence intended only for use of the individual or entity named above. If you are not the intended recipient or the agent responsible for delivering the message to the intended recipient, you are hereby notified that any disclosure, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify the error at the following email address: helpdesk at aspasiel.it<mailto:helpdesk at aspasiel.it> or at Aspasiel Helpdesk Team by phone (phone number +390744203555), and then delete this message from your system.
P Please consider our environment and think before you print. Thank you! q
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170728/8bdcf279/attachment-0001.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ipaddr.txt
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170728/8bdcf279/attachment-0008.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: lspci.txt
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170728/8bdcf279/attachment-0009.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: netstat-su.txt
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170728/8bdcf279/attachment-0010.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ps-auxfw.txt
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170728/8bdcf279/attachment-0011.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: rmem_max.txt
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170728/8bdcf279/attachment-0012.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: syslog-ng-conf.txt
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170728/8bdcf279/attachment-0013.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: top.txt
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170728/8bdcf279/attachment-0014.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: udp.txt
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170728/8bdcf279/attachment-0015.txt>
More information about the syslog-ng
mailing list