- syslog-ng - lists.balabit.hu

Syslog-ng don't Listening UDP packets PORT(514) - Problem History
by William Luiz Ribeiro Vasconcelos Da Silva 08 Nov '19

08 Nov '19

Good Morning, Yesterday, we performed the first test of syslog-ng 3.24 OSE on a RHEL 7.6 VM in our customer's production environment. And we identified a very strange behavior. We saw through tcpdump that packets were being received by the network card but were not captured by syslog-ng. We have a UDP packet send simulator, and when I was pointing my simulator to another IP on the same machine(the users' connection IP), syslog-ng does the capping and consequently the treatment we put into the syslog-ng.conf file. What kind of TROUBLESHOOTING we can do in order to identify the reason for syslog-ng to capture packets on the users' connection IP, but for the application connection IP, it doesn't identify anything. It is informed by the client at the beginning of our project that syslog should always capture all UDP packets received by the application IP. This is the configuration of the "source" we have on this machine in our client's productive environment. source s_model { #udp(port(514)); udp(ip(10.46.105.34) port(514)); # udp(); #network(transport("udp")); #network( # ip("10.46.105.34") # transport("udp") #); }; Tks, Esta mensagem, incluindo seus anexos, pode conter informações privilegiadas e/ou de caráter confidencial, não podendo ser retransmitida sem autorização do remetente. Se você não é o destinatário ou pessoa autorizada para recebê-la, informamos que o seu uso, divulgação, cópia ou arquivamento são proibidos. Portanto, se você recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conteúdo. This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo, informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto

3 3

Syslog-ng don't Listening UDP packets PORT(514)
by William Luiz Ribeiro Vasconcelos Da Silva 08 Nov '19

08 Nov '19

Hello guys, We are doing a test in a VM RHEL 7.6, and the syslog-ng OSE 3.24, do Not capturing UDP packets. This is my configuration: ######################################################################################### source s_net { #udp(port(514)); udp(ip("10.46.105.34") port(514)); #udp(ip("10.25.17.4") port(514)); # udp(); #network(transport("udp")); #network( # ip("10.46.105.34") # transport("udp") #); }; destination d_dest { file("/syslog_out/${HOST}_${YEAR}${MONTH}${DAY}${HOUR}${MIN}${SEC}_GMT${TZ}_${RCPTID}.log" template("${PROGRAM};${MESSAGE};\n") owner(syslog) group(syslog) perm (0755) ); }; log { source(s_net ); destination(d_dest ); }; ########################################################################################### What kind of configuration i have to do, or what I need to check in OS, to do the capturing UDP packets. Tks, William Luiz Esta mensagem, incluindo seus anexos, pode conter informações privilegiadas e/ou de caráter confidencial, não podendo ser retransmitida sem autorização do remetente. Se você não é o destinatário ou pessoa autorizada para recebê-la, informamos que o seu uso, divulgação, cópia ou arquivamento são proibidos. Portanto, se você recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conteúdo. This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo, informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto

1 0

syslog-ng-stable RPM repo
by Peter Czanik (pczanik) 04 Nov '19

04 Nov '19

Hi, Until now I had a single request for a syslog-ng-stable repo, which always contains the latest stable release of syslog-ng. But the discussion about my cleanup plans brought up the topic again, and Bazsi wrote me, that there were more such requests coming from the Splunk Slack channel. So I did it: * fedora/rhel/centos: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng-stable/ * openSUSE/SLES: https://build.opensuse.org/project/show/home:czanik:syslog-ng-stable In theory it should be the exact copy of my unofficial 3.23.1 packages and some fixes backported for openSUSE Tumbleweed. New syslog-ng versions will first appear in versioned repositories, as usual. I plan to push packages to the stable repo only after a week or two delay, so there is time find major bugs, have a .2 release if necessary. Once we have more experience that it works as expected, I'll update my blogs and the syslog-ng website. Until that I'm waiting for your feedback, especially once 3.24.1 comes out. Peter Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

2 2

GLib missing when running compilation with cmake
by Alexandre Santos 04 Nov '19

04 Nov '19

Hi All, I got the following error, when compiling with cmake: ... -- Found PkgConfig: /usr/bin/pkg-config (found version "0.29") CMake Error at /usr/share/cmake-3.7/Modules/FindPackageHandleStandardArgs.cmake:138 (message): Could NOT find GLIB (missing: GLIB_INCLUDE_DIRS GLIB_LIBRARIES GLIB_GMODULE_LIBRARIES GLIB_GTHREAD_LIBRARIES) Call Stack (most recent call first): /usr/share/cmake-3.7/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE) cmake/Modules/FindGLIB.cmake:111 (FIND_PACKAGE_HANDLE_STANDARD_ARGS) CMakeLists.txt:168 (find_package) -- Configuring incomplete, errors occurred! See also "/thanos/src/syslog-ng/build/CMakeFiles/CMakeOutput.log". See also "/thanos/src/syslog-ng/build/CMakeFiles/CMakeError.log". Nevertheless, I have the following GLibs installed: root@5526d87140af:/asantos/src/syslog-ng-git# apt list libglib* Listing... Done libglib-object-introspection-perl/oldstable 0.042-1 amd64 libglib-perl/oldstable 3:1.324-1 amd64 libglib2.0-0/oldstable,now 2.50.3-2+deb9u1 amd64 [installed,automatic] libglib2.0-0-dbg/oldstable 2.50.3-2+deb9u1 amd64 libglib2.0-bin/oldstable 2.50.3-2+deb9u1 amd64 libglib2.0-cil/oldstable 2.12.40-2 amd64 libglib2.0-cil-dev/oldstable 2.12.40-2 amd64 libglib2.0-data/oldstable,now 2.50.3-2+deb9u1 all [installed,automatic] libglib2.0-dev/oldstable 2.50.3-2+deb9u1 amd64 libglib2.0-doc/oldstable 2.50.3-2+deb9u1 all libglib2.0-tests/oldstable 2.50.3-2+deb9u1 amd64 libglib3.0-cil/oldstable 2.99.3-2+b1 amd64 libglib3.0-cil-dev/oldstable 2.99.3-2+b1 amd64 libglibmm-2.4-1v5/oldstable 2.50.0-1 amd64 libglibmm-2.4-dev/oldstable 2.50.0-1 amd64 libglibmm-2.4-doc/oldstable 2.50.0-1 all root@5526d87140af:/asantos/src/syslog-ng-git# Can you explain me what is happening? Thanks, Alex

2 2

email alert on timeout
by Gregg Nicholas 31 Oct '19

31 Oct '19

I'm new to syslog-ng OSE. I've been able to follow the samples, and configured an email alert when a syslog message is received. Now I'm trying to configure an email alert when it has been too long since I've received a particular syslog message. First, I configured a destination: destination d_heartbeat { file("/var/log/heartbeat.log" mark-freq(60) mark-mode(dst-idle) ); }; Then, I configured a filter and log: filter f_heartbeat { match("I am still here" value("MESSAGE) ); }; Log { source(s_network); filter(f_heartbeat); destination(d_heartbeat); }; I was pleasantly surprised that this worked. The heartbeat.log file received the message, and 60 seconds later it received the "syslog -- MARK --" Since I wanted an email alert, I defined a new source, to grab the output of my heartbeat.log as input: source s_heartbeat { file(""/var/log/heartbeat.log"); }; destination d_smtp { smtp( host("10.10.10.25") port(25) from("syslog-ng" "noreply(a)mydomain.com <mailto:noreply@mydomain.com> ") to("me" "myemail(a)mydomain.com <mailto:myemail@mydomain.com> ") subject("host is down") body("no heartbeat received from the program on host\n") ); }; filter f_timeout { match ("MARK --" value("MESSAGE")); }; log { source(s_heartbeat); filter(f_timeout); destination(d_smtp); }; This seemed to work when I tested it with a single heartbeat followed by timeout, but I'm still misunderstanding something. If it gets more than one message written to heartbeat.log, then the timeout MARK doesn't happen. What am I missing? Is there a better way to accomplish this alert? Thanks, Gregg (running syslog-ng v3.19.1-5 on debian 10)

3 3

syslog-ng 3.22.1 consuming a lot of RAM on amqp or kafka destinations
by joao reis 24 Oct '19

24 Oct '19

Hi, I'm having high RAM consuming problems using both amqp or kafka destinations. I'm using patterndb (without any context definition) some basic filters. Changing the destination to "file" or "program" type, the high RAM problem doesn't happen. I tested these backends only in this version of syslog-ng and I don't know if it is a version specific problem. The backends are up and running and receiving correctly from syslog-ng. Here are my configurations: destination d_amqp { amqp( vhost("/email_log") host("127.0.0.1") port(5672) exchange("maillog") username("maillog") password("maillog") disk-buffer( mem-buf-size(10000) disk-buf-size(2000000) reliable(yes) dir("/tmp/disk-buffer") ) exchange-type("fanout") routing-key("") persistent(yes) value-pairs( scope("nv-pairs") ) ); }; destination d_kafka { kafka( bootstrap-servers("127.0.0.1:9092") topic("MYTOPIC") ); }; I added the disk-buffer option to see if it gets any better, but without any good result; Here the syslog-ng-ctl output, when configured to amqp: [root@logserver3 conf.d]# syslog-ng-ctl stats SourceName;SourceId;SourceInstance;State;Type;Number global;payload_reallocs;;a;processed;3635299 src.journald;s_sys#0;journal;a;processed;542 src.journald;s_sys#0;journal;a;stamp;1566648214 global;sdata_updates;;a;processed;0 global;scratch_buffers_bytes;;a;queued;512 source;s_host;;a;processed;5440633 destination;d_boot;;a;processed;0 destination;d_kern;;a;processed;9 source;s_sys;;a;processed;612 dst.amqp;d_amqp#0;amqp,/email_log,127.0.0.1,5672,maillog,fanout;a;dropped;0 dst.amqp;d_amqp#0;amqp,/email_log,127.0.0.1,5672,maillog,fanout;a;processed;2691676 dst.amqp;d_amqp#0;amqp,/email_log,127.0.0.1,5672,maillog,fanout;a;queued;0 dst.amqp;d_amqp#0;amqp,/email_log,127.0.0.1,5672,maillog,fanout;a;written;2691676 src.internal;s_sys#1;;a;processed;70 src.internal;s_sys#1;;a;stamp;1566648339 global;msg_clones;;a;processed;2691634 global;internal_queue_length;;a;processed;0 destination;d_spol;;a;processed;0 destination;d_mlal;;a;processed;0 center;;received;a;processed;5441245 destination;d_mesg;;a;processed;531 destination;d_mail;;a;processed;0 destination;d_auth;;a;processed;29 destination;d_cron;;a;processed;52 destination;d_amqp;;a;processed;2691676 global;scratch_buffers_count;;a;queued;5970004541496 center;;queued;a;processed;2692297 Thanks in advance.

2 3

Use specific python3 binary file
by Armando Martires 24 Oct '19

24 Oct '19

Hi, I have a python destination but I have to use a python3 that is not the system installed python3, in order to use a speficic set of libraries that can't be installed system wide. This alternative python3 is in a folder /apps/python3. How can I configure syslog-ng to use this alternative binary file? Thanks! Regards, Armando Mártires

2 4

Monitoring log traffic
by László Pál 18 Oct '19

18 Oct '19

Hi, My shiny new log infrastructure is almost complete, however I still have some maintenance kind of work to be done, so I asked myself, why not ask community? :) So, it is basically just a server to store logs from various places as usual, however I would like to implement some anomaly detection. I mean, if some logs sources are sending significantly less or more logs a day. Do you think there is an easy solution for this? If needed, I have also a Graylog instance to index logs, so I’m already thinking to implement some kind of alerting there, however obviously I’m not sending everything to Graylog, just what users need to search, so I’m thinking about something I can put on the syslog-ng server and relays to detect anomalies. Any thoughts? Thanks Laszlo

1 0

syslog-ng 3.24 OBS packages for Ubuntu 19.10
by Laszlo Budai 18 Oct '19

18 Oct '19

Hi, syslog-ng 3.24 packages are available for Ubuntu 19.10 wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUb… | sudo apt-key add - echo "deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUb… ./<http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUb…>" | sudo tee /etc/apt/sources.list.d/syslog-ng-obs.list sudo apt-get update sudo apt-get install syslog-ng-core regards, Laszlo Budai

1 0

Install Syslog-ng OSE into Virtual Machine - RHEL OS
by William Luiz Ribeiro Vasconcelos Da Silva 17 Oct '19

17 Oct '19

Good aftenoon ! I will install the syslog-ng OSE into a virtual machine with RHEL 7. Is there anything I should worry about when installing on a virtual machine? Is there something to configurated on OS, before or after the install syslog-ng? Tks, William Luiz Esta mensagem, incluindo seus anexos, pode conter informações privilegiadas e/ou de caráter confidencial, não podendo ser retransmitida sem autorização do remetente. Se você não é o destinatário ou pessoa autorizada para recebê-la, informamos que o seu uso, divulgação, cópia ou arquivamento são proibidos. Portanto, se você recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conteúdo. This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo, informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto

3 4