- syslog-ng - lists.balabit.hu

query on remote logging
by SIMON BABY 07 Jul '21

07 Jul '21

Hi Team, Can I know if there is a configuration in syslog-ng.conf for logging the messages from a remote client. Can you please send me a sample configuration? Regards Simon

2 1

Dynamic setting value out of message?
by Matthias Gruber 01 Jul '21

01 Jul '21

Hi! I hope it is simple and my thoughts and seeks about it were to complicated :-), simply I didnt know how to do that, perhaps someone has a clue for me I am getting e.g. messages like "MESSAGE": "UIMUC4Maintenance.py: \"== deactivate_uc4_monitoring = ENDE ==\"", (thats out of an JSON-formatted syslog-ng output), What I would like to do is, to extract the 'UIMUC4Maintenance.py:' and put it into a SDATA-Custom-Variable or PROG but based on a regex so some sort of rewrite-rule like (no not a correct syntax, only to describe it) rewrite r_fill_program { set(match("^\w*\.py:" value("MESSAGE")) value("PROG")); }; As far as I understand it, set requires a "string" as first parameter, I could use a lots of rewrites with a condition, but I am in "lack of a static string", this should be some sort of variable :-) or I could do that static with a filter for every "^\w*\.py:"-Text, but I hope I could do that dynamic, every time a match of my regex syslog-ng inserts that part into a variable and so on... Is that possible? cheers Matthias ------------------------------------------------------------------------------------ METZLER Informationstechnologie Matthias Gruber IT-Infrastruktur & -Betrieb B. Metzler seel. Sohn & Co. Aktiengesellschaft Untermainanlage 1 60329 Frankfurt am Main Telefon (0 69) 21 04 - 43 30 Telefax (0 69) 21 04 - 40 40 MGruber(a)metzler.com www.metzler.com Vorstand: Harald Illy, Emmerich Müller, Gerhard Wiesheu Vorsitzender des Aufsichtsrats: Dr. Christoph Schücking Sitz der Gesellschaft: Frankfurt am Main, Handelsregister-Nr. HRB 123 365 Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfänger sein, so bitten wir Sie höflich, dies unverzüglich dem Absender mitzuteilen und die Nachricht zu löschen. Es ist unzulässig, die Nachricht unbefugt weiterzuleiten oder zu kopieren. Da wir nicht die Echtheit oder Vollständigkeit der in dieser Nachricht enthaltenen Informationen garantieren oder zusichern können, sind die vorstehenden Ausführungen rechtlich nicht bindend. Eine Haftung hierfür wird ausgeschlossen. This message is confidential. If you are not the intended recipient, we kindly ask you to inform the sender and delete the information. Any unauthorised dissemination or copying hereof is prohibited. As we cannot guarantee or assure the genuineness or completeness of the information contained in this message, the statements set forth above are not legally binding. Accordingly we cannot accept any liability for their contents.

3 4

Is there any compatibility between ver3.2 and ver3.25 about syslog-ng.conf?
by leis 25 Jun '21

25 Jun '21

Hello, I am looking to migrate my environment from CentOS5 syslog-ng ver3.2 to ubnutu20.02 syslog-ng ver3.25. But when I use syslog-ng.conf of ver3.2 to syslog-ng.conf of ver3.25, syslog-ng cannot start. Is there any compatibility between ver3.2's syslog-ng.conf and ver3.25's syslog-ng.conf? Thanks

2 1

Re: [syslog-ng] Tips to diagnose missing syslog messages?
by brian hoffman 15 Jun '21

15 Jun '21

We've spent a lot of time troubleshooting UDP drops, and the so_reuseport option did help quite a bit. There's a max number that the RECVQ can reach (which can be seen with: /usr/bin/netstat -tupln|egrep "(^udp.*syslog-ng)" ) that correlates to the so_rcvbuf. Having more queues won't keep drops from happening, since the hashing in the OSE is related to source IP, so a talkative host can still fill up one of those queues and cause drops. The PE uses eBPF which is apparently better at preventing this, but we found that by using the updated version at "copr:copr.fedorainfracloud.org:czanik:syslog-ng324/x86_64" instead of the stock RH one and bumping up the so_rcvbuf, it did perform much better (and I believe you may need that version for the so_reuseport).-Brian

1 0

Re: [syslog-ng] Request for config help
by Pradeep Gotaparthi 15 Jun '21

15 Jun '21

Hi, Could anybody help us out with the below query? Thanks. On Fri, Mar 26, 2021 at 12:06 AM Pradeep Gotaparthi <pradeep.gp(a)gmail.com> wrote: > Hi, > > We have been using syslog-ng for over a year now and it is working great, > thanks! > > One issue we face is that when we stop the process when there is some > buffer, the threshold is ignored and a bunch of logs get pushed out (as > much as 10 times the threshold value). This is causing problems on the > receiving end as it can't handle the load. We thought the log-fifo-size > parameter can help as syslog uses this parameter to store logs in memory > but it is not helping. > > It would be great if you can guide us. > > > Thanks. >

5 8

Insider 2021-06: Alerting; EoL technologies; Google Summer of Code;
by Peter Czanik (pczanik) 10 Jun '21

10 Jun '21

Dear syslog-ng users, This is the 92nd issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS First steps of sending alerts to Discord and others from syslog-ng: http() and Apprise ------------------------------- A returning question I get is: “I see, that you can send alerts from syslog-ng to Slack and Telegram, but do you happen to support XYZ?” Replace XYZ with Discord and countless others. Up until recently, my regular answer has been: “Take a look at the Slack destination of syslog-ng, and based on that, you can add support for your favorite service”. Then I learned about Apprise, a notification library for Python, supporting dozens of different services. This blog is the first part of a series. It covers how to send log messages to Discord using the http() destination of syslog-ng and an initial try at using Apprise for alerting. https://www.syslog-ng.com/community/b/blog/posts/first-steps-of-sending-ale… Changes in technologies supported by syslog-ng: Python 2, CentOS 6 & Co. ---------------------------------------------------- Technology is continuously evolving. There are regular changes in platforms running syslog-ng: old technologies disappear, and new technologies are introduced. While we try to provide stability and continuity to our users, we also need to adapt. Python 2 reached its end of life a year ago, CentOS 6 in November 2020. Using Java-based drivers has been problematic for many, so they were mostly replaced with native implementations. From this blog you can learn about recent changes affecting syslog-ng development and packaging. https://www.syslog-ng.com/community/b/blog/posts/changes-in-technologies-su… Google Summer of Code 2021 -------------------------- This year, the syslog-ng team participates in Google Summer of Code (GSoC) again as a mentoring organization. Two students paid by GSoC work on syslog-ng under the mentoring of syslog-ng developers. One of the students works on MacOS support, including the new ARM-based systems, while the other one is working developing on a new regular expression parser: https://summerofcode.withgoogle.com/organizations/5548293561516032/ WEBINARS * syslog-ng, Proxy Protocol Support: https://www.syslog-ng.com/event/syslogng-proxy-protocol-support-8148877/ * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Debian/Ubuntu OBS packages: discontinued
by Laszlo Budai 01 Jun '21

01 Jun '21

Hi, unofficial syslog-ng Debian/Ubuntu OBS[1] packages are discontinued. The previously built syslog-ng packages are still available (up to version 3.31.2). Some of you might know that the origin of the repository's name was a kind of joke : at that time I was told that 'we need syslog-ng binary packages', others said: 'we cannot ship binary packages for OSE'. I resolved this conflict by creating an OBS repository under my personal name in my free time... problem solved... at least until now. This was my personal project, under my personal name, working on it in my free time - and unfortunately I don't have the time for maintaining it anymore. I'd suggest to migrate to DBLD[2] packages from my OBS repository (if not possible for some reasons, just contact to me and I'm trying to help you). [1] https://build.opensuse.org/project/show/home:laszlo_budai:syslog-ng [2] https://www.syslog-ng.com/community/b/blog/posts/dbld-a-syslog-ng-developer… kind regards, Laszlo Budai [Twitter]<https://twitter.com/Budai_L> [LinkedIn] <https://www.linkedin.com/in/laszlo-budai-55243a3/> [Github] <https://github.com/lbudai>

1 0

Using custom parser with only a subset of udp traffic
by Faine, Mark R. (MSFC-IS40)[NICS] 27 May '21

27 May '21

Hopefully one of you can help me figure this out. All of my udp traffic is coming in through one source (network driver), some of it is firewall data but I have to filter it out based on where it is coming from and other fields in the log data. I want to use the csv_parser here since it would make it easier for me to get the data I need from the firewall data. Does this require splitting my one source into two sources, since I would need to use flags('no-parse') on a firewall data source? I also think this means that I can't use a single log path to process both sources? Also, how to prevent potential duplication since the non-firewall source will still receive the firewall traffic and try to process it with syslog-parser. I think the flow is something like this: firewall source -> custom parser -> filters ->rewrites / \ all_data -> \ -> destination \ / all udp source -> filters -> rewrites / I understand this is a bit complex to answer simply, I'm mostly looking for answers to the above questions as well as high level guidance for how to proceed. Thanks, -Mark

3 4

Slack or other
by Faine, Mark R. (MSFC-IS40)[NICS] 21 May '21

21 May '21

Have you considered or would you consider a slack or other real time based communication environment for syslog-ng? The mailing list is great but it's kind of slow asynchronous communication. It would be awesome to have a community where I could just fire off a quick question and get a quick answer from anyone who might be around. Thanks, -Mark

3 3

using VARARGS correctly
by Faine, Mark R. (MSFC-IS40)[NICS] 21 May '21

21 May '21

Syslog-ng 3.19 block destination d_default(basepath('/var/log/remote/backup') location("") app("") name("")) { file("`basepath`/`location`/`app`/${HOST}/${HOST}_`app``__VARARGS__`.log" persist-name(`name`) create_dirs(yes) flags("threaded", "no-multi-line")); }; However, when I call it like so: d_default(basepath(`BASEPATH`) location("$location") app("$app") name('d_default_udp') "$session"); I get a syntax error when checking with --syntax-only Error parsing block reference, syntax error, unexpected LL_STRING, expecting ')' in /etc/syslog-ng/conf.d/splunk.conf: 23 categorize_loc(); 24 categorize_app(); 25 }; 26 }; 27 destination { 28----> d_default(basepath(`BASEPATH`) location("$location") app("$app") name('d_default_udp') "$session"); 28----> ^^^^^^^^^^ >From looking at the documentation, it looks like it's a valid way to use it. $session here is just a number from 0-9 that is set from rewriting a user defined macro from the message's sessionid field. It's only going to exist for a certain kind of message. Thanks, -Mark

3 5