- syslog-ng - lists.balabit.hu

Syslog-ng Questions
by Miah Lang 22 Feb '17

22 Feb '17

> Is it possible to configure multiple sources, one with flags(no-parse) and one without? > > e.g. > > source s_syslog-ports { > udp(port(514)); > tcp(port(1514) max-connections(100)); > tcp(port(514) max-connections(100)); > }; > > source s_syslog_np-ports { > udp(port(514) flags(no-parse)); > tcp(port(1514) max-connections(100) flags(no-parse)); > tcp(port(514) max-connections(100) flags(no-parse)); > }; > > filter f_Cisco-router { in-list("/etc/syslog-ng/filter/Cisco-router.txt", value("SOURCEIP")); }; > destination d_Cisco-router {file("/var/log/IT/network/router/cisco/${SOURCEIP}/${SOURCEIP}-${YEAR}${MONTH}${DAY}.log" template(t_message-only));}; > log {source(s_syslog-ports); filter(f_Cisco-router); destination(d_Cisco-router);}; > > filter f_Cisco-switch { in-list("/etc/syslog-ng/filter/Cisco-switch.txt", value("SOURCEIP")); }; > destination d_Cisco-switch {file("/var/log/IT/network/switch/cisco/${SOURCEIP}/${SOURCEIP}-${YEAR}${MONTH}${DAY}.log" template(t_message-only));}; > log {source(s_syslog_np-ports); filter(f_Cisco-switch); destination(d_Cisco-switch);}; > > Whenever I do this, I get an error message when restarting the service. > “Job for syslog-ng.service failed because the control process exited with error code. See "systemctl status syslog-ng.service" and "journalctl -xe" for details.” > “Cannot add dependency job for unit microcode.service, ignoring: Unit is not loaded properly: Invalid argument.” >

4 3

Syslog-no Question
by Miah Lang 22 Feb '17

22 Feb '17

> I’m having trouble determining when you would use the network() source vs the syslog() source? > > We have logs coming in on TCP port 514 from network devices, would it be best to use the network() source for this? Can you setup both the network() and syslog() sources to listen on TCP port 514 or would that cause issues? >

2 1

Have queries related to syslog-ng
by SivaRamaPrasad Ravipati 21 Feb '17

21 Feb '17

Hi, I have some queries. Please clarify me. 1) What are the different filter combinations used by syslog-ng and their unique usecases? 2)What languages are used to write filters? 3)How do we provide security while filtering logs? Thanks, Sivaram.

2 2

Syslog-ng, centos7 and systemd seem to hate each other
by Clayton Dukes 19 Feb '17

19 Feb '17

…or at the very least, say bad things when the other is not looking :) I did a bit of googling for this and found other people with similar issues, but could not find the fix. This is on a cos7.0 box (not 7.2) running syslog-ng v3.9 (user did have v3.5 but I tried upgrading him to resolve this). If I run syslog-ng -Fdve it works fine and sends everything to our python program destination However, if I use the systemd script provided with syslog-ng, then I get nothing. If I tell sng to just log to a file (no filters, etc.), then it works. If I tell sng to just use the program dest (no filters, etc.) it does NOT work. If I do a 'systemctl stop syslog-ng' and then just simply type 'syslog-ng' (no foreground, debug, etc. switches) from the command line, it works fine. Rather confusing, but I can't see why the systemctl file is not working as it should. Any ideas? Other references: https://wiki.archlinux.org/index.php/syslog-ng https://bbs.archlinux.org/viewtopic.php?id=187057 http://serverfault.com/questions/760383/syslog-ng-working-as-foreground-pro… -Clayton

6 9

syslog-ng with sqlite 100% cpu
by work ran 19 Feb '17

19 Feb '17

hi i am running syslog-ng with sqlite i am sending 50 logs per second and i am getting 80% cpu usage. could someone tell me how i can fix this or is it normal thanks Ran syslog-ng 3.2.5 Installer-Version: 3.2.5 Revision: ssh+git://bazsi@git.balabit //var/scm/git/syslog-ng/syslog-ng-ose--mainline--3.2#master#9d4bea28198bd731df1a61e980a2af5b88d81116 Compile-Date: Jul 25 2014 15:20:50 Enable-Threads: on Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-SSL: off Enable-SQL: on Enable-Linux-Caps: off Enable-Pcre: on Enable-Pacct: off

4 5

Re: [syslog-ng] can syslog-ng process email attachments?
by james.r.hendrick 12 Feb '17

12 Feb '17

Thanks, I'll take a look Jim Sent from my Verizon, Samsung Galaxy smartphone -------- Original message --------From: "Fekete, Róbert" <robert.fekete(a)balabit.com> Date: 2/11/17 10:41 AM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] can syslog-ng process email attachments? Hi, Fabien created an mbox source a while back, maybe you can use it: https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-… Regards, Robert On Fri, Feb 10, 2017 at 9:03 PM, Jim Hendrick <james.r.hendrick(a)gmail.com> wrote: We have security appliances that generate CSV files and automatically send them (nightly) I am looking for a way to get them into a log stream and trying to make it as simple as possible. I suppose I could do something with procmail or even openssl and an IMAP connection, but I wanted to ask this esteemed company if any of you have solved this already. even something radical like "source s_mail ( smtp() );" and have config options to tweak what it does with different recipients / mail formats Any (appropriate) suggestions? Thanks!Jim ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

1 0

can syslog-ng process email attachments?
by Jim Hendrick 11 Feb '17

11 Feb '17

We have security appliances that generate CSV files and automatically send them (nightly) I am looking for a way to get them into a log stream and trying to make it as simple as possible. I suppose I could do something with procmail or even openssl and an IMAP connection, but I wanted to ask this esteemed company if any of you have solved this already. even something radical like "source s_mail ( smtp() );" and have config options to tweak what it does with different recipients / mail formats Any (appropriate) suggestions? Thanks! Jim

2 1

Unable to install ubuntu 16.04
by Simon Mousey Smith 11 Feb '17

11 Feb '17

Hi All Im struggling to install the syslog-ng-mod-sql on ubuntu 16.04 It keeps saying 'syslog-ng-mod-sql : Depends: syslog-ng-abi-3.9-0 but it is not installable' root@syslog:~# apt-get install syslog-ng-core syslog-ng-mod-sql Reading package lists... Done Building dependency tree Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: syslog-ng-mod-sql : Depends: syslog-ng-abi-3.9-0 but it is not installable E: Unable to correct problems, you have held broken packages. root@syslog:~# apt-get install syslog-ng-core Reading package lists... Done Building dependency tree Reading state information... Done Suggested packages: syslog-ng-mod-sql syslog-ng-mod-mongodb syslog-ng-mod-json syslog-ng-mod-smtp syslog-ng-mod-amqp syslog-ng-mod-geoip syslog-ng-mod-redis syslog-ng-mod-stomp syslog-ng-mod-riemann syslog-ng-mod-graphite syslog-ng-mod-python syslog-ng-mod-java syslog-ng-mod-java-common-lib syslog-ng-mod-elastic syslog-ng-mod-hdfs syslog-ng-mod-kafka syslog-ng-mod-http syslog-ng-mod-curl syslog-ng-mod-kvformat syslog-ng-mod-add-contextual-data The following NEW packages will be installed: syslog-ng-core 0 upgraded, 1 newly installed, 0 to remove and 5 not upgraded. Need to get 0 B/408 kB of archives. After this operation, 1,839 kB of additional disk space will be used. Selecting previously unselected package syslog-ng-core. (Reading database ... 107008 files and directories currently installed.) Preparing to unpack .../syslog-ng-core_3.9.1-1_amd64.deb ... Unpacking syslog-ng-core (3.9.1-1) ... Processing triggers for systemd (229-4ubuntu16) ... Processing triggers for ureadahead (0.100.0-19) ... Setting up syslog-ng-core (3.9.1-1) ... Processing triggers for systemd (229-4ubuntu16) ... Processing triggers for ureadahead (0.100.0-19) … Success... root@syslog:~# apt-get install syslog-ng-mod-sql Reading package lists... Done Building dependency tree Reading state information... Done Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: syslog-ng-mod-sql : Depends: syslog-ng-abi-3.9-0 but it is not installable E: Unable to correct problems, you have held broken packages. root@syslog:~# Any ideas? Regards Simon

3 7

syslog-ng logging only as foreground process and not as daemon on centos7
by Jordan Ladora 10 Feb '17

10 Feb '17

I've got syslog-ng v3.7.3_7 running on FreeBSD 11-REL as the log server with both FreeBSD 11-REL and CentOS7 clients (the CentOS clients are running v3.5.6-3.el7 as it's the latest version installed by yum epel repo). It's always worked perfectly on the FreeBSD systems, but a few months ago I had the same problem described here when setting up a CentOS client- https://serverfault.com/questions/760383/syslog-ng-working-as-foreground-pr… ...I followed the fix of uncommenting the "ForwardToSyslog=yes" line in /etc/systemd/journald.conf and it started logging perfectly on CentOS clients, too. However, in setting up another CentOS7 client yesterday, this fix no longer worked. If I start syslog-ng via- chkconfig syslog-ng on systemctl restart syslog-ng.service ..it starts and runs and is shown in ps, starts automatically after reboot, but doesn't log to the server. If I start it manually from the terminal with- nohup /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid & ...then it does work and starts logging to the server immediately. In that case there are two identical (except for pid) lines shown by ps, e.g.- 482 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid 904 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid ...but only the one I started from the terminal (nohup...) actually logs. Any help would be much appreciated. -jl

3 3

Insider 2017-02: securing Elasticsearch; Troubleshooting; FOSDEM & SCALE;
by Czanik, Péter 09 Feb '17

09 Feb '17

Dear syslog-ng users, This is the 55th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Securing connections to Elasticsearch ------------------------------------- Recently, news have come out that unprotected MongoDB databases are being actively compromised: content is copied and replaced by a message asking for a ransom to get it back. As “The Register” reports: Elasticsearch is next. Read our latest blog post to learn more about how to secure your Elasticsearch cluster and avoid a ransomware: https://www.balabit.com/blog/secure-your-elasticsearch-cluster-and-avoid-ra… Troubleshooting syslog-ng to syslog-ng connections -------------------------------------------------- syslog-ng users often face the challenge of not being able to send logs over the network from one syslog-ng instance to another. There can be many reasons for this; some are independent from syslog-ng, while others are related to the syslog-ng configuration. Here are a few troubleshooting tips: https://www.balabit.com/blog/troubleshooting-syslog-ng-syslog-ng-connection… Load balancing HTTP connections to Elasticsearch ------------------------------------------------ In addition to developing the HTTPS driver for syslog-ng, Fabien Wernli also contributed HTTP load balancing support to the syslog-ng Elasticsearch driver, which is also used by HTTPS support for Elasticsearch: https://github.com/balabit/syslog-ng/pull/1319 FOSDEM 2017 ----------- This year Peter Czanik, community manager at Balabit, participated with two syslog-ng presentations at the annual FOSDEM conference in Brussels. Read about his experiences at the event: https://www.balabit.com/blog/syslog-ng-fosdem-2017/ How to send JSON log messages to RabbitMQ from syslog-ng -------------------------------------------------------- In this post, you can read about how to configure syslog-ng to send logs in JSON format in the body of an AMQP message: https://sharknet.us/2017/02/04/how-to-send-json-log-messages-to-rabbitmq-fr… UPCOMING EVENTS Balabit will participate at the Southern California Linux Expo this year again: http://www.socallinuxexpo.org/scale/15x You can talk to our engineers at the Balabit booth and listen to Peter Czanik present about parsing security logs using syslog-ng in room 106: http://www.socallinuxexpo.org/scale/15x/presentations/get-most-out-your-sec… Your feedback and news, or tips about the next issue are welcome at documentation(a)balabit.com. To read this newsletter online, visit: https://syslog-ng.org/ Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

1 0