- syslog-ng - lists.balabit.hu

QuickQ syslog source drivers
by Scot 16 Feb '18

16 Feb '18

Will the syslog() source driver support both RFC5424 and RFC3164 from different hosts at the same time in flight? source s_syslog_tcp { syslog(port(514) transport("tcp")); }; source s_syslog_udp { syslog(port(514) transport("udp")); };

1 0

syslog-ng destination to multiple mssql database
by Garry De Toffoli 16 Feb '18

16 Feb '18

Hello, I have configured syslog.ng OSE version 3.5.3 to log to a MSSQL destination and that all works well. However when I have specified 2 MSSQL destination, same SQL Server, same table on different database, syslog-ng have problem to insert on database, both on the first and on the second. Some time will insert, sometimes not. On /var/log/syslog I have found some lines reporting error CREATE TABLE on the second database, Restarting the service, sudo service syslog-ng restart, takes a long time. When I comment out the destination of the 2nd database, syslog-ng run very well. Is it a limit of the OSE version? Or may be an error on the configuration file? See attached my configuration file. Thank you for the patience. ----------------------- Garry De Toffoli -----------------------

2 1

tcp wrappers
by Czanik, Péter 15 Feb '18

15 Feb '18

Hi, I'm running a poll about tcp wrappers on twitter: https://twitter.com/PCzanik/status/961985140128911361 but feedback is also very welcome here: Is there anybody still using tcp wrappers support built into syslog-ng on Fedora / RHEL / CentOS? Fedora rawhide removed support for tcp wrappers, and if nobody opposes I'd remove it from the rest of my pacages as well to keep packaging simple. Some background info for those who don't even know what I'm talking about: https://fedoraproject.org/wiki/Changes/Deprecate_TCP_wrappers (In short: it's really archaic stuff I last time used in 1995... :) ) Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://syslog-ng.com/blog/author/peterczanik/ https://twitter.com/PCzanik

3 3

Umask funkiness
by Declan White 10 Feb '18

10 Feb '18

I have a directory owned by the syslog-ng user. Its group however belongs to a group of which the user is not a member. The directory is g+s, so that all files and dirs made within it inherit the group owner (and the g+s in the case of dirs). syslog-ng is running with a umask of 022 (interrogated running process to be sure). The file("/dir/${FOO}/${BAR}") destination driver has : create-dirs(yes) perm() dir-owner() dir-group() dir-perm() i.e. "don't change any perms" The aim of the game is to end up with files and dirs readable, but not writable, by the inherited group owner. I can't get it working. I am always ending up with drwx--S--- dirs and -rw------- files Solaris. syslog-ng-3.12.1 -- Declan White

4 7

Re: [syslog-ng] problem elasticsearch2 creating index per HOST
by Abe Lebo 08 Feb '18

08 Feb '18

ah! that must be it! I was not aware that upper case characters are not allowed in elasticsearch index names! $HOST = B8-27-AB-23-11-26 Thanks a million Fabien. Op 8-2-2018 om 10:00 schreef Fabien Wernli: > On Thu, Feb 08, 2018 at 09:42:38AM +0100, Abe Lebo wrote: >> i have templates, but not one specific for this index patern. >> >> i see no errors in the logs, i only see the indices being created if i do >> not add the $HOST, but only $YEAR.$MONTH.$DAY >> >> i'll see if i can set syslog-ng logging to debug > > yes, definitely try that: syslog-ng -Fdv > > One other thing, what does your $HOST resolve to? There are some > restrictions in ES index names, perhaps your macro contains upper case or > other illegal chars? > >

2 1

Insider 2018-02: New website; One Identity; Docker; Error messages; Elastic 6; DevConf;
by Czanik, Péter 08 Feb '18

08 Feb '18

Dear syslog-ng users, This is the 64th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS New syslog-ng website --------------------- Our new website – http://syslog-ng.com/ – is now live. It is a central hub of information for all syslog-ng related content, including commercial products built around syslog-ng. All documentation and blogs were migrated here. The old syslog-ng.org site will go away once all open source related content found its place on the new site. Feedback is much appreciated! Balabit acquired by One Identity -------------------------------- You might have already heard that Balabit, the company behind syslog-ng, has been acquired by One Identity (http://www.oneidentity.com) and you may be wondering what would happen to syslog-ng in the future. Learn how Balázs Scheidler, original author of syslog-ng, sees the future: https://lists.balabit.hu/pipermail/syslog-ng/2018-January/024254.html Tl;dr: nothing to worry about! Logging in Docker using syslog-ng --------------------------------- Collecting logs can be especially important in a containerized environment, where you start and stop containers around the clock, and the container might not even exist anymore by the time you investigate an event. Read this white paper to find out how to: Run your central log server in Docker Collect Docker infrastructure logs using syslog-ng Collect logs from containers using Docker volumes https://syslog-ng.com/white-papers Common syslog-ng error messages and their solutions --------------------------------------------------- In this post, we explain a few common syslog-ng error and warning messages, what they mean, and how to solve them. https://syslog-ng.com/blog/common-syslog-ng-error-messages-solutions/ Getting started with Elasticsearch 6 on RHEL/CentOS --------------------------------------------------- Version 6 of the Elastic Stack has now been available for some time, packed with new features and improved performance. Compatibility of syslog-ng was checked already during the alpha phase of development, as syslog-ng is becoming popular among Elasticsearch users: it can greatly simplify logging to Elasticsearch. There are no major changes from a syslog-ng point of view but – to improve your copy & paste experience – I updated my getting started guide from Elastic Stack 5 to 6. https://syslog-ng.com/blog/syslog-ng-and-elasticsearch-6-getting-started-on… DevConf 2018 ------------ DevConf is a yearly conference for developers, administrators, and users of Linux and related technologies. It is organized by Red Hat in Brno, home to one of their major development centers. Since the majority of syslog-ng users are running their logging server on Red Hat Enterprise Linux or CentOS, it is important to know their needs and the directions where RHEL is heading. It is also a good possibility to discuss logging-related questions with Red Hat engineers in person. https://syslog-ng.com/blog/devconf-long-live-containerization/ UPCOMING EVENTS SCALE: Logging Docker using syslog-ng: https://www.socallinuxexpo.org/scale/16x/presentations/logging-docker-using… Pasadena Convention Center March 8 - 11, 2018 Your feedback and news, or tips about the next issue are welcome at documentation(a)balabit.com. To read this newsletter online, visit: https://syslog-ng.org/ Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://syslog-ng.com/blog/author/peterczanik/ https://twitter.com/PCzanik

1 0

problem elasticsearch2 creating index per HOST
by Abe Lebo 07 Feb '18

07 Feb '18

LS, i want to create an Elasticsearch index per host i receive logs from (also syslog-ng) , somehow that fails. in have a setup that creates an index per day (see below), that works. I can build a file output per host per day by adding the $HOST to the filename, that works too.(see below) but if add $HOST to the indexname it no longer creates indices. Anybody know what's wrong? Thanks in advance, Abel ...... destination d_testfile { file( "/tmp/testlogabel-$HOST$YEAR$MONTH" flags(syslog-protocol) template("$(format-json @timestamp=$ISODATE --scope nv-pairs)\n") ); }; ################################################################################ # Elasticsearch destination ################################################################################ destination d_es_per_day { elasticsearch2( client-lib-dir("/usr/share/elasticsearch/lib/:/usr/lib/syslog-ng/3.13/java-modules/") index("test-abel_$YEAR.$MONTH.$DAY") type("test") cluster-url("http://node01:9400 http://node02:9400 http://node03:9400") client-mode("http") template("$(format-json @timestamp=$ISODATE --scope nv-pairs)\n") ); }; ###### # sending logs to ES destination log { source(s_net); source(s_src); filter(f_abel); parser(p_abel); destination(d_es_per_day); flags(flow-control); };

2 2

OSE, Shared Storage, & Load Balanced syslog Messages
by Pierce, Stevan - PHX 07 Feb '18

07 Feb '18

I am working on setting up OSE as a relay and would like to have storage set aside in case a destination upstream from the relay goes down. How does load balancing syslog messages work when clustering at the OS level? If I am sharing storage across multiple relays, how would I go about setting it up so that messages from one server does not overwrite another server? swp

2 1

Rsyslog relay or syslog-ng ?
by Scot 06 Feb '18

06 Feb '18

Hi, Hoping someone has seen an easy fix for this. Sorry is it's specifically referenced somewhere I'm not seeing. Dealing with a vendor who is not able to leverage the RFC headers or TCP input. We have rsyslog relays in remote sites sending TCP/514 to syslog-ng and others locally sending directly to syslog-ng TCP/UDP 514. The devices sending directly to syslog-ng are reporting to the IDS correctly. Hosts relaying through rsyslog are showing a source address of the relay. */etc/rsyslog.d/forward.conf* *$ActionQueueFileName fwdRule1 # unique name prefix for spool files* *$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)* *$ActionQueueSaveOnShutdown on # save messages to disk on shutdown* *$ActionQueueType LinkedList # run asynchronously* *$ActionResumeRetryCount -1 # infinite retries if host is down* *# remote host is: name/ip:port, e.g. 192.168.0.1:514 <http://192.168.0.1:514>, port optional* **.* @@syslog-ngIP:514* */etc/syslog-ng/conf.d* *source s_net_tcp {tcp(ip(0.0.0.0) port(514) max-connections(300) keep_hostname(yes) so_rcvbuf(262142) log_iw_size(25000)); };* *source s_net_udp {syslog(ip(0.0.0.0) keep_hostname(yes) port(514) transport("udp") flags(no-hostname) so_rcvbuf(262142));};* *destination d_ids {network("IDSHOSTNAME" spoof_source(yes) transport(udp) port(514) flags(syslog-protocol)); };* *log { source(s_net_udp);* * channel {filter(f_ids); destination (d_ids);};* * channel {parser(pattern_db); destination (d_es);};* *};* *log { source(s_net_tcp);* * channel {filter(f_ids); destination (d_ids);};* * channel {parser(pattern_db); destination (d_es);};* *};*

3 4

RFC5424 sending - dest=unix-stream. Framing?
by Declan White 05 Feb '18

05 Feb '18

I have syslog-ng pumping data into a unix-stream destination socketfilething. Data going into this domain socket has lots of adventures, and then emerges as a source unix-stream on another host. Both have flags(syslog-protocol), but the data gets framing errors at the receiver. I think it's become a bytestream (instead of a message stream) enroute, and the recv() is relying on only getting one message at a time, because it can't know the message length any other way, as unix-stream gets messages without framing(?). Is there a way to salvage this? (I noticed TLS might add framing, but since the tunnel is already TLS ...) -- Declan White

2 5