May 2018 - syslog-ng - lists.balabit.hu

exclude empty pairs from json-parser()
by Nik Ambrosch 23 May '18

23 May '18

Hi - I'm trying to figure out how to instruct json-parser() to exclude empty (or nonexistent) json pairs from being passed to it's template. This is possible with the csv parser using null("") but not json. It seems that if i specify all my json-parser() pairs using --pair example=\"${MYMACRO}\" the desired behavior is achieved, however using --scope nv-pairs instead will log the empty values.

2 2

syslog-ng Open Source Edition official binary repository
by Gergely Bodnár 23 May '18

23 May '18

Dear syslog-ng users, Last autumn, we asked for your feedback ( https://syslog-ng.com/blog/new-website-and-official-repo-coming-up-tell-us-…) on our plan to introduce official binary repositories for syslog-ng Open Source Edition. The overwhelming majority of those who got back to us agreed that it is fair to provide this service tied to a simple registration. We are happy to announce that although a bit late compared to our originally planned schedule, we are launching this service, first as a test run with Debian Jessie packages. We will always host the latest release, and these packages will be built and tested by us, so you won't have to worry about dependencies - provided you are running a suitable Linux distribution. We are planning to add an RPM repository as well in the future, knowing well that many of our open source users require that. How to use repository? ------------------------------- To use the newly introduced syslog-ng Open Source Edition Debian Jessie repository, you only have to complete a few quick and easy steps. 1. Create a MyBalabit account (https://my.balabit.com/signup) Just click on MyBalabit in the top right corner of the page, and then choose Sign Up (under OS X, the sign up option will be hiding under the hamburger menu). At the top of the form, there will be a drop down menu where you have to specify why are you requesting access to this portal. Choose syslog-ng OSE Binary Repository Access, then fill out the rest of the form, read and agree to our privacy policy, then hit create you account.If you already happen to have a MyBalabit account, simply log in and look for Request Access in the top menu. Once in there, select syslog-ng OSE Binary Repository Access and click on request access. 2. Once you finished with the initial steps, look for the Access tokens option at the bottom of the left menu bar. In there, you will find your token to be used with the syslog-ng OSE DEB repository. If for some reason you need a new token, use the regenerate button. 3. Now you should be all set to configure your compatible Linux distribution to access the repository. # echo "deb http://JXJiI9Znyt:@apt.syslog-ng.com/syslog-ng-ose jessie main" >/etc/apt/sources.list.d/oserepo.list # apt-get update # apt-get install --no-install-recommends -y curl # curl http://JXJiI9Znyt@apt.syslog-ng.com/syslog-ng-ose/pubkey.gpg | apt-key add - # apt-get update # apt-cache policy syslog-ng # apt-get install syslog-ng And that's it, you should be all set! Please, don't forget that this is still very much a trial for us to gauge the interest in such a service. Bugs might surface, the repository might be inaccessible due to technical glitches, etc. We encourage you to share your feedback with us. It would be much appreciated! Regards, Gergely Bodnar -- *Gergely Bodnar* | syslog-ng Product Owner gergely.bodnar(a)balabit.com follow Balabit on: web <http://go.balabit.com/F9ZZ500bh0U013LX000VAE5> | blog <http://go.balabit.com/VZZi000AL90V300U55X1cE0> | linkedin <http://go.balabit.com/CZ100j0d50LU0EAZ3X09V05>| facebook <http://go.balabit.com/I090eUZ0551kEA3X0L0ZV00> | twitter <http://go.balabit.com/c0503X1U05EZ09AVl0ZfL00> *© BalaBit Corp.* Weiser Mazars 135 West 50th Street New York, NY 10020 USA

5 7

Forwarding flat files to a remote syslog-ng server
by Song, Young 22 May '18

22 May '18

Hi, I'm trying to send various flat files to a central syslog-ng server. So, I have the following setups but somehow using the "local1" and "local2" facilities don't work, although no errors when restarting syslog services on both client & server. Any advice would be appreciated much! ======================================================================= My syslog-ng CLIENT (running syslog-ng 3.6.2): source s_file1 { file("/var/log/syslog/file1.log" follow-freq(1) flags(no-parse) program_override("audit")); }; filter f_file1 { facility(local1); }; destination d_file1 { syslog("10.10.10.10" transport("udp") port(514)); }; log { source(s_file1); filter(f_file1); destination(d_file1); }; source s_file2 { file("/var/log/syslog/file2.log" follow-freq(1) flags(no-parse) program_override("audit")); }; filter f_file2 { facility(local2); }; destination d_file2 { syslog("10.10.10.10" transport("udp") port(514)); }; log { source(s_file2); filter(f_file2); destination(d_file2); }; ======================================================================== My syslog-ng SERVER (10.10.10.10 - running syslog-ng 2.0.9): destination d_file1 { file ("/var/log/syslogs/file1-$YEAR$MONTH$DAY.log" owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes)); }; filter f_file1 { facility(local1); }; log { source(remote); filter(f_file1); destination(d_file1); }; destination d_file2 { file ("/var/log/syslogs/file2-$YEAR$MONTH$DAY.log" owner(root) group(root) perm(0644) dir_perm(0755) create_dirs(yes)); }; filter f_file2 { facility(local2); }; log { source(remote); filter(f_file2); destination(d_file2); }; Thanks. - Young

3 3

Error binding socket;
by vinod samant 14 May '18

14 May '18

Hi I am getting below error while restarting server . [root@localhost logs]# /etc/init.d/syslog-ng restart Stopping syslog-ng: [FAILED] syslog-ng is stopped Starting syslog-ng: [2018-05-11T16:41:55.756615] Error binding socket; addr='AF_INET(0.0.0.0:514)', error='Address already in use (98)' [2018-05-11T16:41:55.756684] Error initializing message pipeline; plugin name='network', location='/etc/syslog-ng/syslog-ng.conf:24:3' [FAILED] My requirement :- OS:- centos 6.5 1. I want to use wildcard to send all apache custom logs from client to server . Configuration of both client and server :- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>client conf<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< @version:3.13 @include "scl.conf" @define allow-config-dups 1 source s_local { system(); internal(); }; destination d_network {udp("192.168.122.67" port(514)); }; destination d_local { file("/var/log/messages"); }; log{ source(s_local); destination(d_network); destination(d_local); }; #################### source s_local{ wildcard-file( base-dir("/usr/local/apache/logs") filename-pattern("*.log") recursive(no) follow-freq(1)); }; destination d_network {tcp("192.168.122.67" port(514)); }; log{ source(s_local); destination(d_network); }; >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>server conf<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< @version:3.13 @include "scl.conf" source s_local{system(); internal();}; source s_network{ udp(); tcp(); }; destination d_local{file("/var/log/messages");}; destination d_from_net{file("/var/log/from_net");}; log{ source(s_local); destination(d_local); }; log { source(s_network); destination(d_from_net); }; source s_net { network(flags(no-parse)); }; parser p_apache { csv-parser( columns("apache.FILE_NAME", "apache.MESSAGE") flags(greedy) ); }; destination d_apache { file("/usr/local/apache/logs/${apache.FILE_NAME}" template("${apache.MESSAGE}\n")); }; log{ source (s_net); parser (p_apache); destination(d_apache); }; What should be changed on configuration file on both side ? *Thanks & Regards :-* *VINOD SINGH SAUD* *(M):- 09718663552* *(W):-09997645597* *(E) :- vinod.samant.123(a)gmail.com <vinod.samant.123(a)gmail.com>*

3 6

How to configure client and host server
by vinod samant 11 May '18

11 May '18

Hi , I am beginner in linux and first time i am going to installation and configure syslog-ng .So, I hope ,will get more help from your side because i am being confuse while reading documentation.first of all i want to mention my requirements. OS:- centos 6.9 My Requirements:- 1. we have 20 apache webserver ,we are writing apache custom logs or php applications logs in a particular directory ,But directory location is different different for both apache log and php application logs. For example we are writing log in "/usr/local/apache/logs " and apache log format or php application log format are mentioned below . *apche custom logs Fromta:- * abc_access_2018-05-09.log ,abc_error_2018-05-09.log and xyz_access_2018-05-09.log etc.. *Php application log format:- * hello_vidoes_20180509.txt , world_music_20180509.txt 2. I want to store these logs of 20 web servers on a centralised server at real time and we do not want to change logs file and format . Is this possible? 3. I want to make 20 web server as client server and centralised server (as host server). Here ,I can not understand from documentation ,So Can you provide the configuration for both side client and host side ? Client ip:- 192.168.122.184 server IP(centralised):- 192.168.122.61 *Thanks & Regards :-* *VINOD SINGH SAUD* *(M):- 09718663552* *(W):-09997645597* *(E) :- vinod.samant.123(a)gmail.com <vinod.samant.123(a)gmail.com>*

3 9

Best way to pipe "application" logs to central syslog-ng server.
by Delon Lee Di Lun 09 May '18

09 May '18

Hi, I am working on a mini project that requires the receipt of "application" logs using rsyslog client to pipe it to a syslog-ng central server. An example of the "application" logs im referring to would be for instance Apache HTTPD logs, I want to separate the "application" logs, in this example, the apache logs, and the "OS" logs into different directories. I am thinking of certain crude way, using regex to filter the messages at my server end ,to do it which might not be as clean. Would like to ask the floor if anybody had experience with working in this environment. Would change my rsyslog client and my syslog-ng server to use RFC5424, IETF's compare to the legacy BSD syslog protocol help? Comparing the headers there is a "app-name" variable in the IETF's syslog protocol I can use? Yours Sincerely, Delon Lee

5 12

Disk based queuing with rsyslog?
by Delon Lee Di Lun 09 May '18

09 May '18

Hi, I read that syslog-ng can be setup to use disk based with flow control with the client to prevent lost of logs. However I was just thinking, how does this mechanism works? Does the client have to be using syslog-ng? Will it work if the client is using rsyslog? Native appliance that purely support syslog? Yours Sincerely, Delon Lee

4 9

Difference in version
by Delon Lee Di Lun 08 May '18

08 May '18

Hi, I am trying to download the source code of syslog-ng 3.15.1 from the link below: https://github.com/balabit/syslog-ng/releases May I know what is the difference between the file with "orig" and the one without? Yours Sincerely, Delon

6 13

hostname not appearing correctly when receiving logs from switches
by Clayton Dukes 07 May '18

07 May '18

Interesting! We’ve been getting a lot of support tickets for this very problem. I can easily recreate the issue. Balabit Team: is this a new bug? [cid:image001.png@01D306E3.0FEBC990] Clayton Dukes Founder & CEO LogZilla Corporation 2900 N. Quinlan Park Rd, B240-341 Austin, TX, 78732 Tel: 936-4NetOps (463-8677) Web: www.logzilla.net<http://www.logzilla.net/> [cid:image002.png@01D306E3.0FEBC990]<https://twitter.com/logzilla>[cid:image003.png@01D306E3.0FEBC990]<https://youtu.be/drg5wv_mgfA>[cid:image004.png@01D306E3.0FEBC990]<https://www.linkedin.com/in/lzcdukes/> For NetOps, By NetOps! From: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu> on behalf of Joshua <aces621(a)yahoo.com> Reply-To: Joshua <aces621(a)yahoo.com>, Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Date: Monday, April 30, 2018 at 7:09 PM To: "syslog-ng(a)lists.balabit.hu" <syslog-ng(a)lists.balabit.hu> Subject: [Suspected Spam] [syslog-ng] hostname not appearing correctly when receiving logs from switches Hi All, I am pretty new to syslog-ng but do have some basic knowledge. I have deployed syslog-ng v3.14 on a newly deployed Linux server because syslog-ng v3.5 is working very well on another syslog server. On this new deployment, the syslogs received from most of the servers are able to show IP/host, however, the syslogs from our switches contains IP/host showing as ":" (colons). I copied the current working custom build .conf from another syslog server into our new server. Can someone help me figure out what I am missing? It is working for some components but not for switches. I tested the same switch by sending syslog to another syslog server and the hostname is appearing but just not appearing on the new syslog server. The only difference between the two server is that one uses v3.5 (the working one) and the other uses syslog-ng v3.14. I have set: "keep_hostname (yes)" but it still doesn't work. Can someone please help? Am I missing something here? Thanks Joshua Lai

5 5

save Syslog-ng daemon logs locally
by Hirbod Moriani 05 May '18

05 May '18

Hi everyone, I want to save log from syslog-ng daemon itself. For example "start a TLS session"/"end of a TLS session" or something like that should be logged. It's possible in syslog-ng CE? and how to config syslog-ng to save daemon logs locally?

2 1