November 2017 - syslog-ng - lists.balabit.hu

Hourly Log rotate
by craig bowser 20 Nov '17

20 Nov '17

So according to https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng- ose-guide-admin/html/example-logrotate.html using this format: destination d_sorted { file("/var/log/remote/${HOST}/${YEAR}_${MONTH}_${DAY}.log" create-dirs(yes)); }; I can create logs folders and files based on the timestamp. And this is working nicely for us... With a caveat. We are getting a TON of logs, so we want to rotate hourly and archive quickly. My config is this: destination d_msg { file("/var/log/message_${YEAR} _${MONTH}_${DAY}_${HOUR}.log"); }; But instead of creating one file per hour according to the time the event is received, it is creating files based on the timestamp of the event (which, while useful for discovering and tracking down machines with time synch problems, is not so useful for managing log files on the syslog server). Is there a way to make it create files based on the time the event is received and NOT the timestamp of the event? Thanks.

2 2

Cisco ACS logs truncated
by Scot 15 Nov '17

15 Nov '17

Hi, Has anyone worked with ACS logs and solved the message header limit ? We can get syslog working but as expected the message gets truncated. Local logs on the ACS have the entire payload. Thinking there may be a way to script a log fetch or something. Thanks Scot

4 8

(no subject)
by craig bowser 15 Nov '17

15 Nov '17

I installed syslog-ng on an offline CentOS 6.x machine by first downloading all required rpms using this command: yumdownloader --resolve syslog-ng There were two files needed, that did not download and I downloaded them separately using: yumdownloader eventlog-0.2.13-1.el6.x86_64.rpm and: yumdownloader libnet-1.1.6-7.el6.x86_64.rpm These are the files I moved to the offline server: 17,684 eventlog-0.2.13-1.el6.i686.rpm 17,804 eventlog-0.2.13-1.el6.x86_64.rpm 122,940 gamin-0.1.10-9.el6.i686.rpm 1,738,964 glib2-2.28.8-9.el6.i686.rpm 4,570,040 glibc-2.12-1.209.el6_9.2.i686.rpm 117,400 libgcc-4.4.7-18.el6.i686.rpm 60,464 libnet-1.1.6-7.el6.i686.rpm 59,196 libnet-1.1.6-7.el6.x86_64.rpm 111,688 libselinux-2.0.94-7.el6.i686.rpm 309,920 libstdc++-4.4.7-18.el6.i686.rpm 161,168 nss-softokn-freebl-3.14.3-23.3.el6_8.i686.rpm 201,096 pcre-7.8-7.el6.i686.rpm 452,548 syslog-ng-3.2.5-4.el6.i686.rpm 451,092 syslog-ng-3.2.5-4.el6.x86_64.rpm 63,536 tcp_wrappers-libs-7.6-58.el6.i686.rpm 74,284 zlib-1.2.3-29.el6.i686.rpm Once I moved all these files to the offline server into their own folder, I ran: rpm –ivh *.rpm to install. I then checked the config file, made a couple of tweaks, then started syslog-ng using: service syslog-ng start I got the following errors: Plugin module not found in 'module-path'; module-path='/lib64/syslog-ng', module='afsql' Plugin module not found in 'module-path'; module-path='/lib64/syslog-ng', module='afsql' [ OK ] Syslog-ng was now running, just with these two errors. A quick google pointed to the problem, syslog-ng is looking for the sql plugin and the solution: yum install syslog-ng-libdbi But when I tried to run yumdownloader syslog-ng-libdbi I got "Nothing to download" And when I tried yum install --downloadonly syslog-ng-libdbi I got "Nothing to do" So here are my questions: 1. Will syslog-ng run just fine receiving udp and sending udp with those errors and without that module? If, so, I don’t care if I don’t get that module. 2. Is it possible to run syslog-ng and disable that module in the run command? 3. Can I remove syslog-ng and re-install disable that module on install? 4. I’d rather not recompile if I don’t have to, but I found instructions for how to do that. Any tips would be appreciated. Thanks. Craig L Bowser ____________________________ This email is measured by size. Bits and bytes may have settled during transport.

2 2

Install Issue
by Elphick, Matthew Thomas 08 Nov '17

08 Nov '17

Hello, I am trying to install syslog-ng OSE on RHEL 7 but I get the following error. Transaction check error: file /etc/logrotate.d/syslog from install of syslog-ng-3.12.1-2.el7.centos.x86_64 conflicts with file from package rsyslog-8.24.0-12.el7.x86_64 I can't remove rsyslog from the system as other people us it. What is the best way to resolve this issue? Thanks for your help. Matthew Elphick, Network Engineering Marsh & McLennan Companies Global Technology Infrastructure (MGTI) | Enterprise Networking Services Tower Place, London, EC3R 5BU O +44 (0)20 73573667 | M +44 (0)7392 122399 | matthew.elphick(a)mmc.com<mailto:matthew.elphick@mmc.com> ________________________________ Marsh Ltd. Registered in England and Wales Number: 1507274 Registered office 1 Tower Place West, Tower Place, London, EC3R 5BU. Marsh Ltd is authorised and regulated by the Financial Conduct Authority. This message and any attachments are confidential. If you have received this message in error please delete it from your system. If you require any assistance please notify the sender. Thank you.

4 6

logstash to syslog-ng
by Czanik, Péter 08 Nov '17

08 Nov '17

Hi, In the past few weeks I have seen questions about sending logs from Logstash to syslog-ng, so I tried it myself and put together a blog with simple configurations to help you: https://www.balabit.com/blog/sending-logs-logstash-syslog-ng/ Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

3 3

any limitation with in-list filter
by Gopi Joshi 07 Nov '17

07 Nov '17

I am using in-list filter to match logs for specific users , these specific users are saved in a plain text file . is there any restriction or limitation on number of lines it can match in one file ? I currently have less than 10 users to compare with but this list is going to increase . Regards Gopi Joshi

3 2

Beat to syslog-ng
by Czanik, Péter 07 Nov '17

07 Nov '17

Hi, Somehow related to my previous e-mail: I was asked if Filebeat can send logs directly into syslog-ng. None of the network destinations of Beat are supported by syslog-ng, but I did a quick check and we can read and parse the file destination of Beat. Filebeat module functionality can be practically replaced by syslog-ng and one of its parsers. But there are quite a few other Beat applications. Question: is there a practical use case for syslog-ng reading the file output of a Beat module? Do you use it anywhere? Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

2 1

Question about using multiple amqp destination
by Alicia Smith 06 Nov '17

06 Nov '17

I am trying to configure syslog-ng to send multiple json formatted logs to a rabbitmq endpoint. The consumer for the logs checks for a particular field to determine the type of log it is. However, I can't seem to figure out how to send all the different logs as the field it checks is "source" that I add using --pair within the format-json template destination d_amqp_ssh { amqp( vhost("/") host("mozdefqa2.private.scl3.mozilla.com") port(5672) exchange("eventtask") exchange-type("direct") routing-key("eventtask") body("$(format-json --scope nv_pairs --pair category=\"bro\" --pair source=\"ssh\" --pair customendpoint=\" \" --pair tags=\"bro\")") persistent(no) username("guest") password("guest") ); }; So I added another destination with the name d_amqp_conn and it's source field value is "conn" and the amqp endpoint is the same host as the ssh log. But syslog-ng fails to start with this particular setup. It seems to not like having more than one amqp destination. How would something like this typically be solved? -- Alicia Smith @phrozyn Information Security Engineer asmith(a)mozilla.com

3 10

Syslog-NG Statistics
by Schoonover, Mark E HHHH 01 Nov '17

01 Nov '17

Hello, I'm working on a Splunk dashboard for our NG PE servers. Looking at the stats from the output of /opt/syslog-ng/sbin/syslog-ng-ctl stats, this one has me puzzled: src.program,,49587758,d,processed,1 I don't know what "program" means as well as the 3rd column. I understand the remaining columns. What do they mean? Thanks! Regards, Mark Schoonover - KA6WKE Infrastructure Engineering Manager, Splunk Architect ENE : Tools, Instrumentation and Common Services Team Office: 32.8697° N, 116.9711° W Phone : 770-261-7934 Email : mark.schoonover(a)cigna.com<mailto:mark.schoonover@cigna.com> HPSM Team: ENE NMS Engineering Confidential, unpublished property of CIGNA. Do not duplicate or distribute. Use and distribution limited solely to authorized personnel. ¬© Copyright 2017 CIGNA ------------------------------------------------------------------------------ CONFIDENTIALITY NOTICE: If you have received this email in error, please immediately notify the sender by e-mail at the address shown. This email transmission may contain confidential information. This information is intended only for the use of the individual(s) or entity to whom it is intended even if addressed incorrectly. Please delete it from your files if you are not the intended recipient. Thank you for your compliance. Copyright (c) 2017 Cigna ==============================================================================

2 1